Getting Started with MASDER Surveys

MASDER maintains a family of six instruments that, when administered jointly, allow us to cohesively understand the undergraduate student learning experience in introductory statistics and data science courses.

• The student instruments measure student attitudes about learning statistics or data science.
• The instructor instruments measure instructor attitudes about teaching statistics or data science.
• The environment inventory collects information on the course and/or section learning environment.

SOMA stands for Survey of Motivational Attitudes and EPIC stands for Environment Pedagogy Institution Course. The S or DS after SOMA or EPIC indicates the subject of the survey. Therefore, S-SOMAS stands for Survey of Motivational Attitudes toward Statistics. I-SOMAS stands for Instructor Survey of Motivational Attitudes toward teaching Statistics.

The student surveys are meant to be administered to undergraduate students taking an introductory statistics (S-SOMAS) or introductory data science (S-SOMADS) course, respectively.

They can be administered in courses designed to be similar to these. The I-SOMAS/DS surveys are meant to be taken by the instructors of these courses, respectively.

Lastly, EPIC-S/DS should be completed by the instructor for each course in which the student survey was administered.

The students surveys are designed to be perfectly matching, aside from one block of items. The matching structure is intentional to facilitate pre/post comparisons on identical items, and to allow the survey to be administered longitudinally with no awkward language.

One block of items about Teacher-Student Relationships appears on the Post but not on the Pre. This is because students are not able to evaluate their relationship with their instructor before the course has started.

The EPIC-S/DS Post inventories contain a parallel set of items so that the instructor and student perspectives on their relationship can be compared.

EPIC-S/DS Pre is a brief inventory that collects basic information that is not dependent on what happens across the course of the term. For example, it includes questions on the prerequisites, course textbook, modality, meeting times (if synchronous), and more. Because EPIC Pre is not focused on teaching practices, it can be completed by a course coordinator or supervisor if needed. (EPIC Pre can be completed the the MASDER website holder, even if the person is not teaching the course.)

EPIC-S/DS Post includes items that ask an instructor to reflect on how they taught during the term. It includes questions on high-impact teaching practices, statistics - or data science-specific pedagogy, and course assessments. Because EPIC Post does focus on teaching practices, it should only be completed by the course instructor. (If the MASDER account holder created the survey instance on behalf of other instructors, they should send the EPIC post link to the actual course instructors for them to complete.)

Step 1: Start by creating a course survey on the My Courses page. Enter a new course, following the instructions on the page. This step is also where you will indicate if the course is a Statistics or Data Science course, which will determine the type of surveys associated with the course. Entering accurate Course Information is crucial! This is defining course information is passed into each survey, providing linkage between student, instructor, and environment. If you make a mistake, you can edit the course information by clicking the course Prefix/Number in the resulting table.

*Teaching a multi-section course? You can enter multiple sections within one Course Instance! See the My Courses page for details.

Step 2: Customize your MASDER surveys, if you want to. From the My Courses page, click the link to Customize Surveys. From here, you can customize either the Student surveys or the Environment inventories. This includes adding your own consent form, removing optional items, and adding your own custom items. See details below for how you can customize surveys.

Step 3: Complete EPIC Pre. Navigate to the Environment Inventories page to find your EPIC links. Click the link for the Pre and enter the relevant information. When you are done, the page will show that you completed the inventory.

Step 4: Administer the S-SOMAS/DS survey to your students. From the Student Surveys page, not that a unique set of links is created for each section of your course. Clicking on the link will automatically copy it to your clipboard. Be sure that each section receive the correct link. We recommend administering the Pre survey within the first week of class, and the Post within the final week.

Step 5: Complete the EPIC post.

*Multi-section courses: In contrast to the Student surveys, only one EPIC Post link is created for each course instance. EPIC Post contains a looping feature that will ask questions about each section of a multi-section course.

Step 6: I-SOMAS/DS can be completed at any time via the Instructor Surveys page. Currently, account holders are not able to download this data. However, the data is triangulated with the corresponding course and student data when incorporated into the national sample. Additional functionality around creating instructor survey instances and downloading instructor data is forthcoming.

The survey links that are generated are specific to your course. They should not be shared with others who are interested in administering MASDER surveys. Survey links are tied to the MASDER website account holder's name.

The only survey links that may end up sharing is the EPIC Post, IF you as the account holder are, managing data collection for a larger group of instructors. For example, a course coordinator who generated links on behalf of all faculty many send the EPIC post link to the corresponding instructors. EPIC Post, will ask instructors to select the relevant sections of the course that they teach. See more details below under Generating Surveys on Behalf of Others.

When customizing surveys, you will see that some survey blocks are required and some are optional. Required survey blocks cannot be edited. This includes all attitudes constructs, some crucial demographic items, and more. Additionally, student first name, last name, and email are required for matching purposes. Optional blocks of items are shaded gray and allow the removal of items one at a time.

No, we do not currently allow the editing or removal of attitudes items and/or constructs. Attitudes items are intentionally included in a randomized block to eliminate any order effect. Accurately studying differences between attitudes constructs requires constructs to be intermingled and randomly ordered. Additionally, surveys are validated at the construct level, making it inappropriate to remove or edit individual items from constructs.

Questions can be added to S-SOMAS/DS, EPIC-S/DS Pre, and EPIC-S/DS Post. Questions can be added at the end of survey blocks (throughout the surveys) as well as at the end of the survey. Additionally, a custom Consent question can be added at the start of the survey. See Considerations for Human Subjects Research (IRB) for details.

Currently, functionality allows you to add Consent, Text Entry questions or Multiple Choice questions. We plan to add additional functionality in the future.

Data download feature is coming soon. Note that data should only be downloaded for person use or quality improvement purposes, unless your data collection is approved by an IRB. See " Considerations for Human Subjects Research (IRB)" for more information.

Yes. As the MASDER account holder, you will retain access to all information collected from the student survey. Since MASDER requires the collection of student first name, last name and email address for matching purposes, this information will be included in your data download.

Instructor reports, comparing your students to the national distributions and averages, will be available online soon.

This feature is currently not available, but is under consideration for future website development. If you have IRB permission to access this level of data, please contact MASDER.

If you are using the data for your own personal used, you likely do not need IRB approval. However, it is the responsibility of each account owner to determine whether it is needed or not. You can use the MASDER website for data collection regardless of whether your project is considered Human Subjects Research (HSR).

AS the MASDER account holder, you are responsible for determining if your project meets the criteria for HSR and therefore needs IRB approval. You should either have IRB approval, or you should have verification that your project is not Human Subjects Research (HSR).

If your project is HSR, you are responsible for gaining approval through your authorizing institution and ensuring you have followed appropriate protocols regarding consent, etc. MASDER does not take responsibility for your IRB approval or check the status of your IRB compliance.

Yes! You can add a custom consent question to the beginning of your MASDER survey. This options allows you to enter your own consent language and edit two multiple choice response options for either giving or declining consent. If you use this option, please note that survey respondents who decline to consent will not be shown the survey and will be redirected to the final Thank You page.

1. MASDER survey data is not anonymous. On student surveys, re require collection of student first name and last name, and email address. These fields allow us to match pre/post survey responses, and return identifiable data to instructors to use to assign credit for survey completion, etc. Therefore, on your IRB protocol, make sure to say the data are identifiable, not anonymous.
    

   We ask for the student information before showing consent forms. This means that even if a student declines to consent and does not complete the survey, your data will have a record of the student name and email address whether or not they consented.

2. In you IRB protocol, make note that your survey data will be administered and downloaded via the MASDER website. You can cite the benefits of using this website, as mentioned at the top of this Getting Started page.

Yes. The MASDER project and website have been declared Exempt by the Villanova University IRB, and prior data was collected and managed under the California State University Monterey Bar IRB. If you need documentation of the approved IRB status, please contact MASDER. This IRB protocol approves the MASDER team for website development, data collection, cleaning, and reporting.

Each survey administered through the MASDER website has a MASDER consent form at the beginning. This form asks respondents to consent to their data being used as part of the MASDER research project. Consenting to MASDER allows us to include data in our national sample, which appears in comparison reports for each class. It also allows the data to be included in our de-identified national database and allows the MASDER teach to study the data. If a respondent doe not consent to MASDER research, they are still able to complete each survey, but their data will be excluded from the national sample in reports, the national database, and further analysis. In summary, whether or not a respondent consents to MASDER does not affect their ability to complete any surveys; it simply determines if their data will be included in the broader MASDER research study.

If you are teaching multiple sections of a course, in the same way, you only need to create the course once on the website. When entering the course information, add as many sections as needed. The EPIC Pre will only need to be completed by the account holder (course coordinator) and it will loop through all the sections identified when creating the course.

The EPIC Post will need to be completed by each instructor associated with the sections added. The instructor will be able to select the sections they need to enter information about.

If you are teaching the same course but in different ways, you will need to enter a course for EACH on the website. For example, STAT 110 - online and STAT 110 - face-to-face. Student survey links and EPIC links will get generated for each type of course.

The situation may arise where you need to generate survey links on behalf of someone else - perhaps you are a course coordinator or conducting professional development. You, as the account holder will create the course on the website making sure to include all the sections of the course being offered.

Then, you will copy and distribute the student survey links to the instructors of the course; making sure the correct link gets sent to each instructor is important as they are tied to the section of the course.

The EPIC Pre will be filled out by you as all the information required is based on course-specific information. At the end of the term, you will want to send the EPIC Post survey link to each instructor for them to complete about their specific section(s) of the course; some of the information collected pertains to teacher-student relationships and specific teaching practices.