SRO collaborates with researchers on the development and testing of survey questionnaires, working to minimize potential sources of survey error and ensure validity, reliability, and cultural appropriateness. SRO has extensive experience in creating and developing a variety of instruments in collaboration with research teams. SRO’s focus is to prepare a data collection instrument in which measures meet the scientific priorities of validity and reliability. We will work with you to ensure the measures are culturally-appropriate and comparable across subgroups of the population, employing an iterative process of translation, adaptation, testing, and revision as needed. SRO offers a variety of options for developing questionnaires.
Expert Review
Whether collaborating on survey content or starting with a draft survey instrument, SRO will provide methodological review of study questionnaires and provide feedback on question format, question wording, response options, potential context effects, mode effects, or any other potential sources of error in the instrument. This review is centered on the principles of the survey-response process and Total Survey Error (TSE) framework. It is intended to minimize potential errors related to comprehension, recall, judgement, or response, and also takes into account design considerations related to mode, optimization of surveys for mobile devices, and broader constructs such as asking sensitive questions and social desirability in response.
Focus Groups and Cognitive Interviews
In the earliest stages of questionnaire development, SRO may conduct focus groups with members of the target population to identify the language people use to talk about a topic, to further our understanding of complex concepts, and to identify the dimensions of a construct that need to be measured. Focus groups can inform the creation of new survey items or the modification of existing items to better capture the intended meaning. Cognitive interviews are typically administered one-on-one and they are designed to identify potential sources of response error in survey questions. The interviewer uses a combination of structured and semi-structured probes to collection information about how the respondent is interpreting or responding to selected questionnaire items.
As pretest tools, focus groups and cognitive interviews can be valuable and efficient means to inform question wording, item construction, and overall questionnaire structure. By uncovering different interpretations of words or phrases, these tools help us to ensure questions are understood as intended, leading to more valid and reliable data. SRO will review and analyze transcripts, report on the findings, and make recommendations for reducing potential sources of survey error.
Pilot Testing and Pretests
SRO can pilot or pretest an existing survey instrument with a convenience sample, a list sample, or an existing representative sample. Pilot tests often include the entire project protocol (survey administration plus any additional tasks) and are also a test of our data collection technical systems. In addition to receiving actual data from completed surveys, SRO’s experienced interviewers can provide qualitative feedback on the interviewing experience.
Translation
SRO has a dedicated, experienced translation team with a focus on Spanish-language translation into survey-friendly language. Languages other than Spanish are handled by an experienced social science translation partner, facilitated by SRO.
Questionnaire Specification
SRO will translate questionnaires into programming specifications and work with the research team as needed to identify variable naming conventions and code frames to facilitate production of data files for analysis. The specifications prescribe the full functionality of the computer-assisted questionnaire application including question text versus response code frames, question format and display on the screen, logic and flow such as skip patterns and calculated checkpoints, and response data range and consistency checks.
Would you like someone to review your survey questionnaire?
If you have a draft questionnaire you would like reviewed, methodologists on the SRO team will provide feedback and suggestions on question wording, context effects, mode effects or other potential sources of response error in your survey. We will provide design recommendations and help you identify and reduce potential sources of error related to respondent comprehension, recall, judgement, and communication. SRO will provide 2-3 hours of design consultation at no charge to faculty, staff, and students who are members of the U-M community using surveys in their work.
Submit a request form for questionnaire design consultation.