So YOU want to survey UVA students...here's how:
IMPORTANT NOTE: Requests for access to student data for the purpose of conducting a survey need to be submitted electronically through the Office of the Vice President and Chief Student Affairs Officer (VPCSAO). IRA works with VPCSAO to review and approve submissions related specifically to proposed survey research and if approved, to provide the requested data.
Ten Steps to Successful Surveys…
1. Ask yourself: Do you really need to conduct a survey?
The information you want may already be available.
On behalf of the University, the Office of Institutional Assessment and Studies regularly administers large surveys to students at various times throughout their academic career. The survey questionnaires and the results are available on the IRA Survey Results page for your review. Please consider whether this archived data may fit your needs. If you have questions about how available data can address your research questions, please contact Sarah Schultz Robinson (email@example.com).
A survey may not be the best technique to get the information you want. Especially if you are conducting exploratory research or wish to gather direct, in-depth information, then focus groups or structured interviews may be a better fit for your research questions.
2. Develop a reasonable timeline
Give yourself time – four to six months – to develop a quality questionnaire AND have your survey reviewed and approved by the Office of Student Affairs and IRA.
Draft a survey that respondents can complete in 15 minutes or less. Write survey instructions and the invitation to take the survey—be consistent and clear. Pilot test your survey on a small group from the population of interest before launching it more widely. This will help you to identify instructions or questions that are confusing or that don’t elicit good information.
The Associate Vice President for Student Affairs will need to review your survey before you will be allowed to contact UVA students. The form to complete to request this review as well as access to student data can be found at https://vpsa.virginia.edu/forms.
You will also need to have your survey administration schedule reviewed by IAS to make sure that there are no potential conflicts between your survey schedule and an existing survey schedule. This is an important step in minimizing respondent survey fatigue and helps all survey researchers increase response rates.
3. Define your research questions
This is a critical part of the research process that can be overlooked when a researcher gets too involved with the details of the project. Step back and make sure that you know exactly what you are trying to measure. Vet your research questions with colleagues, advisors, relevant decision-makers, and other professionals in the field. Extensively review the literature on your topic so you know how other professionals have (or have not) framed their research questions. Write down your research questions and put them in a conspicuous place throughout your research process.
Before you begin your survey, develop a plan for analyzing the data and reporting the results. How will you use the data you collect? With whom will your results be shared? In what format will results be shared – as visual presentations, written or electronic reports?
4. Design your survey instrument
The content of the survey stems directly from your research questions. Still, much care is needed to develop survey questions that will effectively and efficiently elicit the information you want. Ideas for specific survey questions can come from existing instruments, colleagues, members of the target population (collected via focus groups or interviews), and your own observations. It is important to balance adequate coverage of your research questions (comprehensiveness) with conciseness. Avoid the temptation to include questions that may provide interesting but not particularly useful results. Also, consider whether some of the data you want is available through other sources such as institutional files (see #1 above).
Surveys should begin with a statement that clearly explains:
- The purpose of the survey
- That participation in the survey is voluntary
- That respondent can skip questions he/she would prefer not to answer
- Whether responses provided will be treated as anonymous or confidential data
- How information from the survey will be reported and used
When composing survey questions, here are some general guidelines to bear in mind:
- Survey methodologists specialize in the construction of survey questions and their response categories. Consider having someone with survey design expertise review your survey instrument.
- Response categories should reflect a comprehensive array of choices, including “not applicable,” “don’t know,” and/or “other,” where appropriate
- Limit the use of open-ended questions; as much as possible, position these at the end of the survey instrument
- Short surveys generate more responses and minimize the imposition on the valuable resource of our students' time
Survey questions should not be “leading” or contain jargon or technical terms that may not be understood by all respondents
Pretest your survey instrument with a subset of your target population. This provides a critical test of the clarity, comprehensiveness, and length of your survey.
5. Review by the Institutional Review Board (IRB) may be necessary
If your survey can be considered “research” it needs to be reviewed by the Institutional Review Board. Research is defined by federal regulations as “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.”
If you plan to publish your results or present them beyond UVA, you will need to contact IRB for review.
If you are unsure whether your study meets the requirements for review, please contact the IRB before completing and submitting a protocol form to the board.
6. Select the sample
It is not necessary to survey an entire population in order to have valid, generalizable results. A random sample will do the job while minimizing costs—including the costs of survey fatigue. If you need help determining an appropriate sample size for your project, please contact Sarah Schultz Robinson (firstname.lastname@example.org) for assistance. Texts on survey research design provide guidelines for setting sample sizes; sample size calculators are also available on the Internet. If your survey is not for research purposes (i.e., is a class project) you may have to accept a larger margin of error than is ideal (or suggested by sample size calculators) and use a smaller sample size. Again, this helps to control widespread survey fatigue.
IAS can provide your sample (that is, contact information for a random sample of students from your population of interest), but only after your survey has been approved by the Office of Student Affairs (see #2 above.)
7. Consider incentives
Many surveys offer respondents the chance to be entered into a raffle drawing for prizes. Gift cards and personal electronic devices are common prizes. The literature on survey methodology suggests that these kinds of incentives have a modest impact, increasing the response rate slightly. Financial incentives are not necessary, however; students may respond to a survey because they think the topic is important or the results will be put to good use.
Surveys that offer incentives to respondents must track respondent identities in some way so that “prizes” can be awarded. If identifying information (such as names, e-mail addresses, or student identification numbers) are kept with the survey responses and confidentiality is promised to respondents, the study needs some security protocol for keeping the data safe (see next item).
8. Collecting data: data security
Survey respondents need to be told if their responses will be anonymous, kept confidential, or are entirely non-confidential.
Anonymous data do not include names, addresses, student identification numbers, or any other personal information that would make it possible to associate a response with any given individual.
Data that are confidential contain information that may identify an individual respondent. There are significant advantages to collecting identifiers, including the ability to do “pre and post” studies through linked data files. However, files containing individual identifiers must be stored with considerable attention to data security and access.
9. Analyzing and reporting the results
At a minimum, most reports of survey results provide a full set of frequencies (e.g., “how many/what percentage of students answered “yes” to question #1?”). Cross-tabulations of responses (e.g., “how do men and women compare in their responses to question #1?”) across subsets of respondents are also useful. For cross-tabulations especially, care must be taken to protect individuals’ privacy; as a general rule, do not report results for categories containing five or fewer respondents. Consider how you might share your results with others on Grounds who are interested in related questions. Remember that survey results cannot be presented or published beyond UVA without IRB approval.
10. Contact us if you want help
We are committed to promoting excellence in surveying at UVA and we will be happy to help you. For more information about conducting surveys at UVA, please contact Sarah Schultz Robinson at (434) 982-2321 or email email@example.com.
Much of the content of this page was developed by the Office of Institutional Research and Planning at Cornell University, and is being used with permission.