Effective research relies on high quality data. We look at many factors when evaluating data quality:
Two properties of a sample affect the kinds of conclusions that can be safely drawn from it: the sample’s size, and the degree to which it is representative of the larger group you are trying to understand.
With instruments like surveys, the more responses we have, the more sophisticated the analyses can be. In order to run statistical analyses on survey responses, we would probably need a minimum of 200 responses, and if we wanted to compare two groups within a company, we would probably need an even larger sample.
Companies can take measures to increase participation rates:
- Explaining to employees why survey results are important
- Making participants eligible for a prize draw
- Compensating participants directly, with a gift certificate or something else
Some questions are hard for participants to answer accurately, even if they are trying their best. If the company is interested in team performance or individual innovation or something similar, then a supervisor or other team leader will need to answer some questions. We would then match those answers to the right team or employee.
Surveys and other studies can be performed once (cross-sectional) or multiple times at set intervals (longitudinal). Longitudinal surveys are a good tool to explore what kind of effect company interventions or changes are having on workers.
With surveys, finding a good length is a question of balance. The more questions we ask, the clearer our understanding will be, but overly-long surveys may be a problem for participants. Most surveys take at least 20 minutes to fill. For longer surveys, we sometimes recommend dividing the survey in half, and asking the most important questions first and the second half a few weeks later.
For surveys, people are more likely to respond, and to respond honestly, if they are confident that their answers will remain confidential. We can improve responses by letting employees know that their surveys will be stored securely, and that their managers will have never have access to individual answers, but only to overall findings.