The amount and quality of the data collected will determine the success of any research study. The greatest challenge in conducting any form of survey research is to get a high response rate.
Three Primary Types of Data Collection
face-to-face format
telephone interview
mail / email / Internet
Response Rate
Response rate (RR) = total completed questionnaires / eligible sample
Common (acceptable?) mail response rates are between 25% to 35% (Mitra & Lankford, 1999).
Example of Town of Parker Master Plan Survey Methodology and Response Rate
City utility customers (N=840) were used to develop the survey sample. | |
Business customers and duplicate entries were deleted from the survey list and only residential customers (N=525) were used. | |
A random sample of 230 addresses were drawn. | |
A stamped, pre-addressed envelope was included within the survey to facilitate survey responses. | |
Survey participants were asked to respond within five days. | |
A total of 56 usable surveys were returned. | |
The response rate was 24.3% (56/230). | |
This return rate is comparable to other Arizona city recreation surveys. |
Example of Whitewater Motivation Study Response Rate
Whitewater enthusiasts (N = 724) were identified from the initial list of 788, and were mailed a Letter of Introduction, as well as a Leisure Motivation Questionnaire. | |
Five days after the initial mailing a follow-up postcard was sent. | |
Of the 724 participants identified, |
36 questionnaires were returned as undeliverable by the postal service.
454 were returned for analyses.
16 were unusable (incomplete) leaving a total of 438 responses to be included in the analysis.
The response rate was 62% (438/724). |
Factors that Reduce Return Rate
incomplete questionnaires | |
incorrect addresses (mail surveys) | |
questionnaires returned after deadline | |
duplicate listings | |
ineligible individuals in sample |
Problems of Non-response
Low response rates introduce nonresponse error which is more difficult to estimate the the sampling error. Nonresponse creates a question of why these individuals did not reply. This also causes an image problem for the agency that is sponsoring the research.
Some researchers attempt to conduct a follow-up with a sample of nonrespondents to attempt to determine is any differences exist between the answers from respondents and nonrespondents. This would improve the validity of the study and improve it's generalizibility to the larger population.
Incentives to Increase Response Rates
Many researchers include some form of incentive to try to increase their response rate. It may be a one dollar bill included with the survey, a discount at a local business, or a drawing for a free prize.
Sample of Drawing in the Town of Parker Recreation Survey
SPECIAL DRAWING ENTRY FORM If you would like to have your name submitted for the Special Drawing for a free season swim pass, please print your name and phone number below. Name ______________________________________ Phone: _______________________ |
Modes of Data Collection
Face-to-face: Has a high response rate, up to 98%. It is also an expensive technique requiring training of interviewers and several call-backs to households. | |||
Telephone Interviews: Costs more than a mail survey but can be completely in a shorter time frame. Has a comparable response rate to face-to-face interviews but a better response rate than mail surveys. Problems associated with phone samples:
Random Digit Dialing (RDD) Two-digit RDD is the most popular form of RDD. In this case, the phone book is first used to select a phone number; this is followed by randomization of the last two digits by replacing them with a two-digit string of random numbers. For example, if the selected phone number is 123-4567, then the randomized number could be 123-4578, where "67" has been replaced by "78." This process increases the probability of finding a working number at the randomized location, since the number uses not only the prefix, but also the first two digits of an existing phone number, which makes it more likely that there is another working line in that bank of numbers. In other words, if 123-4567 is a working number, then 123-4578 is much more likely also to be a working number than 123-3489. Two-digit RDD thus reduces the likelihood of reaching nonworking numbers and increases the possibility of reaching numbers that are not included in the phone book. The biggest drawback with the RDD process is the fact that it leads to the dialing of nonworking and nonresidential numbers instead of households. |
|||
Mail - Self Response: A common form of survey research.
It requires a carefully developed data collection technique to have a
adequate response rate. These techniques include:
|
|||
|
Internet: is growing in popularity as a data collection tool. It
raises questions about validity which can be addressed in the research
design. Online techniques include:
|
Respondent Honesty in Survey Responses
face-to-face interviews = less honest answers | |
phone interviews = moderately honest answers | |
mail surveys = most honest answers |
Multi-Methods
Many survey methodologies will combine several different data collection techniques to get the best response rate. Combinations may include:
mailing with a phone follow-up | |
mailing with a face-to-face interview | |
phone with an Internet site reply |
Copyright 2001. Northern Arizona University, ALL RIGHTS RESERVED