Module 7: Data Collection - Chapter 8

On-line Lesson

The amount and quality of the data collected will determine the success of any research study. The greatest challenge in conducting any form of survey research is to get a high response rate.

Three Primary Types of Data Collection

  1. face-to-face format

  2. telephone interview

  3. mail / email / Internet 

Response Rate

Response rate (RR) = total completed questionnaires / eligible sample

Common (acceptable?) mail response rates are between 25% to 35% (Mitra & Lankford, 1999).

Example of Town of Parker Master Plan Survey Methodology and Response Rate

bullet

City utility customers (N=840) were used to develop the survey sample. 

bullet

Business customers and duplicate entries were deleted from the survey list and only residential customers (N=525) were used. 

bullet

A random sample of 230 addresses were drawn. 

bullet

A stamped, pre-addressed envelope was included within the survey to facilitate survey responses. 

bullet

Survey participants were asked to respond within five days. 

bullet

A total of 56 usable surveys were returned.

bullet

The response rate was 24.3% (56/230). 

bullet

This return rate is comparable to other Arizona city recreation surveys. 

Example of Whitewater Motivation Study Response Rate

bullet

Whitewater enthusiasts (N = 724) were identified from the initial list of 788, and were mailed a Letter of Introduction, as well as a Leisure Motivation Questionnaire. 

bullet

Five days after the initial mailing a follow-up postcard was sent. 

bullet

Of the 724 participants identified,

  1. 36 questionnaires were returned as undeliverable by the postal service.

  2. 454 were returned for analyses. 

  3. 16 were unusable (incomplete) leaving a total of 438 responses to be included in the analysis.

bullet

The response rate was 62% (438/724).

Factors that Reduce Return Rate

bullet

incomplete questionnaires

bullet

incorrect addresses (mail surveys)

bullet

questionnaires returned after deadline

bullet

duplicate listings

bullet

ineligible individuals in sample

Problems of Non-response

Low response rates introduce nonresponse error which is more difficult to estimate the the sampling error. Nonresponse creates a question of why these individuals did not reply. This also causes an image problem for the agency that is sponsoring the research.

Some researchers attempt to conduct a follow-up with a sample of nonrespondents to attempt to determine is any differences exist between the answers from respondents and nonrespondents. This would improve the validity of the study and improve it's generalizibility to the larger population.

Incentives to Increase Response Rates

Many researchers include some form of incentive to try to increase their response rate. It may be a one dollar bill included with the survey, a discount at a local business, or a drawing for a free prize.

Sample of Drawing in the Town of Parker Recreation Survey

SPECIAL DRAWING ENTRY FORM 

If you would like to have your name submitted for the Special Drawing for a free season swim pass, please print your name and phone number below. 

Name ______________________________________ 

Phone: _______________________ 

Thank you for your time and interest in the Parks and Recreation Department's efforts to improve our community’s recreation facilities and services. 

 

Modes of Data Collection

Face-to-face: Has a high response rate, up to 98%. It is also an expensive technique requiring training of interviewers and several call-backs to households.

Telephone Interviews: Costs more than a mail survey but can be completely in a shorter time frame. Has a comparable response rate to face-to-face interviews but a better response rate than mail surveys. Problems associated with phone samples:

  1. not everyone has a telephone, and some people prefer not to be listed in the phone directory.

  2. not a current source of information, it is only printed once a year.

Random Digit Dialing (RDD)      

Two-digit RDD is the most popular form of RDD. In this case, the phone book is first used to select a phone number; this is followed by randomization of the last two digits by replacing them with a two-digit string of random numbers. For example, if the selected phone number is 123-4567, then the randomized number could be 123-4578, where "67" has been replaced by "78." This process increases the probability of finding a working number at the randomized location, since the number uses not only the prefix, but also the first two digits of an existing phone number, which makes it more likely that there is another working line in that bank of numbers. In other words, if 123-4567 is a working number, then 123-4578 is much more likely also to be a working number than 123-3489. Two-digit RDD thus reduces the likelihood of reaching nonworking numbers and increases the possibility of reaching numbers that are not included in the phone book.

The biggest drawback with the RDD process is the fact that it leads to the dialing of nonworking and nonresidential numbers instead of households.

Mail - Self Response: A common form of survey research. It requires a carefully developed data collection technique to have a adequate response rate. These techniques include:
 
  1. use first class personalized mailing
  2. have a well written cover letter
  3. have a simple and concise questionnaire
  4. include a postage-paid, self addressed return envelope.
  5. mailed by a date to reach all recipients at least a week before the return deadline.
  6. send a follow-up letter or postcard, two weeks after initial mailing.

Internet: is growing in popularity as a data collection tool. It raises questions about validity which can be addressed in the research design. Online techniques include:
 
  1. An introductory Email
  2. An online link for the survey instrument
  3. Follow-up emails requesting participation
     
bulletSample online survey emails
 

Respondent Honesty in Survey Responses

bullet

face-to-face interviews = less honest answers

bullet

phone interviews = moderately honest answers

bullet

mail surveys = most honest answers

Multi-Methods

Many survey methodologies will combine several different data collection techniques to get the best response rate. Combinations may include:

bullet

mailing with a phone follow-up

bullet

mailing with a face-to-face interview

bullet

phone with an Internet site reply

[Class] [Module 7]

Copyright 2001. Northern Arizona University, ALL RIGHTS RESERVED