Module 2: Methods of Data Collection - Chapters 2 

On-line Lesson

Leisure Research Methods

Once a research question has been determined the next step is to identify which method will be appropriate and effective.

The table below describes the basic characteristics of different methodologies.

Qualitative and Quantitative Research Methodologies

Quantitative research methods include:

Qualitative research methods include:

Each research method has it's strengths and weaknesses. When designing a research study it is important to decide what the outcome (data) the study will produce then select the best methodology to produce that desired information. 

Data Collection Techniques

There are two sources of data. Primary data collection uses surveys, experiments or direct observations. Secondary data collection may be conducted by collecting information from a diverse source of documents or electronically stored information. U.S. census and market studies are examples of a common sources of secondary data. This is also referred to as "data mining." 

Key Data Collection Techniques

Writing an Introduction

In any research proposal the researcher should avoid the word "investigation." This word is perceived in a negative sense. 

The key components of a good introduction include 

1. a description of the purpose of the study, 

2. identification of any sponsoring agency, 

3. a statement regarding confidentiality, 

4. a description of how sample or respondents were selected, and 

5. an explanation of the results and their applications.

Experimental Treatments

Experimental designs are the basis of statistical significance. An example of the fundamentals of an experimental design is shown below.

Experimental Control

Experimental control is associated with four primary factors (Huck, Cormier, & Bounds, 1974).

1. The random assignment of individual subjects to comparison groups;

2. The extent to which the independent variable can be manipulated by the researcher;

3. The time when the observations or measurements of the dependent variable occur; and

4. Which groups are measured and how. 

Treatment Group: The portion of a sample or population that is exposed to a manipulation of the independent variable is known as the treatment group. For example, youth who enroll and participate in recreation programs are the treatment group, and the group to which no recreation services are provided constitutes the control group.

Validity Issues

There are two primary criteria for evaluating the validity of an experimental design. 

Confounding Errors

Errors: are conditions that may confuse the effect of the independent variable with that of some other variable(s).

1. Premeasurement and interaction errors

2. Maturation errors

3. History errors

4. Instrumentation errors

5. Selection bias errors

6. Mortality errors

EXPERIMENTAL DESIGNS

1. True Designs

2. Quasi Designs

3. Ex Post Facto Designs

True Designs - Five Basic Steps to Experimental Research Design

1. Survey the literature for current research related to your study.
2. Define the problem, formulate a hypothesis, define basic terms and variables, and operationalize variables.
3. Develop a research plan:
     a. Identify confounding/mediating variables that may contaminate the experiment, and develop methods to control or minimize them.
     b. Select a research design (see Chapter 3).
     c. Randomly select subjects and randomly assign them to groups.
     d. Validate all instruments used.
     e. Develop data collection procedures, conduct a pilot study, and refine the instrument.
     f. State the null and alternative hypotheses and set the statistical significance level of the study.
4. Conduct the research experiment(s).
5. Analyze all data, conduct appropriate statistical tests and report results.

Quasi Designs

The primary difference between true designs and quasi designs is that quasi designs do not use random assignment into treatment or control groups since this design is used in existing naturally occurring settings.

Groups are given pretests, then one group is given a treatment and then both groups are given a post-test. This creates a continuous question of internal and external validity, since the subjects are self-selected. The steps used in a quasi design are the same as true designs.

Ex Post Facto Designs

An ex post facto design will determine which variables discriminate between subject groups.

Steps in an Ex Post Facto Design

1. Formulate the research problem including identification of factors that may influence dependent variable(s).

2. Identify alternate hypotheses that may explain the relationships.

3. Identify and select subject groups.

4. Collect and analyze data

Ex post facto studies cannot prove causation, but may provide insight into understanding of phenomenon.

OTHER FIELD METHODS/GROUP TECHNIQUES

Nominal Group Technique (NGT)

The NGT is a group discussion structuring technique. It is useful for providing a focused effort on topics. The NGT provides a method to identify issues of concern to special interest groups or the public at large. Ewert (1990) noted that the NGT is a collective decision-making technique for use in park and recreation planning and management. The NGT is used to obtain insight into group issues, behaviors and future research needs.

Five Steps of the NGT 

1. Members of the group identify their individual ideas in writing, without any group discussion;

2. Each member lists his/her own ideas and then rank-orders them, again without any group discussion;

3. A facilitator gives each participant an opportunity to state his/her ideas ( one item per person at a time, in round-robin fashion) until all ideas are exhausted; 

4. As a group, participants discuss and consolidate ideas into a list; and 

5. Finally, members vote to select priority ideas. The final list of ideas becomes the focus of further research and discussion. These ideas can also be used to generate a work plan for a formal strategic planning process, a basis for a survey or interview, or the development of a scale.

Source: (Mitra & Lankford, 1999)

Delphi Method

The delphi method was developed to structure discussions and summarize options from a selected group to:

Although the data may prove to be valuable, the collection process is very time consuming. When time is available and respondents are willing to be queried over a period of time, the technique can be very powerful in identifying trends and predicting future events.

The technique requires a series of questionnaires and feedback reports to a group of individuals. Each series is analyzed and the instrument/statements are revised to reflect the responses of the group. A new questionnaire is prepared that includes the new material, and the process is repeated until a consensus is reached. 

The reading below is a research study that used the delphi technique and content analysis to develop a national professional certification program.

Sample Research Article: Job Competency Analyses of Entry-Level Resort and Commercial Recreation Professionals

Focus Groups

Richard Krueger (1988), describe the focus group as a special type of group in terms of purpose, size, composition, and procedures. A focus group is typically composed of seven to twelve participants who are unfamiliar with each other and conducted by a trained interviewer. These participants are selected because they have certain characteristics in common that relate to the topic of the focus group.

The researcher creates a permissive environment in the focus group that nurtures different perceptions and points of view, without pressuring participants to vote, plan, or reach consensus. The group discussion is conducted several times with similar types of participants to identify trends and patterns in perceptions. Careful and systematic analysis of the discussions provide clues and insights as to how a product, service, or opportunity is perceived.

A focus group can be defined as a carefully planned discussion designed to obtain perceptions on a defined area of interest in a permissive, nonthreatening environment. It is conducted with approximately seven to twelve people by a skilled interviewer. The discussion is relaxed, comfortable, and often enjoyable for participants as they share their ideas and perceptions. Group members influence each other by responding to ideas and comments in the discussion.

CHARACTERISTICS OF FOCUS GROUPS

Focus group interviews typically have four characteristics: 

1. Identify the target market (people who possess certain characteristics);

2. Provide a short introduction and background on the issue to be discussed;

3. Have focus group members write their responses to the issue(s);

4. Facilitate group discussion;

5. Provide a summary of the focus group issues at the end of the meeting.

Other types of group processes used in human services (delphic, nominal, planning, therapeutic, sensitivity, or advisory) may have one or more of these features, but not in the same combination as those of focus group interviews.

Behavior/Cognitive Mapping

Cognitive and spatial mapping information provides a spatial map of:

All types of recreation activities and travel involve some level of environmental cognition because people must identify and locate recreation destinations and attractions.

Cognitive mapping allows recreation resource managers the opportunity to identify where users and visitors perceive the best recreation areas are located. It is important to understand user perceptions in order to manage intensive use areas in terms of maintenance, supervision, budgeting, policy development and planning. 

Cognitive maps grid the research site into zones. The zones identify existing geographic, climatic, landscape, marine resources, and recreation sites. The grids allow respondents to indicate primary recreation sites, and then a  composite is developed to identify high impact areas. Researchers collect data at recreation areas (beach, campground, marina, trailhead, etc.) by interviewing visitors and recreationists. During the data collection process, random sites, days, times, and respondents (every nth) should be chosen to increase the reliability and generalizability of the data.

Observations

Observational research is used for studying nonverbal behaviors (gestures, activities, social groupings, etc.).

Sommer & Sommer (1986) developed the list shown below to assist in observation research.

Casual observation is normally done like unstructured interviews. During the early stages of a research project, casual observation allows the researcher(s) to observe subjects prior to designing questionnaires and/or interview formats. 

Types of Observation Studies

Documents (also called Secondary Data or Data Mining)

Data mining is commonly used in both qualitative and quantitative research. Secondary data provides data which provides a framework for the research project, development of research question(s), and validation of study findings. 

Frequently used sources of secondary data are:

Content Analysis

Content analysis systematically describes the form or content of written and/or spoken material. It is used to quantitatively studying mass media. The technique uses secondary data and is considered unobtrusive research.

The first step is to select the media to be studied and the research topic. Then develop a classification system to record the information. The techniques can use trained judges or a computer program can be used to sort the data to increase the reliability of the process.

Content analysis is a tedious process due to the requirement that each data source be analyzed along a number of dimensions. It may also be inductive (identifies themes and patterns) or deductive (quantifies frequencies of data). The results are descriptive, but will also indicate trends or issues of interest.

The reading below is a research study that used the delphi technique and content analysis to develop a national professional certification program.

Sample Research Article: Job Competency Analyses of Entry-Level Resort and Commercial Recreation Professionals

Meta-Analysis

Meta-analysis combines the results of studies being reviewed. It utilizes statistical techniques to estimate the strength of a given set of findings across many different studies. This allows the creation of a context from which future research can emerge and determine the reliability of a finding by examining results from many different studies. Researchers analyze the methods used in previous studies, and collectively quantify the findings of the studies. Meta-analysis findings form a basis for establishing new theories, models and concepts.

Thomas and Nelson (1990) detail the steps to meta-analysis:

1. Identification of the research problem.

2. Conduct of a literature review of identified studies to determine inclusion or exclusion.

3. A careful reading and evaluation to identify and code important study characteristics.

4. Calculation of effect size. Effect size is the mean of the experimental group minus the mean of the control group, divided by the standard deviation of the control group. The notion is to calculate the effect size across a number of studies to determine the relevance of the test, treatment, or method.

5. Reporting of the findings and conclusions.

Historical Research

Historical research is also referred to as analytical research. Common methodological characteristics include a research topic that addresses past events, review of primary and secondary data, techniques of criticism for historical searches and evaluation of the information, and synthesis and explanation of findings. Historical studies attempt to provide information and understanding of past historical, legal, and policy events.

Five basic procedures common to the conduct of historical research were identified by McMillan & Schumacher (1984). They provide a systematic approach to the process of historical research.

Step 1: Define the problem, asking pertinent questions such as: Is the historical method appropriate? Are pertinent data available ? Will the findings be significant in the leisure services field?

Step 2: Develop the research hypothesis (if necessary) and research objectives to provide a framework for the conduct of the research. Research questions focus on events (who, what, when, where), how an event occurred (descriptive), and why the event happened (interpretive ). This contrasts with quantitative studies, in which the researcher is testing hypotheses and trying to determine the significance between scores for experimental and control groups or the relationships between variable x and variable y.

Step 3: Collect the data, which consists of taking copious notes and organizing the data. The researcher should code topics and subtopics in order to arrange and file the data. The kinds of data analysis employed in historical research include (based on McMillan & Schumacher, 1984):

1. Analysis of concepts. Concepts are clarified by de- scribing the essential and core concepts beginning from the early developmental stages. Clarification allows other researchers to explore the topic in other fashions.

2. Editing or compilation of documents, to preserve documents in chronological order to explain events. For ex- ample, an edition of Butler's park standards, the National Recreation and Park Association's first minutes, or letters from early pioneers in the field preserves the documents for future researchers.

3. Descriptive narration tells the story from beginning to end in chronological order, utilizing limited generalizations and synthesized facts.

4. Interpretive analysis relates one event to another event. The event is studied and described within a broader con- text to add meaning and credibility to the data. For example, an examination of the development of a local jurisdiction's ability to dedicate land for parks may be related to the urbanization and loss of open space in our communities.

5. Comparative analysis examines similarities and differences in events during different time periods-for example, the budget-cutting priorities and procedures of the Proposition 13 era of the early 1980s in parks and recreation as compared to the budget-cutting priorities and procedures of today.

6. Theoretical and philosophical analysis utilizes historical parallels, past trends, and sequences of events to suggest the past, present, and future of the topic being researched. Findings would be used to develop a theory or philosophy of leisure. For example, an analysis of public recreation agency goals and objectives of previous eras can be used to describe the future in the context of social, political, economic, technological, and cultural changes in society.

Step 4: Utilizing external and internal criticism, the re- search should evaluate the data. Sources of data include documents (letters, diaries, bills, receipts, newspapers, journals/magazines, films, pictures, recordings, personal and institutional records, and budgets), oral testimonies of participants in the events, and relics ( textbooks, buildings, maps, equipment, furniture, and other objects).

Step 5: Reporting of the findings, which includes a statement of the problem, review of source material, assumptions, research questions and methods used to obtain findings, the interpretations and conclusions, and a thorough bibliographic referencing system.

Multimethod Approach

The multimethod approach encourages collecting, analyzing and integrating data from several sources and the use of a variety of different types of research methods.

[Class] [Mod 2]

Copyright 2001. Northern Arizona University, ALL RIGHTS RESERVED