Practical Research: Planning and Design Twelfth Edition Chapter 6 Descriptive Research Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Descriptive Research Designs • Observation studies • Correlational research • Developmental design • Survey research Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Quantitative Observation Studies • Involve humans, other animals, plants, nonliving objects • Focus is limited, pre-specified • Quantify behavior • Require planning, attention to detail, and time • Provide a quantitative alternative to qualitative approaches Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Maintaining Objectivity in Observation Studies (1 of 2) • Define the behavior precisely and concretely – Should be easily recognized • Divide the observation period into small segments – Record whether the behavior does or does not occur Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Maintaining Objectivity in Observation Studies (2 of 2) • Use a rating scale to evaluate the behavior in terms of specific dimensions – Have people rate the same behavior independently • Train the raters to use specific criteria until consistent ratings are obtained Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Correlational Research (1 of 2) • Examines the extent to which differences in one variable are related to differences in other variables • Researchers gather data about two or more characteristics for a particular group to see if these characteristics are interrelated Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Correlational Research (2 of 2) • Scatter plots show the overall pattern and describe the interrelationship • Correlation does not, in and of itself, indicate causation Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Developmental Designs • Cross-sectional study – people from several different groups are sampled and compared • Longitudinal study – a single group of people is followed over time, and data are collected at various times Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Pros and Cons: Cross-Sectional V s. Longitudinal Study (1 of 2) ersu • Cross-sectional studies: – Pro ▪ All the data can be collected at one time – Con ▪ Different populations may represent different life experiences (threat to internal validity) Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Pros and Cons: Cross-Sectional V s. Longitudinal Study (2 of 2) ersu • Longitudinal studies: – Pro ▪ Correlations between characteristics at different times can be computed – Con ▪ Participants may be lost to follow-up ▪ Characteristic being measured may change because participants have experience with the instrument Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Compromise: Cohort-Sequential Design • Addresses weaknesses of longitudinal and crosssectional designs • Includes two or more age groups (the cross-sectional piece), followed over a period of time (the longitudinal piece) • Allows calculation of correlations between measures taken at two different time periods • Predictions can be made across time Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Experience-Sampling Methods (ESM) • An experience-sampling method (ESM) is an approach in which a researcher collects frequent and ongoing data about people as they live their normal, everyday lives • Successfully used in both quantitative and qualitative projects • Advantages of ESM methods: – Potential for increased accuracy and validity of assessments – Researcher gains data that might be useful in determining testretest reliability. – Useful if the researcher wants to collect longitudinal data as a means of investigating any short-term changes in characteristic as environmental or behavioral variables change Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Survey Research • Goal is to learn about a large population by surveying a sample of that population • Also called a descriptive survey or normative survey • Simple design – researcher poses a series of questions, quantifies responses, and draws inferences about a population • Captures a fleeting moment of time — extrapolation can be made about a longer period of time Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Types of Survey Research • Interview – Structured or semi-structured – Face-to-face, telephone, video conference – High response rate • Questionnaire – Paper-and-pencil or computerized – Low return rate – Assurance of remaining anonymous Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Data Collection (1 of 2) • Checklist: a list of behaviors, characteristics, or other entities under investigation • Limited information: observed or not observed • Rating scale: used to evaluate a behavior, attitude etc. on a continuum (“never” to “always”) Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Data Collection (2 of 2) • May be ordinal or interval scale – People may not interpret scale the same way • Rubric: two or more rating scales, with concrete descriptions of behavior for each scale point – Scales may not address the same things Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Computerizing Observations • Use a computer to record what you see • Use a spreadsheet to organize the data • Consider software specific to your purpose Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Conducting Interviews in a Quantitative Study (1 of 2) 1. Ask questions that help answer the research question 2. Write questions with quantifiable answers (numerical codes) 3. Restrict questions to a single idea 4. Consider asking a few questions to elicit qualitative data 5. Use a computer to streamline the process Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Conducting Interviews in a Quantitative Study (2 of 2) 6. Conduct pilot test(s) 7. Introduce yourself and explain the purpose of the study 8. Be sure participants offer informed consent in writing 9. Ask controversial questions in the latter part of the interview 10.Seek clarifying information as needed Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Constructing a Questionnaire (1 of 3) 1. Keep it short 2. Keep the respondent’s task simple 3. Provide specific instructions 4. Use simple, clear, unambiguous language Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Constructing a Questionnaire (2 of 3) 5. Give a rationale for any item for which the purpose is unclear 6. Check for unwarranted assumptions implicit in the question 7. Word your questions in ways that don’t give clues about preferred or more desirable responses 8. Determine in advance how you will code the responses Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Constructing a Questionnaire (3 of 3) 9. Check for consistency 10.Conduct one or more pilot tests to determine the validity of your questionnaire 11.Scrutinize the almost-final product to make sure it addresses your needs 12.Make the questionnaire attractive and professional looking Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Using Technology When Administering Questionnaires 1. Ask participants in the same location to answer directly on a laptop or tablet 2. When participants are at diverse locations, use email to request participation and obtain responses 3. If you use paper mail delivery, us a word-processing program to personalize your correspondence 4. Use a scanner to facilitate data tabulation 5. Use a computer database to keep track of who has responded and who has not Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Maximizing the Return Rate (1 of 2) 1. Consider the timing – Avoid holidays and vacation times 2. Make a good first impression 3. Motivate potential respondents – Write a great cover letter – Include a self-addressed envelope with prepaid postage – Offer to send the results of your study 4. If mailing your questionnaire, include a self-addressed envelope with return package. Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Maximizing the Return Rate (2 of 2) 5. Offer the results of your study 6. Be gently persistent – Consider sending two follow-up reminders – Send reminders a week or two after the previous mailing Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Choosing a Sample in a Descriptive Study • Any researcher who conducts a descriptive study wants to determine the nature of how things are… • It isn’t possible to survey an entire population of interest, so researchers select a subset, or sample, of the population. • A good sample is representative of the population. • This is what quantitative researchers refer to as external validity, of the extent to which the study’s findings can be reasonably generalized beyond the partcipants in the study. Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Probability Sampling • Probability sampling: researcher specifies in advance that each segment of the population is represented in the sample – Requires random selection Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Random Selection • Each member of the population has an equal chance of being selected • Characteristics of the sample are assumed to approximate the characteristics of the total population • Tables of random numbers or computer programs are used to select from a list of the population Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Probability Sampling Techniques (1 of 4) • Simple random sampling: Researcher numbers everyone in the population and then uses random number generator to select participants • Stratified random sampling: Researcher identifies strata — different groups in population — and samples equally from each one – Example: 10 students in each grade Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Probability Sampling Techniques (2 of 4) • Proportional stratified sampling: Researcher identifies strata and samples from each one based on its proportion in the population – Example: ▪ population: 100 first graders, 200 second graders ▪ sample: 10 first graders, 20 second graders Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Probability Sampling Techniques (3 of 4) • Cluster sampling: Researcher subdivides a large area into smaller units (clusters), selects a subset of clusters, and then selects individuals randomly from each identified cluster • Example: – Population = all students in a district with 1200 schools – Clusters = townships within the district Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Probability Sampling Techniques (4 of 4) • Systematic sampling: researcher selects individuals/clusters according to predetermined sequence, which must originate by chance • Example: – Scramble the list of people randomly – Then pick every nth person Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Nonprobability Sampling • Nonprobability sampling: researcher cannot guarantee that each element of the population will be represented in the sample – Some members of the population have little or no chance of being sampled Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Nonprobability Sampling Techniques (1 of 3) • Convenience sampling (accidental sampling) – Researcher takes samples that are readily available. ▪ Example: People who arrive at the store for breakfast Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Nonprobability Sampling Techniques (2 of 3) • Quota sampling: – Researcher conveniently selects participants in the same proportion that they are found in the general population, but not in a random fashion ▪ Example population: 100 first graders, 200 second graders ▪ Example sample: the first 10 first graders and the first 20 second graders who arrive at school that day Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Nonprobability Sampling Techniques (3 of 3) • Purposive sampling: Researcher choose participants for a particular purpose – People from voting districts that, in the past, have been helpful in predicting the election outcome – The researcher must always provide a rationale explaining the selection of a particular sample Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Surveys of Very Large Populations (1 of 2) • Multistage sampling: – Divide country into primary areas, randomly select areas to sample – Divide the primary areas into sample locations, randomly select locations to sample – Divide sample locations into chunks, randomly select chunks to sample Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Surveys of Very Large Populations (2 of 2) • Multistage sampling: – Divide chunks into segments, randomly select segments to sample – Divide segments into units, randomly select units to sample Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Identifying a Sufficient Sample Size (1 of 2) • Basic rule: The larger the sample, the better • For smaller populations (N=100 or fewer), survey the entire population • If population is around 500, sample 50% • If population is around 1,500, sample 20% Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Identifying a Sufficient Sample Size (2 of 2) • If population is over 5,000, a sample size of 400 is fine • The larger the population, the smaller the percentage • Need a representative sample Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Sources of Bias (1 of 4) • Bias – any influence, condition, or set of conditions that distort the data • Researchers should try to avoid bias, but acknowledge that it occurs Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Sources of Bias (2 of 4) • Sampling Bias – occurs when any factor(s) leads to a nonrepresentative sample of the population • Examples: – Selecting from phone book (no land line?) – Using an online survey (no Internet?) – Mailing questionnaires (low or selective response rate?) Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Strategies for Identifying Sampling Bias • Look for items that may be influenced by factors that distinguish respondents from nonrespondents. – interests, education level, age, etc. • Compare responses that were returned quickly with those that were returned later. – late responses often look like what you’d expect from nonrespondents • Randomly select a small number of nonrespondents and try to contact them. Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Sources of Bias (3 of 4) • Instrumentation bias – measurement instruments slant the results – questions lead to particular answers • Response bias – participants say what they think researcher wants to hear – participants want to create favorable impression (social desirability effect) Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Sources of Bias (4 of 4) • Researcher bias – researchers have a point of view – researchers choose what they want to study – researchers make subjective interpretations Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Interpreting the Data • Don’t forget — it’s “descriptive” research, but you still have to interpret the data • Two basic principles of research: 1. The purpose of research is to seek the answer to a problem or question in light of data that relate to the problem or question. 2. Although collecting data for study and organizing it for inspection require care and precision, extracting meaning from the data is paramount and should never be neglected. Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Questions to Ask Yourself (1 of 2) 1. Why is a description of this population and/or phenomenon valuable? 2. What specific data will you need to address your research problem and its subproblems? 3. What procedures should you use to obtain the needed information? How can you best implement those procedures? 4. How can you get a sample that will reasonably represent the overall population about which you are concerned? Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Questions to Ask Yourself (2 of 2) 5. How can you control for possible bias in your collection of data? 6. What will you do with the data once you have collected them? How might you best organize and prepare them for analysis? 7. Above all, in what ways might you reasonably interpret your data? What conclusions might you reach from your investigation? Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Copyright Copyright © 2019, 2016, 2013 Pearson Education, Inc. All Rights Reserved Homework 6: Descriptive Research Instructions Q1. The authors of your textbook suggest that sampling bias is virtually unavoidable and that it is important to disclose and discuss possible sources of bias in the study report. Do you agree? Explain your position. What are the types of bias? Give examples related to IA discipline (cybersecurity related examples). 
Purchase answer to see full attachment