Running Head: LITERATURE REVIEW 1 Literature Review: Comparing and Contrasting Articles Name Institutional Affiliation LITERATURE REVIEW 2 In a well-researched qualitative study conducted by Schildkraut & Hernandez (2014) to examine the effectiveness of strict gun control legislations, it was concluded that gun control legislation in the United States is not effective in preventing shootings in schools. The study conducted by Schildkraut & Hernandez (2014) is in contrast with the study conducted by Spitzer (2014). According to Spitzer (2014), strict gun laws on ownership are essential because it helps to minimize public safety and health policy goals that were designed to minimize the unwarranted ownership of the guns. Spitzer (2014) believes that strict gun laws minimize illegal possession hence resulting in effective eradication of careless shootings. Both Schildkraut & Hernandez (2014) and Spitzer (2014) agree to the fact that the topic of strict gun policies should be approached carefully and with caution to avoid infringement of human rights of others. Even though Spitzer (2014) argues that enactment of strict gun control laws helps in curbing illegal possession, he shares similar sentiments with Schildkraut & Hernandez (2014) that gun legislation should be enacted to ease the minds of the public and further research should be conducted to ascertain the perceptions and attitudes of the public towards strict guns laws. By doing so, policymakers will be in a position to understand its reliability and effectiveness. In a study conducted by Luca, Malhotra, and Poliquin (2017) to examine the impacts of handgun waiting on the rates of gun violence, it was concluded that increasing the waiting periods for guns to be delivered to those who purchase helps to reduce the cases of gun violence in the society. The findings of this study are in agreements with the findings of a study conducted by Smith-Walter et al (2016). According to Smith-Walter et al (2016), restricting the delivery periods of a gun purchased helps to minimize the gun violence in the society. Smith-Walter et al LITERATURE REVIEW 3 state that NRA and Brandy Campaign should be centered on sensitizing the gun sellers to increase the delivery period. In a well-researched qualitative study conducted by Taxman (2016) to examine whether the tri-vector models help to reduce the gun violence in the United States, it was concluded that tri-vector offers potential solutions to minimizing gun shootings. According to Taxman (2016), tri-vector models help one to apprehend and understand the complexities revolving the mental and psychological conditions of the shooters in American. Taxman recommends that there is a need for the development of programs that increase emotional attachment, empathy, and improve the mental satisfaction and happiness of individuals past shooters. By doing so, the chances of past shooters committing such crimes in the future are minimal. Similar to Taxman (2016) findings that being mentally and psychologically attached to past gun shooters help to reduce future gun violent acts, the study conducted by Hickey (2013) revealed that limiting people with mental illness to access guns is important because it minimizes unconscious shootings that happen because of mental illness. According to Hickey (2013), a mentally ill person should first be counseled and encouraged to adopt with normal life before being given with gun license. Counseling of mentally ill persons is relatively important because it changes their mental perspective towards gun violence. Taxman (2016) and Hickey (2013) shares one sentiment that mental aspects of an individual contribute to gun violence, therefore, gun sellers should first try to understand the past mental conditions a person before giving a license to own a gun. Wu (2018) conducted a qualitative study by to examine whether cueing and framing has an impact on youths’ attitudes towards gun rights and gun control. Wu concluded that majority LITERATURE REVIEW 4 of the youths and especially female do not believe that arming and training staff help to protect schools from gun shootings. The students who are politically inclined with Republic affiliations believe that arming of school guards help to reduce gun violence whereas those who subscribe to Democrats ideologies believe that arming and training of staff escalate the situations. Similar to Wu (2018) findings, the study conducted by Zgonjanin (2014) to examine the feasibility of the NRA's proposed actions that support the arming and training of school personnel in order to counter school shooting incidents revealed that arming of school personnel is not effective because it affects qualified immunity to such personnel. According to Zgonjanin, arming personnel to quell gun violence complicates the immunity regarding perils committed in the course of duty. Zgonjanin states that arming of personnel exposes them to liability for actions committed by third parties despite the fact that the duty of the personnel is beyond the scope of their official duties. In a well-researched qualitative study conducted by Yamane (2014) to examine the sociological facts surrounding the US gun culture, it was found that misinformed stigma has been placed on the ownership of the guns. According to Yamane (2014), most of the people do not own a gun in order to commit crime or shootings but rather they own guns for leisure and others own it for game hunting. Yamane states that gun ownership should not only be seen in crime shootings perspective but rather it should be viewed in the angle of participation in nature. According to Yamane (2014), "gun culture 2.0" should not only be seen in the lenses of criminology perspective and owners' desire to inflict pain on others but should be seen from people's desires to leisure, prestige, and other sociological perspectives. Yamane believes that changing the people's mindset on the ownership of gun requires demystification of the wide-held LITERATURE REVIEW 5 notions and misconceptions that people in possession of the guns only intends to commit the crimes. Yamane states that public health reasons for gun controls are totally skewed and only designed to make people hate gun ownership. The gun controls are skewed because it does not consider the social desire for ownership. In a qualitative research conducted by Rajan et al (2016) examine whether the funding towards gun violence prevention has any impacts on safety and health of Americans, it was found that amendments such as Dickey Amendments that supports the reduction of funds for gun research prevention are contributing to an increase in gun violence. Rajan et al (2016) argue that the Congress should remain active and pass laws that support increasing of funding channeled towards gun research prevention. According to Rajan et al, the current state of gun violence prevention research is understudied and underfunded. The authors believe that underfunding is a drawback in ending gun violence because it denies an opportunity for human security personnel to mount and sustain the security surveillance. In addition, underfunding and understudy denies an opportunity for security personnel to pinpoint causes and potential solutions to gun violence. LITERATURE REVIEW 6 References Hickey, J. D. (2013). Gun Prohibitions For People With Mental Illness—What Should the Policy Be?. DEV. MENTAL HEALTH L., 32, 1-2. Luca, M., Malhotra, D., &Poliquin, C. (2017). Handgun waiting periods reduce gun deaths. Proceedings of the National Academy of Sciences, 201619896. Rajan, S., Branas, C. C., Hargarten, S., &Allegrante, J. P. (2018). Funding for Gun Violence Research Is Key to the Health and Safety of the Nation. American Journal Of Public Health, 108(2), 194-195. doi:10.2105/AJPH.2017.304235 Schildkraut, J., & Hernandez, T. (2014). Laws That Bit The Bullet: A Review of Legislative Responses to School Shootings. American Journal Of Criminal Justice, 39(2), 358-374. doi:10.1007/s12103-013-9214-6 Smith-Walter, A., Peterson, H. L., Jones, M. D., & Nicole Reynolds Marshall, A. (2016). Gun Stories: How Evidence Shapes Firearm Policy in the United States. Politics & Policy, 44(6), 1053-1088. doi:10.1111/polp.12187 Spitzer, R. J. (2014). New York State and the New York SAFE Act: a case study in strict gun laws. Alb. L. Rev., 78, 749. Taxman, J., (2016). Gun Violence in America- A Tri-Vector Model. J. Appl. Psychoanal. Studies 2016, 13: 113-123. Wu, S., (2018). The Effects of Cueing and Framing on Youth Attitudes towards Gun Control and Gun rights. Social Sciences, 72(2), 29. Published 15th, February, 2018. LITERATURE REVIEW 7 Yamane, D. (n.d). The sociology of U.S. gun culture. Sociology Compass, 11(7), Zgonjanin, M. (2014). When Victims become Responsible: Deputizing School Personnel and Destruction of Qualified Immunity [notes]. Journal Of Law & Education, (3), 455. Research Design: •Experimental Design •Non-Experimental Design Research Design • A research design is a plan of action for executing a research project: – Plan for how to test hypothesis – Usually involves the following: • When you need to collect data: Before and after comparison – You take a measurement – then modify something – then take another measurement – These are called pre and post tests • How many groups do you need to compare – Usually: Side by side comparison – There are 2 groups, but only one receives the treatment – These are called treatment and control groups – Ultimately the research design a guide to collecting, analyzing, and interpreting results – You must have observable data • Original or secondary Research Design • The choice of research design is affected by: – The purpose of the research • Exploratory (one-shot case study) Prediction (aggregate random sample data) – Resources like time, money, and skill or ethical concerns • Should you experiment with prison conditions? – Randomly assign people to max or min security? • Should you experiment with effects of syphilis/HIV? • Every research design has positive and negative attributes – Talking about internal and external validity here – Tradeoffs • You should match a design with your requirements – Not one size fits all Research Design • All research designs attempt to: – Establish a causal relationship between two or more variables (internal validity) • The different designs do this with varying degrees of success – Demonstrate that the results are generally true (External Validity) • Generalization – Eliminate as many alternative explanations as possible • Biggest problem with being able to make causal statements • We want variation to be a result of IV and only IV Research Design • The goal of scientific research is to find causal relationships. Causal relationships meet three characteristics: • Causal is more than correlation – One event leads to or produces a change in something else – This is the gold standard and it can be difficult to achieve 1. Covariation: the alleged cause varies with the supposed effect. 2. Time order: the cause precedes the effect in time. 3. Elimination of alternative explanations to isolate causation to one factor. • The best research designs incorporate all three New Twist on Old Themes • Validity – Formerly: accurate measure of core idea (UCR measure of rape) – Now: Internal Validity • Manipulation of X is what is really causing Y • Controlled experiments are great at this – Now: External Validity • Generalizable – Classical Experiments are great but lack this » Conditions are often times artificial. » Experimental conditions do not mirror reality. – Do laboratory conditions hold in real world? » Example: Gibson and Judicial Advertisements » Example: Sommers and Juror race characteristics Terms • Experimental or Treatment Group – Receives IV/Treatment • Control Group – Does not receive IV/Treatment • Independent variable = X – Treatment, shock, intervention • Dependent Variable = O • Pre-test: measure of DV before IV • Post-test: measure of DV after IV Experimental Design • The classical randomized experiment has five basic characteristics: 1. 2. 3. 4. 5. At least one experimental group that will have exposure to the treatment and one control group that will not. Randomly assigned individuals to each group, avoiding selfselection. Controlled administration of the treatment, including the circumstances under which the experimental group is exposed. Measurement of a dependent variable before and after the treatment with a pre-test and post-test. Any difference between the pre and post-tests can be attributed to the experimental effect of exposure to the treatment. Controls the environment of the experiment (time, location, and other physical aspects). Experimental Design • Experimental research designs are especially good for isolating causal factors. – Excellent internal validity • Experimentation allows a researcher to make causal inferences with great confidence in the design through control over exposure to an experimental treatment. • Experiments often suffer from weak external validity. Experimental Design • We will discuss the following experimental designs: • Classic • Post-Test • Repeated Measurement • Multiple group • Field experiments Classic Experiment • R O(t) X O(t) • R O(c) O(c) • Intention to vote – negative ad – intention to vote • Intention to vote – neutral ad – intention to vote • High Internal Validity – Lower External Validity Experimental Design • Post-test – Shares the characteristics of the classical randomized experiment except that no pre-test is used because the sample is truly random and sufficiently large that one can assume that the control and experimental group(s) are equivalent. – R X O(t) – R O(c) – Negative ad – intention to vote – Neutral ad – intention to vote – High Internal Validity (lower than classic) and lower external validity • Random assignment improves chance that groups are similar, but without that pretest you can’t be quite as sure. Experimental Design • Repeated-measurement/Interrupted Time Series – Adds additional pre-tests, post-tests, or both in an effort to measure longer-term effects of experimental treatments to the classic example. – R OOOOOOOOOOXOOOOOOOOOOO – R OOOOOOOOOOOOOOOOOOOOOO – Juvenile crime – Scared Straight – juvenile crime – Juvenile crime – No Scared Straight – juvenile crime Experimental Design • Multiple group – Is a modification of the classic example where more than one experimental group is created to compare the effects of different treatments. – R OX1O – R OX2O – R OX3O – R OX4O Experimental Design • Field experiment – Study performed outside laboratory – An experiment in a natural setting in which the investigator may or may not have control over group membership but does have control over one or more independent variables. – Effects of having a criminal record on employment • Matched individuals were identical except one applicant had a criminal record • Outcome measure is whether or not applicant was called back • Those with criminal records were half as likely to be called back – Minneapolis Domestic Violence Experiment – Randomly select residents to receive negative ads and then measure who voted • Random sample of 1,400 FL residents in mayoral election • Treatment group received negative ads • Official voting records was the DV Quasi-Experimental Design • Nonexperimental designs are characterized by at least one of the following: – Presence of a single group – Lack of control over the assignment of subjects to groups – Lack of control over the application of the independent variable – Inability to measure the dependent variable before and after exposure to the independent variable occurs Quasi-Experimental Designs • • • • • • Case Studies Cross section designs Longitudinal designs Trend analysis Panel studies Intervention analyses Quasi-Experimental Design • Case Studies • Similar to Field Study but researcher does not control shock – Shock is observed by researcher • Foot patrol v. Car patrol – Compare 2 high crime areas – Police policies control shock • New welfare programs – State and federal government control the shock • Individual mandate before ACA (compare TX to MA) – State legislators and governors control shock Quasi-Experimental Design • One shot case study – – – – X–0 No Pretest and no comparison groups No random assignment We do something and make inferences based on outcome measures – This is a very weak design but it is used all the time – Example: Teacher gives instruction and we assume it causes students to learn – Example: Officer sensitivity training – excessive force claims Quasi-Experimental Design • One-Group pretest-posttest – OXO – Lacks comparison group and random assignment – Improvement over one shot case study because we know if something changed from time 1 to time 2 • But we still can’t isolate the independent variable as being the main cause – Example: Test on day one – student takes class – final exam • Final exam should be better – Example: Anti-gang program in school. Measure violence at beginning and end of year. – Example: Excessive force claims – training – excessive force claims Quasi-Experimental Design • Static Group Comparison • X-0 –0 • Addition of second group but still no random assignment – We don’t know how different groups are before experiment • Teacher teaching 2 of same classes uses new technique in one • Lack of pretest raises possibility that the 2 groups are different to begin with so it is not only the IV that causes change in DV Quasi-Experimental Design • Cross Sectional designs (survey, aggregate analysis) – Characterized by measurements of the independent and dependent variables at approximately the same time. – No control over assignment or administration of independent variable – Data analysis, rather than a treatment, is necessary for making causal inferences. – Many subjects – one point in time – Low internal validity but higher external – Survey – did you see negative ad? » Did you vote? Quasi-Experimental Design • Longitudinal design – Allow for the measurement of variables at different points in time. – Can model change across time, examine the time order of a causal relationship, and estimate age, cohort, and period affects. – Time Series – Panel and Cohort Quasi-Experimental Design • Panel studies – Cross-sectional designs that include a time element. – The design relies on measurement of the same units of analysis at different points in time— creating waves of data for analysis over time. – Example: • Analysis of DARE programs – use same students at same schools during elementary school Quasi-Experimental Design • Intervention analysis • Measurements of a dependent variable both before and after the introduction of an independent variable that is observed but not controlled by the researcher. • Ex: Presidential Debates • OOOOOOOOXOOOOOOOOO Quasi-Experimental Design • Non-experimental designs are generally characterized as having less internal reliability but better external validity than experimental designs. • There is always a tradeoff when moving from one design to another. Threats to Internal Validity • Primary Question: Are changes in the independent variable indeed responsible for the observed variation in the dependent variable • This is a problem that mainly impacts longitudinal (long-term) studies • High internal validity means we have evidence of a strong causal relationship – X causes Y • Think of these as alternative reasons for a change in the DV – Things other than X that change Y • The following things lower internal validity because they are all reasons we might see a change in the DV that is not attributable to the IV Threats to Internal Validity • History (group) – Large scale event that influences most or all of the participants in your study – Unanticipated event occurs during course of study – Changes in the real world – Example: New anti-depression drug: Pretest: Administered August 2001 Posttest: Administered October 2001: Result – people became more depressed – Example: Pretest: % of underage sale of alcohol – Intervention (police issue citation) – Posttest: % of underage sales • Threat: highly publicized death of person who was underage Threats to Internal Validity • Maturation (individual) – Events that affect the individual – Got sick, bored, death in family, lost insurance, had a baby, lost job during the course of the study – Example: New anti-depression drug: Pretest – treatment and someone lost job – posttest – Example: New anti-depression drug: Pretest – treatment and someone gets large inheritance – posttest Threats to Internal Validity • Mortality – Subjects drop of out study for variety of reasons – This become problematic if the students that leave are systematically different than those that remain • Example: DARE evaluations from middle school through high school • Students move, drop out, refuse to participate etc. • This is a problem if those that are no longer in the study are different from those who remain • Those that drop out may be the ones most likely to not be affected by DARE which makes program more effective Threats to Internal Validity • Instrumentation – Hawthorn Effect • People know they are being studied and alter their behavior • Sometimes the pretest will make people aware of what you are studying • Implementation – Researcher unintentionally influences respondents • Usually not deliberate • Researches may give subtle cues as to how they want person to respond (tone of voice, facial expressions, etc) • Double-blind Summary • Experimental vs. Quasi-Experimental • Must carefully specify design in order to make conclusions • Validity and Reliability The Survey: Asking People Questions Chapter 6 Introduction to Political Research Critical Assumption • Goal of Survey: People reveal true preferences – People pick choice that is closest to their true attitude • Assumption: People have crystallized opinions – Based on the rational-choice model of behavior – Well thought out, defined, fixed, and actual opinions • They have weighed the costs and benefits of each issue and have selected alternative that provides them with maximum benefit. – People know about topics, issues, subjects asked about – People have single peaked preferences • They have one and only one choice that gives them maximum utility Introduction to Political Research Critical Assumption • If this is true we should see consistent answers: – When people are asked same question at different times: • Only about 50% give the same answer with repeated questioning • In open ended questions – 50%-75% give conflicting views on the same exact policy – They see good and bad in welfare, food stamps, ACA, foreign policy, etc. • People should not be swayed by variations in word choice – Welfare v. financial assistance to low wage earner • 20% more likely to support assistance to low wage earners – ACA v. Obamacare • 48% favor Obamacare vs. 70% favoring ACA – Terminating pregnancy v. abortion Introduction to Political Research Critical Assumption • People won’t be affected by question order – Religion first then abortion • 15%-20% less likely to offer support – If you ask about life of mother first • Support increases for overall ability to have abortion – Soviet reporters before US reporters • 37% in favor if asked before allowing US • 73% in favor if asked about Soviet reporters after – Policy questions after knowledge questions • More likely to decline to answer Introduction to Political Research Question Wording Effects • Obamacare vs. Affordable Care Act Introduction to Political Research Zaller and Feldman • 2 Primary theories – On-line Theory • We have crystalized opinions • Opinions are stable but not completely static – Constantly updating our opinions with new information • Have consistent view though maybe not detailed • Example: • People have consistent views on Voter ID laws though they may not know the rules of their own state or the recent decisions regarding the legality of the laws Introduction to Political Research Zaller and Feldman • Doorstep Theory – Formulate opinions off the top of our heads • People have conflicting opinions • Answer based on what is most salient to them at the time – We have a mix of emotions that are made salient by what is immediately present to us • Leads to response instability and different answers at different times – People are not rational actors – What really matters when I’m asked to offer an opinion (survey, voting booth) I base that decision on what is most relevant and salient at the time. – Opinions are unstable and constantly fluctuating Introduction to Political Research Social Surveys • Surveys are some of the most widely used datagathering technique in the social sciences and other fields – But expensive • Available option is omnibus survey • In a social survey, a large number of people (called respondents) hear and give an answer to the exact same questions – They may answer many questions about past behaviors, experiences, opinions and characteristics – Respondents are revealing their true opinions to interviewers Introduction to Political Research Types of Information Gained from Surveys • Attitudes/Beliefs/Opinions – Govt shutdown – Obamacare – Contraception mandate – Presidential Approval – Elections – Building a Wall – Jailing people that get abortions – Senate and Supreme Court nomination Introduction to Political Research Types of Information Gained from Surveys • Individual Behavior – People • Drug use, where get information (TV, newspapers, etc), healthcare, smoking, drinking, voting • Characteristics of Individuals – People on food stamps, those receiving welfare, those who have been incarcerated, unemployment, victims of crime Introduction to Political Research Surveys and Variables • You can use one survey to measure many variables and test multiple hypotheses at the same time • Survey Questions are variables – Use answers to test hypotheses • Ask about demographic characteristics and presidential vote • Ask about adequacy of care coverage and support for Obamacare • Which is why it is critical you know what information you want before you introduce survey into field – If I don’t ask the question, I can’t test the hypothesis • I may want to know if respondent has son or daughter who is gay and how that impacts support for same-sex marriage Introduction to Political Research Steps in Conducting a Survey Introduction to Political Research Conducting a Survey • Start Up Stage: In this stage, you address the following three questions, – Who will be the respondents of your survey? • Random sample of people, potential voters, people in halfway houses, students, etc… – What information do you want to learn from them? and • Becomes the basis for your questions (based on your hypotheses) – Type of and design of survey? • If self-administered: Questionnaire = fixed collection of questions in a social survey that respondents answer. – Mail, Online surveys – Physical layout should be clear and easy to follow • If you have interviewers: Interview schedule = a questionnaire specifically designed for an interviewer asking respondents the questions. – Telephone or personal interviews Introduction to Political Research Conducting a Survey • Execution stage – Locate the sampled respondents • Call them, drive to house, etc. – Provide respondents with information about the survey and instructions on how to complete it. – Record information • Data Analysis Stage – This stage will be discussed in later chapters Introduction to Political Research Principles of Good Question Writing • Avoid Confusion (be clear!) – Do you go to doctor regularly? – What is your Income? • Avoid jargon, slang and abbreviations – DOMA, Polarized, ABA, ACA • Avoid emotional language (neutral language) – Socialist Obamacare – Doctor who murders babies vs. abortion doctor • Avoid Prestige bias (prestige bias – associating a statement with a prestigious person or group) Introduction to Political Research Good Question Writing • Avoid double-barreled questions – Support marriage and civil unions? – Support lifetime caps and keeping children on insurance? • Avoid leading questions – Bush/McCain example (2000) • Would you be more likely or less likely to vote for John McCain if you knew that he fathered an illegitimate black child? • Don’t you think it is your patriotic duty to not burn the flag? • Avoid overlapping or unbalance response categories – People are 10-20% more likely to agree with something if you only offer them an opportunity to agree. Introduction to Political Research Types of Questions • Threatening Questions – Questions about sensitive topics • Using drugs, unconventional sexual behavior, health questions (venereal disease), income, evading taxes – Easier after rapport established – Face to face more successful • Skip or Contingency Questions – If answer yes to one question, ask another – If no, skip to another section • Open v. Closed Ended Questions • Open-ended question format = Survey questions that allow respondents to give any answer. • Closed-ended question format = Survey questions in which respondents must chose among fixed answer choices. – Predetermined answer categories Introduction to Political Research Introduction to Political Research Open v. Closed Ended Questions Open Ended Questions Advantages Disadvantages Permit an unlimited number of possible answers Different respondents give different degrees of detail Respondents can answer in detail and can quality/clarify responses Responses may be irrelevant or buried in useless detail Unanticipated findings can be discovered Comparisons and statistical analysis become very difficult Permit adequate answers to complex issues Coding is difficult Permit creativity, self-expression and richness of detail Articulate and highly literate respondents have an advantage Reveal a respondent’s logic, thinking process and frame of reference Questions may be too general for respondents who lose direction Responses are written verbatim, which is difficult for interviewers Great amount of respondent time, thought and effort is necessary Respondents can be intimidated by questions Answers take up a lot of space in questionnaires Introduction to Political Research Open v. Closed Ended Questions Closed Ended Questions Advantages Disadvantages Easier and quicker for respondents to answer Can suggest ideas that the respondent would not otherwise have Answers of different respondents are easier to compare Respondents with no opinion or no knowledge an answer anyway Answers are easier to code and statistically analyze Respondents can be frustrated because their desired answer is not a choice Response choices can clarify questions meaning for respondents It is confusing if many (e.g., 20) response choices are offered Respondents are more likely to answer about sensitive topics Misinterpretation of a question can go unnoticed There are fewer irrelevant or confused answers to questions Distinctions between respondent answers may be blurred Less articulate or less literate respondents are not at a disadvantage Clerical mistakes or marking the wrong response is possible Replication is easier They force respondents to give simplistic responses to complex issues They force people to make choices they would not make in Introduction to Political Research the real world Writing Good Closed Format Response Choices • Answer choices should have three features: – Mutually exclusive • Response categories do not overlap • Likert – Exhaustive • Each respondent has a choice • To be safe – don’t know option – Balanced • Offer the favorable or unfavorable choices equally – Reduces bias Introduction to Political Research Caution…. • Social Desirability Bias = A tendency for survey respondents to answer in a way that conforms to social expectations or makes they look good rather than to answer honestly. • Can result in overestimating some things: – Did you vote? – Do you support efforts to reduce discrimination? • Can result in underestimating other things: – Are you going to vote for Trump? – Are you using illegal drugs? Introduction to Political Research Caution • Interviewer bias – Race and gender of interviewer effect – Support for Affirmative Action increases if someone perceives that they are speaking with minorities – Support for abortion rights increases if women are asking the questions • Length of Survey – People become tired, bored, and impatient – Hang up the phone – Will just start answering questions without really thinking about them – Phone interviews must be shorter (20 minutes) vs. in person interviews (can be an hour or longer) Introduction to Political Research Types of Surveys Mail and Self Administered Surveys Advantages Disadvantages Give questionnaires directly to respondents or mail them Respondents to not always complete or return them Least expensive Mail surveys have a low response rate Can be conducted by a single researcher Some are returned in 2 weeks, other trickle in Can cover a wide geographical area Conditions can not be controlled for mail surveys Respondents can complete the questionnaire when it is convenient and check personal records if necessary Questions can not be clarified and probing is not an option Anonymity Someone other than the targeted individual may complete Avoids interviewer bias Respondents may not complete them in a single sitting Effective Format limits the kinds of questions that can be used Response rates may be high for educated target population that has a strong interest in the topic or survey organization Questions requiring visual-aids or open-ended questions are difficult Introduction to Political Research Types of Surveys Web Surveys Advantages Disadvantages Fast Coverage Inexpensive Privacy and confidentiality Flexible design options Design Issues Can include visual aids Introduction to Political Research Types of Surveys Telephone Surveys Advantages Disadvantages 95% of the population be reached by phone Higher cost and limited interview length Response rates can reach 90% Impossible to reach some respondents Flexible with most of the strengths of a face-to-face interview, but half the cost Call may come at an inconvenient time for respondent Reduces anonymity Introduces potential interviewer bias Introduction to Political Research Types of Surveys Face-to-Face Surveys Advantages Disadvantages Highest response rate High cost Permits the longest questionnaires Training, travel and supervision costs and efforts are high Observe surroundings and use nonverbal communication and visual aids Interviewer bias is the greatest Well-trained interviewers can ask all types of questions, can ask complex questions and use extensive probes Introduction to Political Research Types of Surveys Introduction to Political Research Introduction to Political Research Interviews • Role of an interviewer – Structured conversation – They may simulate conversation, but everything they say is scripted – There should be no variation in how one interview is conducted compared to the next – There should be no variation from interviewer to another – The same questions should be asked to different people Introduction to Political Research Interviews • Stages of an interview – Intro – purpose and confidentiality – Middle – ask substantive questions – Exit – Thank the respondent and let them know when results will be available • Training interviewers – Receive extensive training before being placed in the field – They must know how to use computer programs – Probe = A neutral request made by an interviewer to clarify an ambiguous answer, complete an incomplete answer, or obtain a relevant response. Introduction to Political Research Time Dimension • Cross-Sectional – Single point in time – snapshot – Surveys • Longitudinal – Time series – Panel – same people – DARE Evaluation – Cohort – different people with same characteristics • People born in 1980, people that began employment at same time Introduction to Political Research Sampling • Random Digit Dialing = Computer based random sampling of telephone numbers. – Not using a telephone book as sampling frame • Will miss unregistered numbers which in some areas can be 40% – Using all possible numbers in an area • Area code – exchange – 4 numbers • Within Household Samples = Random sampling from within households – Ask question to determine respondent. Introduction to Political Research Public Support For Against No Opinion 2005 May 2-5 74% 23% 03% 2004 May 2-4 71% 26% 03% 2003 Oct 6-8 64% 32% 04% 2003 May 19-21 70% 28% 02% 2003 May 5-7 74% 24% 02% 2002 October 10 70% 25% 05% 2002 May 6-9 72% 25% 03% 2001 Oct 11-14 68% 26% 06% 2001 May 10-14 65% 27% 08% March 2015 Sampling: How to Select a Few to Represent the Many Chapter 4 Introduction to Political Research Introduction • • • • 70% doubt accuracy of polls How do I know that 70% don’t trust? Did I ask everyone in the US? I using a sample to tell you that between 67% and 73% of people don’t trust samples • If conducted properly with a high enough sample size we can make inferences about billions of people Introduction to Political Research Introduction • But people remain skeptical • Which is sometimes justified – Poorly conducted surveys do not provide reliable data – But if done scientifically, they should be trusted • Pertinent Questions: – Who collected it? – What is sample size? – How was sample generated? • “If you don’t believe in random sampling, next time tell the doctor to take it all” Introduction to Political Research Introduction • There are good polls and bad polls • CNN: – Hillary: 62 – Trump: 27 – Scientific Poll: National random sample of 530 registered voters who watched the debate – CI = 4.5% • Online Polls: – – – – – Hillary: 39 Trump: 53 Non-Scientific Poll: Asked people to vote who found the poll Prone to bias People that cast votes are not representative, can cast multiple votes, people can cast votes even if they don’t live in the united states. – And these poll results were reported! Introduction to Political Research Overview • 8 Main Types of Samples • 4 are representative – Which means I can make a prediction of an overall population based on my sample – I can construct confidence intervals • Margin of error • Because we are not 100% certain (it is an estimate after all) • 4 are not representative – There is no know probability of being selected for a sample out of a population – Which means I can’t make inferences Introduction to Political Research Overview • Known Probability of Selection (generalization) – Random • Known • Everyone has equal probability of selection • Size of probability will be based on size of sample vs. overall population – Systematic • Known probability • Probability will vary based on sampling interval – Stratified • Known probability • Probability is based on strata used – Cluster • Known probability • Probability based on cluster units Introduction to Political Research Overview • Non-Probability (difficult to generalize) – Convenience/haphazard • No know probability – Quota • No known probability – Purposive • No known probability – Snowball • No known probability Introduction to Political Research Generalizability • What do we do when we generalize our results? – How applicable to situations outside of our sample • Random sample is critical – Probability theory is based on random samples – Reduces bias • Generalize Minneapolis study of mandatory arrest for abuse to US? – 1980 – When police were called for domestic violence they were instructed to 1. arrest 2. counsel 3. mandatory separation – Lower recidivism for those arrested • Interviews of 7 Louisiana Supreme Court Justices – Generalized to all state supreme court judges • Generalize innovative reentry program from Cleveland, OH to entire US? – Inmates begin to meet with coach 6 months prior to release • Random sample of 1,067 registered voters to US? Introduction to Political Research Reasons People are Skeptical • My phone didn’t ring • I see bumper stickers and signs all over • Wrong a few times – Literary Digest • FDR v. Landon (1936) • Incomplete sampling frame (automobile lists) – 2000? • Most were correct but within margin of error • Personal experiences – Individual experiences in TX vs aggregate trends Introduction to Political Research Introduction to Political Research Why Sample? • Cost of census is too high – 13 billion for 2010 – Hired 500,000 people to follow up on 34% of houses that did not return form • Estimation is accurate – no need to capture everyone – Probability theory – Confidence Intervals • Effort by Clinton Administration to change census process – Majority do not state that estimates are precise Introduction to Political Research Sampling • Process for selecting sources of the observations used in a research study • Sample – a smaller collection of units taken from a larger collection • Two Types – Nonprobability • Cheap and easy • Exploratory research – Probability • Known probability of being sampled • Allows for estimation of population parameter and confidence interval Introduction to Political Research Fundamental Question to Always Act • How representative is my sample? Introduction to Political Research Types of Nonprobability Samples • Haphazard, Accidental or Convenience – – – – Easy and cheap but of limited value Select cases until we achieve our sample size No specific criteria Select cases that are available, willing, or look “normal” • Can lead to selection bias • May avoid some people because they look threatening – But those people are still part of overall population we are attempting to sample from – Ex. Cable News surveys – You get information, just not representative information • You can get very unrepresentative samples – Good for exploratory research purposes Introduction to Political Research Convenience Sample Example • Your question: – How do US students intend to vote in the presidential election? • Sampling Technique: – Stand outside of Olin on Friday afternoon between 9:00am and noon. – Talk to people until I get 200 responses • Not Representative Introduction to Political Research Types of Nonprobability Sampling • Quota – Still fast and cheap to conduct – Improvement over convenience sampling but still not great – First: You select relevant categories – Second: Determine overall sample size and size required for each category – Finally: Select cases haphazardly until you fill quota Introduction to Political Research Example • Your Question: How do UA students intend to vote in the presidential election? – And I think gender matters – And I know that UA has 60% female and 40% male enrollment • Sampling Technique – First: Decide on overall sample (we will use 100) – Second: Determine the breakdown based on your prior information (60-40 gender) – Finally, haphazardly select 60 women and 40 men to interview. • More accurate data – So now I am representative on gender – But I may not be on age, race, religion, etc. – So still very problematic Introduction to Political Research Introduction to Political Research Types of Nonprobability Samples • Purposive or Judgmental – Best used if you have a specific target population in mind and if that population may be difficult to reach. – Here you do not want a representative sample as you are not interested in what the overall population looks like – Uses the judgment of an expert in selecting cases or selected with a specific purpose in mind – Fenno (soak and poke) • Members of congress who spend most of their time in either districts or Washington – Deviant populations (Drug Dealers/Users) – Tearoom study • Also raised several privacy concerns • Don’t want a sample of anyone using a public restroom Introduction to Political Research Types of Nonprobability Samples • Snowball (network) – Best used for difficult to reach populations – Each case is connected with another through direct and nondirect linkages • Begin with a few cases and then spread or snowball out – Drug dealers, prostitutes, gang leaders, etc – Sometimes people use financial incentives Introduction to Political Research Snowball Sampling Introduction to Political Research Why Use Probability Sampling • Minimize bias – They are purely mathematical or mechanical – Selection due to chance (flip of coin) • Reduce bias because everyone has chance of selection – Allow calculation of probability of outcomes with great precision • Can obtain a sampling error (e.g., the degree to which a sample deviates from a population) • Obtain a representative sample – Generalizability Introduction to Political Research Important Terms • • • Population – larger pool of cases from which a sample is drawn or taken – Who or what we generalize to – Usually unknown – Does not need to be people: Can be countries, newspaper articles, etc. – All felons in OH – All voters in OH Sampling Ratio – ratio of the size of the sample to the size of the target population – n/N – Target Population = 50,000 people – I sample 150 – Sample Ratio = 150/50,000 which is .3 Sampling Frame – a specific list that closely approximates all elements in the population – ODC records, phone book, voting registration, etc. – I sample from this list – Incomplete lists lead to biased and inaccurate results Introduction to Political Research Important Terms • • Population Parameter – any characteristic of a population that we estimate – Unknown (data I don’t have) – Percentage of people that smoke, percentage of people who support Hillary/Trump, percentage of people who own a gun, percentage of people who have had an abortion, percentage of people who have been sexually assaulted Statistic – information from the sample, most often used to estimate population parameters – Estimate of unknown (based on data I do have) – Point estimate with a bubble of confidence around it – Percent support for Hillary/Trump is a population parameter – We do not know the true level of support but we do know within +/- 3% Introduction to Political Research Sampling Ratio • Sampling Ratio – ratio of the size of the sample to the size of the target population Introduction to Political Research Key of Probability Sampling • RANDOM SAMPLING – These samples are most likely to produce a sample that truly represents the unknown population – Random in sampling world means a technique where each element in a population has a known and equal chance of selection – Probability theory requires random sampling to generate inferences • Statistical theory and mathematical proofs are based on a known probability of selection • i.e. the match between our sample and the population • It allows us to know how much our sample estimates deviate from the population Introduction to Political Research Random Sampling • Need Unbiased Sampling Frame • A sample drawn in which a random process is used to select units from the population • Best to get an accurate representation of the population – Essential for treatment and control groups – Difficult to conduct – Expensive • Always have a margin of error (e.g., confidence interval) – N = 1,067 3% – Confidence Level= on repeated sampling – usually 95% – Margin of error = 50% +/- 3 Introduction to Political Research Types of Probability Sampling • Simple Random Sample – This is a type of random sample (obviously) • This is the easiest to do – Sample elements selected from the frame based on a mathematically random selection procedure – Steps: • 1st: Select Sampling Frame – Each elements in sampling is numbered from 1 and on until the last element of sample • 2nd: Randomly select elements – A computer does this – Import sample frame, tell computer the sample size and then it computes a random list • 3rd: Locate elements – Call, go to home, send questionnaire, etc. Introduction to Political Research Simple Random Sample • Accurate sampling frame critical (Literary Digest) • Even phone book might contain non-response bias • Selection based on mathematically random procedure – The researchers selects the sampling frame – Computers randomly select who is selected • Confidence Intervals = • CI: sample statistic +/- confidence level (SD) Introduction to Political Research Types of Probability Sampling • Systematic – This is a type of probability sample – Elements are selected at predetermined intervals rather than randomly – An approximation to random sampling in which you select one in a certain number of sample elements, the number is from the sampling interval – Sampling interval (the size of the sample frame over the sample size, used in systematic sampling to select units) • N/n • If I need a sample of 100 and my sample frame is 1,000, my sampling interval is 10 • Thus, I select every 10th case Introduction to Political Research Types of Probability Samples • Steps: – – – – 1. 2. 3. 4. Locate Sampling Frame Determine sample size Calculate Sampling Ratio Locate every case based on sampling ratio • Congressional bills, newspaper articles, judicial decisions, etc. • Bias – underling order to list (Military Squads, GPA 1, 51, and 101 different from 50,100, and 150) Introduction to Political Research Types of Probability Sampling • Stratified • This is a type of random sample • A type of random sampling in which a random sample is drawn from multiple sampling frames, each for a part of the population • You have prior information about the overall population (just like in quota) • You want to be certain to include these cases Introduction to Political Research Types of Probability Samples • Steps – 1st: Population is divided into subpopulations (strata) • Based on the criteria you select – Age, Gender, Race, Religion, etc… – 2nd: Random sample is drawn from each strata • This is what separates it from quota – Example: • I want a sample of 1,000 students from a University with 10,000 students • I know that 60% are female and 40% are male • My 2 strata consist of 6,000 females and 4,000 males • Next I randomly sample from each group until I have 600 females and 400 males Introduction to Political Research Stratified Sampling Introduction to Political Research Types of Probability Samples • Cluster (multi-stage) • This is a type of random sample • A multi-stage sampling method, in which clusters are randomly sampled, then a random sample of elements is taken from sampled clusters. – NO SAMPLING FRAME • No national list of professors working at colleges with less than 10,000 students or crime victims – Clusters are temporarily treated as cases themselves – Each cluster contains cases that will eventually be sampled Introduction to Political Research Types of Probability Samples • • • • • • Example: National Crime Victimization Survey There is no national list of crime victims DoJ uses cluster sampling Randomly select Counties (First Cluster) Randomly select geographical districts (Second Cluster) • Randomly select houses (Third Cluster) • Randomly select anyone over age 12 in the house (Final Sample) Introduction to Political Research Cluster (Multi-Stage) Sampling Introduction to Political Research Cluster (Multi-Stage Sampling) Introduction to Political Research Introduction to Political Research Sampling • Random Digit Dialing = Computer based random sampling of telephone numbers. – Not using a telephone book as sampling frame • Will miss unregistered numbers which in some areas can be 40% • Miss people without landlines • People who have moved • People who only have a cellphone – Using all possible numbers in an area • Area code – exchange – 4 numbers Introduction to Political Research Cell Phones • • • • It is important to try and sample cellphone only individuals/households 2004: 10% were cellphone only 2016: 47% are cellphone only If not accounted for, sample is not representative because cell phone only individuals are different from those with landlines – Cellphone only users are younger, have less education, less income, etc. – Missing this group results in sample that is not representative • They can pay companies for their lists of cellphone only numbers • Or they can use RDD – Cellphones have area codes and unique exchanges – Once you know the area code and the unique 3 number exchange you randomly generated last 4 numbers • Both are expensive – Must pay for list – Federal law requires that cell phones be dialed by an actual human which costs money because you have to hire people to make calls and ask questions Introduction to Political Research Within-Household Sampling • Within Household Samples = Random sampling from within households – Ask question to determine respondent. – Not only randomly sample house, you randomly sample person living there – Example: “Can I talk to the person over 18 who had the last birthday?” • Sampling Hidden Populations – Hidden Population = A group that is very difficult to locate and may not want to be found, and therefore, are difficult to sample. • Use purposive or snowball Introduction to Political Research Size of the Sample • Depends on the kind of data analysis and desired accuracy of the sample – Large sample size does not automatically guarantee representative sample • Bad sampling frame produces inaccurate results no matter how many you sample • With an accurate sample, your estimates become more accurate the greater your sample size but there are diminishing returns based on mathematical proofs • This is the central limit theorem • Rule of Thumb – to be 95% confident in your estimate with a +/- 3% margin of error… – – – – Diminishing returns Small populations (under 1000), large sampling ratio (about 50%) Moderately large populations (10,000), smaller sampling ratio (about 10%) Large populations (over 150,000), less than 1% • Magic number, based on mathematical proofs, is 1067 • And the same to say we are 95% confident in our estimate if there are 300,000,000 Introduction to Political Research
Purchase answer to see full attachment