Chapter 11 (Smith and Davis) Designing, Conducting, Analyzing, and Interpreting Experiments with More Than Two Groups Overview of this Chapter  The Good News and the Bad News First up, the Bad News. Once more, we look at statistics. Here, that means a One Way ANOVA, statistic that once again relies on interval or ratio scales, but with more than two groups The Good News? This chapter duplicates a lot of information that we covered in Chapter 10 (SD) on two group designs Overview of this Chapter  This chapter does add a layer of complexity, focusing on studies that have one IV but more than two groups. Part One: Experimental Design: Adding Basic Building Blocks Part Two: Statistical Analysis: What Do Your Data Show? Part Three: Interpretation: Making Sense of Your Statistics Part Four: An Eye Toward The Future Part One Experimental Design: Adding to the Basic Building Blocks Experimental Design  Recall that the experimental design refers to the general plan for selecting participants, assigning them to experimental conditions, controlling extraneous variables, and gathering data Chapter 10 focused on One IV, Two Group designs (correlated or independent) – I want to take just a quick moment to remind you of the twogroup design using an example that we all know very well living in Miami – the nightmare of dealing with traffic! Experimental Design  Experimental Design One IV, Two Group Designs – Imagine a researcher wants to see how long it takes for participants driving convertible cars to honk their horns if the car in front of them sits at a green light for up to 12 seconds – Our brave researcher stops in front of convertibles that have their top either open or closed (Note: with the tops open, she can see the convertibles’ drivers, and they can see her!) – Thus there are two groups: top open versus top closed Ellison, Govern, et al. 1995 Experimental Design  Experimental Design One IV, Two Group Designs – In this real-life study, Ellison found that participants who had their convertible tops closed honked longer and faster than convertible drivers who had their top open (presumably because the “top closed” drivers were more anonymous) Pop Quiz – Quiz Yourself  Imagine you design a study to see if a new organizational policy on sexual harassment is effective You train half of the department using the old policy, half using the new policy, and then test their comprehension of appropriate and inappropriate behaviors in the workplace What is the appropriate t-Test to use? A). t-Test for independent samples B). t-Test for correlated samples Pop Quiz – Quiz Yourself  Imagine you design a study to see if a new organizational policy on sexual harassment is effective You train half of the department using the old policy, half using the new policy, and then test their comprehension of appropriate and inappropriate behaviors in the workplace What is the appropriate t-Test to use? A). t-Test for independent samples B). t-Test for correlated samples Pop Quiz – Quiz Yourself  Now, imagine you design a study to see if the new policy on sexual harassment is effective You measure everyone’s comprehension before training them on the new policy and then compare that score to their comprehension score after the training. What is the appropriate t-Test to use? A). t-Test for independent samples B). t-Test for correlated samples Pop Quiz – Quiz Yourself  Now, imagine you design a study to see if the new policy on sexual harassment is effective You measure everyone’s comprehension before training them on the new policy and then compare that score to their comprehension score after the training. What is the appropriate t-Test to use? A). t-Test for independent samples B). t-Test for correlated samples Experimental Design  Experimental Design One IV, Two Group Designs Convertibles with their tops open or closed present a good twogroup comparison, but sometimes researchers are interested in looking at more than just two levels of the IV – This is where a multi-level (three or more levels) IV study comes into play … Experimental Design  Picture not found Experimental Design One IV, Three or More Groups – Researcher Daniel Linz exposed participants to one of three kinds of movies with an “adult” element to them Slasher Films Teen sex films Non-violent Porn Experimental Design  Experimental Design One IV, Three or More Groups – Researcher Daniel Linz exposed participants to one of three kinds of movies with an “adult” element to them Linz was interested in looking at participants’ empathy levels for a (fictional) victim of rape after they had just viewed one of the three types of films Thus his DV was empathy while his IV was type of film (three levels), and participants were randomly assigned to view one of the three films Experimental Design  Experimental Design One IV, Three or More Groups – Researcher Daniel Linz exposed participants to one of three kinds of movies with an “adult” element to them Linz found that participants who viewed the teen-sex films and the soft-core pornography films had MORE empathy than those who viewed the slasher film. In other words, participants found rape more acceptable after viewing a slasher film (Kind of gives you secondthoughts about watching those movies, huh!) Experimental Design  Experimental Design One IV, Three or More Groups – As in Chapter 10, we have several questions to ask before beginning our analysis, one related to the number of IVs, one related to the number of levels of each IV, and one related to whether we use independent assignment to groups versus correlated designs assignment to groups – Before starting that discussion, let’s see how much review you need of independent vs. correlated designs … Pop Quiz – Quiz Yourself  Participants are angered by a confederate and then wait in a room with an object in the corner (a gun or a tennis racket). Later, they can retaliate against the confederate by giving him shocks. Findings show greater retaliation by p’s who saw a gun than p’s who saw a racket. What kind of design is this? A. One IV, Two-Group Design for Independent Groups B. One IV, Two-Group Design for Correlated Groups C. One IV, Multiple Group Design for Independent Groups D. One IV, Multiple Group Design for Correlated Groups Pop Quiz – Quiz Yourself  Participants are angered by a confederate and then wait in a room with an object in the corner (a gun or a tennis racket). Later, they can retaliate against the confederate by giving him shocks. Findings show greater retaliation by p’s who saw a gun than p’s who saw a racket. What kind of design is this? A. One IV, Two-Group Design for Independent Groups B. One IV, Two-Group Design for Correlated Groups C. One IV, Multiple Group Design for Independent Groups D. One IV, Multiple Group Design for Correlated Groups Pop Quiz – Quiz Yourself  Participants are angered by a confederate and then wait in a room with an object in the corner (a rifle, a green plastic water gun, or a tennis racket). Later, they can retaliate against the confederate by giving him shocks. Findings show greater retaliation by p’s who saw a gun than p’s who saw a racket. NOW what kind of design is this? A. One IV, Two-Group Design for Independent Groups B. One IV, Two-Group Design for Correlated Groups C. One IV, Multiple Group Design for Independent Groups D. One IV, Multiple Group Design for Correlated Groups Pop Quiz – Quiz Yourself  Participants are angered by a confederate and then wait in a room with an object in the corner (a rifle, a green plastic water gun, or a tennis racket). Later, they can retaliate against the confederate by giving him shocks. Findings show greater retaliation by p’s who saw a gun than p’s who saw a racket. NOW what kind of design is this? A. One IV, Two-Group Design for Independent Groups B. One IV, Two-Group Design for Correlated Groups C. One IV, Multiple Group Design for Independent Groups D. One IV, Multiple Group Design for Correlated Groups Experimental Design  Experimental Design One IV, Three or More Groups – In this section, we will cover multiple-level studies in more detail, focusing on – 1). The Multiple-Group Design – 2). Comparing Multiple-Group and Two-Group Designs – 3). Comparing Multiple-Group Designs – 4). Variations on the Multiple-Group Design – 5). Post Hoc Tests in Multiple-Group Designs Experimental Design  1). The Multiple-Group Design Similar to Chapter 10, we need to ask several questions about our research design before analyzing the study data – A. How many IV’s are there? – B. How many groups are there? – C. How do we assign participants to groups? – To begin answering these questions, think back to the chart we looked at in Chapter 10 … Now, of course, we focus on three or more levels for our single IV As in Chapter 10, we look at Independent vs. Correlated groups Experimental Design  1). The Multiple-Group Design A. How many IV’s are there? – Similar to in Chapter 10, we are only concerned with one IV in this chapter, but here it has three or more levels. Still, we have ONLY ONE independent variable in Chapter 11 Yet we are still beholden to the principle of parsimony: “Keep It Simple Student” KISS Experimental Design  1). The Multiple-Group Design B. How many groups are there? – In multiple group designs, we compare three or more levels To keep a study practical, you should keep the number of levels manageable, like four to five levels maximum (as a rule of thumb, you need 20 participants per level) If your single IV has five levels, you’ll need 100 p’s If your single IV has ten levels, you’ll need 200 p’s! Theoretically, you can have an unlimited number of IV levels, but there is only one IV in this our current design Pop Quiz – Quiz Yourself  ___ is the real limit on the numbers of levels of the IV you can have in a multiple group design, but ____ is the practical limit A. Five; Five B. Two; Two C. As many as you want; Four or Five D. As many as you want; Twenty or Thirty Pop Quiz – Quiz Yourself  ___ is the real limit on the numbers of levels of the IV you can have in a multiple group design, but ____ is the practical limit A. Five; Five B. Two; Two C. As many as you want; Four or Five D. As many as you want; Twenty or Thirty Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? – Again, like Chapter 10, we can assign participants to groups using either an … Independent group design OR Correlated group design Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? Independent – Independent group designs involve random assignment, or a method of assigning participants to groups so that each participant has an equal chance of being in any group Random assignment serves a control function: Potential extraneous variables are controlled because they should (hopefully) occur in all conditions Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? Independent – Independent group designs involve random assignment, or a method of assigning participants to groups so that each participant has an equal chance of being in any group Random assignment helps us avoid confounding as well, limiting the impact of extraneous variables on the IV Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? Independent – In Linz’s study, he randomly assigned participants to watch one of the three videos: Soft-Core Pornography Slasher Movies Teen-Sex Movies – Participants only watched one of the films, giving them some “independence” from the other groups in the study Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? Correlated – We can also use correlated groups in a multi-group design If we have only a few participants (a small n) or the IV is weak or very too subtle, we may use a correlated group design (non-random assignment) Matched Sets (not pairs, but sets) Natural Sets (not pairs, but sets) Repeated Measures Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? Correlated – We can also use correlated groups in a multi-group design We use “matched sets” rather than “matched pairs” We match participants on a matching variable (any potential non-IV variable that may affect scores on our dependent variable) Such variables might include: • Demographic characteristics • Attitudes, beliefs, experiences, etc. Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? Correlated – We can also use correlated groups in a multi-group design We use “matched sets” rather than “matched pairs” Think about Linz. Which variable(s) might we want to match on before having participants to their film? • Gender: We match three males and assign one to the teen comedy, one to the slasher film, and one to the soft-core porn film. Then repeat • We could look at movie habits, marital status, etc. Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? Correlated – We can also use correlated groups in a multi-group design In multi-group designs, we might also use “natural sets” (NOT natural pairs). However, finding natural sets gets tougher the more levels you have for your IV Finding three or more “natural” siblings is tough, but if your IV has 4, 5, or 6 levels, it may be impossible! Three or more friends / animals might be easier to get, but it still gets tough Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? Correlated – We can also use correlated groups in a multi-group design Finally, multi-group designs can use repeated measures. This gets tough on participants, who must sit through three or more sessions • Maybe they watch all three films in Linz’s study and give their rape empathy assessments after each. This is very time consuming, but possible! Experimental Design  1). The Multiple-Group Design C. How do we assign participants to groups? Correlated – We can also use correlated groups in a multi-group design Finally, multi-group designs can use repeated measures As with One IV, Two Level studies, we also have to worry about carryover effects here • Are participant scores after the third film based on that specific film, or based on their responses to the prior two films? Experimental Design  2). Comparing the Multiple-Group and Two-Group Designs A question we haven’t looked at yet but we need to address is, “Does your research question require more than two groups?” There are a few things to consider here … – First, review the literature to see if the simple two-group test has already been done If not, try a two group test first, and do a multi-group later For example, I might want to see how well Methods Two students do in a Live class methods version vs. an Online class version first, and then add in “Hybrid” and “Independent Study” comparison versions later Experimental Design  2). Comparing the Multiple-Group and Two-Group Designs A question we haven’t looked at yet but we need to address is, “Does your research question require more than two groups?” There are a few things to consider here … – Second, “What will the addition of more groups tell you?” Maybe you want to see if a new ADHD skills class helps students learn, so you get a baseline score (time 1). You might want to then get a time 2 score (right after the first skills class). You might want a time 3 score to see if the lessons persist a month after the course, or two months after the course (time 4), etc. Experimental Design  2). Comparing the Multiple-Group and Two-Group Designs A question we haven’t looked at yet but we need to address is, “Does your research question require more than two groups?” There are a few things to consider here … – Third, determine how many participants are available If there are only a few participants available, then keep the design simple. Sometimes it is tough to find enough participants to run a multiple-group study, so a simple two-group study design might be best Experimental Design  3). Comparing Multiple-Group Designs There are two additional elements will impact your decision to use either an independent or correlated design, including: – A. Control Issues – B. Practical Issues Experimental Design  3). Comparing Multiple-Group Designs A. Control – Independent Designs – Independent designs rely on random assignment, creating (hopefully) three or more groups that are equal Of course (to beat a dead horse yet again), such random assignment does not guarantee group equality! Consider our basic formula again Experimental Design  3). Comparing Multiple-Group Designs A. Control – Independent Designs – Independent designs rely on random assignment, creating (hopefully) three or more groups that are equal Our hope is that the error variability is low among our groups, making the Statistic higher (especially with high between group variability). But we have less ability to control such error variability in independent designs Experimental Design  3). Comparing Multiple-Group Designs A. Control – Correlated Designs – Correlated designs reduce error variability through better by controlling sources of error (using participants matched on a key characteristic or using the same participant repeatedly). We don’t need to hope that our three or more groups are similar, as we know they are similar! Experimental Design  3). Comparing Multiple-Group Designs B. Practical issues also arise in multiple-group design studies – Independent groups designs require a lot of participants – Correlated designs have their own issues It may be tough to match three or more participants in matched sets and natural sets designs For repeated measures, having more levels means more participants may need to spend more time in the study. There are also potential confounds Pop Quiz – Quiz Yourself  You should use correlated designs (rather than independent designs) in all of the following situations EXCEPT: A. When you only have a few participants B. When you wish to limit variance based on key participant characteristics that are relevant to your independent variable C. When you are hoping to avoid carryover effects D. When you want to see how repeated exposure to the treatment influences participants Pop Quiz – Quiz Yourself  You should use correlated designs (rather than independent designs) in all of the following situations EXCEPT: A. When you only have a few participants B. When you wish to limit variance based on key participant characteristics that are relevant to your independent variable C. When you are hoping to avoid carryover effects D. When you want to see how repeated exposure to the treatment influences participants Experimental Designs  Pause-Problem #1 (Three Group Designs) For this first Pause-Problem, I want you to design three (brief!) studies (they can all be variations on the same idea). Make sure to note the independent and dependent variables for all – 1). One should use an independent three group design – 2). One should use a matched OR natural set design – 3). One should use a repeated measures design #1 If this question looks familiar, it is! I asked a similar question in Chapter 10 (SD). Feel free to expand on your answer to that Chapter 10 pause problem! Experimental Design  4). Variations on the Multiple-Group Designs Comparing Differing Amounts of an IV – I mentioned in Chapter 10 that sometimes rather than using a “control” group, we use a “comparison” group. The same comparison groups can occur in multiple-group designs However, in multiple group designs, we might have: Multiple control groups! Multiple treatment groups! Multiple comparison groups! Experimental Design  4). Variations on the Multiple-Group Designs Comparing Differing Amounts of an IV – Several years ago, I did a study looking at juror’s ability to comprehend death penalty instructions. I was living in St. Louis at the time, so I made sure to use Missouri Approved Instructions (MAI), or actual Missouri capital instructions! These instructions were notoriously bad, with jurors not really understanding them. Could we write better death penalty instructions using research from psychology about how jurors process information? Experimental Design  4). Variations on the Multiple-Group Designs Comparing Differing Amounts of an IV – To see if we could increase juror comprehension, we gave five groups of participants different instructions, some based on actual legal language used in Missouri, some based on everyday English, some based on visual flow charts, etc. In this study, we had multiple instruction conditions … Experimental Design  4). Variations on the Multiple-Group Designs Comparing Differing Amounts of an IV – Three control groups and two treatment groups: 1. Skeletal instructions – We gave jurors only basic legal information with little to no definitions of legal concepts 2. Legal definitions only – Also very basic, no real help 3. Missouri Instructions – Those actually used in Missouri 4. Flow Chart – Our first experimental version, a step-bystep visual approach to understanding the death penalty 5. Simplified – Second version, everyday simple English Death Penalty Instructions Death Penalty Instructions Can you figure out what we found? Does it surprise you? Experimental Design  4). Variations on the Multiple-Group Designs Comparing Differing Amounts of an IV – If we hadn’t used as many comparison / control groups as we did, it would have been hard to understand how bad the Missouri death penalty instructions were! Attorneys could say that offering legal definitions would help (Nope! We showed they wouldn’t) Attorneys could say the Missouri instructions are better than nothing (again, nope! This is not true) The only thing that aids comprehension of death penalty instructions are flow charts and simplified language! Experimental Design  4). Variations on the Multiple-Group Designs Dealing with Measured IV’s – The final element of this section focuses on Measured IV’s Like Chapter 10, sometimes a researcher cannot directly manipulate an IV, but she can still classify, categorize, or measure them (especially predetermined participant IV characteristics, like gender, age, race, or attitudes) We cannot draw cause-effect conclusions if using these “quasi-independent” variables, but they still might be useful in multiple-group designs Experimental Design  4). Variations on the Multiple-Group Designs Dealing with Measured IV’s – Going back to my death penalty study, I know death penalty attitudes might greatly impact how jurors perceive the death penalty instructions For example, when it comes to the death penalty, jurors must be “death-qualified” to serve as a capital juror. This means jurors must be willing to give the death penalty (compared to those completely unwilling) Would “death qualification” impact my results? Recall our data when death qualification wasn’t measured … Death Penalty Instructions Experimental Design  4). Variations on the Multiple-Group Designs Dealing with Measured IV’s – But what happens if we ONLY look at participants who are “death-qualified” (that is, those who can actually serve on a death penalty panel)? Time to see something scary: This is what we found … Death Penalty Instructions Experimental Design  4). Variations on the Multiple-Group Designs Dealing with Measured IV’s – Results show that our flow chart and simplified instructions did not help when only “death-qualified” jurors sit on a trial. – Apparently, our new instructions only help if both deathqualified and non death-qualified jurors are present. Unfortunately, it appears there is little that researchers can do to educate death-qualified jurors who actually serve on death penalty trials, which is quite depressing Experimental Design  4). Variations on the Multiple-Group Designs Dealing with Measured IV’s – Results show that our flow chart and simplified instructions did not help when only “death-qualified” jurors sit on a trial. – Apparently, our new instructions only help if both deathqualified and non death-qualified jurors are present. Yet remember that we cannot draw causal conclusions here, as we did not randomly assign jurors to be deathqualified (or not death-qualified) Pop Quiz! Quiz Yourself  Darryl conducts a study in which participants rate their liking for their professor on the first day of class, at midterm, and on the last day of class. Darryl has used which research design? A. One IV, Two-Group Design for Independent Groups B. One IV, Two-Group Design for Correlated Groups C. One IV, Multiple Group Design for Independent Groups D. One IV, Multiple Group Design for Correlated Groups Pop Quiz! Quiz Yourself  Darryl conducts a study in which participants rate their liking for their professor on the first day of class, at midterm, and on the last day of class. Darryl has used which research design? A. One IV, Two-Group Design for Independent Groups B. One IV, Two-Group Design for Correlated Groups C. One IV, Multiple Group Design for Independent Groups D. One IV, Multiple Group Design for Correlated Groups – More specifically, a repeated measures design Pop Quiz! Quiz Yourself  If we wished to compare the personality traits of firstborn kids (in families with 2 or more children), lastborn kids (in families with 2 or more children), and single kids (only one child in the family), what design would we use? A. One IV, Two-Group Design for Independent Groups B. One IV, Two-Group Design for Correlated Groups C. One IV, Multiple Group Design for Independent Groups D. One IV, Multiple Group Design for Correlated Groups Pop Quiz! Quiz Yourself  If we wished to compare the personality traits of firstborn kids (in families with 2 or more children), lastborn kids (in families with 2 or more children), and single kids (only one child in the family), what design would we use? A. One IV, Two-Group Design for Independent Groups B. One IV, Two-Group Design for Correlated Groups C. One IV, Multiple Group Design for Independent Groups D. One IV, Multiple Group Design for Correlated Groups Firstborn, lastborn, and single are the three levels in our single IV. Pop Quiz! Quiz Yourself  If the personality traits of firstborn kids (in families with 2 or more children), lastborn kids (in families with 2 or more children), and single kids (only one child in the family), did in fact differ (the p value was < .01), what would we use to see which of these groups differed from the other group(s)? A). An omnibus test B). Post hoc comparisons C). A follow-up correlation D). A follow-up regression Pop Quiz! Quiz Yourself  If the personality traits of firstborn kids (in families with 2 or more children), lastborn kids (in families with 2 or more children), and single kids (only one child in the family), did in fact differ (the p value was < .01), what would we use to see which of these groups differed from the other group(s)? A). An omnibus test B). Post hoc comparisons C). A follow-up correlation D). A follow-up regression Do you recall talking about post hoc tests in Methods One? Well, let’s do a quick reminder Experimental Design  5). Post Hoc Tests in Multiple-Group Designs A Brief Introduction to Post Hoc Tests – Recall the One IV, Two Group design. If our t-Test shows significance, all we need to do is look at the descriptive statistics to determine if Mean A is higher than Mean B – In a One IV, Multiple Group design, we are now dealing with Means A, B, and C (and possibly D, E, F, etc.), so looking at descriptive statistics alone will not tell us which means differ. We need post hoc tests for this Experimental Design  5). Post Hoc Tests in Multiple-Group Designs A Brief Introduction to Post Hoc Tests – I’ll talk about post hoc tests later in this chapter, but you might recall post hoc tests from Methods One when we covered Salkind (Chapter 13) – One of the most common in psychology is the Tukey post hoc test (which we will learn all about shortly!) – For now, how about a Pause-Problem … Experimental Design  Pause-Problem #2 (ANOVA versus the t-Test) For this Pause-Problem, I want you to do three things: – 1). Tell me how an ANOVA is similar to a t-Test – 2). Tell me how an ANOVA is different from a t-Test? – 3). Give me an example study idea for a t-Test as well as an example study idea for an ANOVA #2 Part Two Statistical Analysis: What Do Your Data Show? Statistical Analysis  In this section, we cover three main statistical analysis elements, including … 1). Analyzing Multiple-Group Designs 2). Planning Your Experiment 3). The Rationale of the ANOVA Statistical Analysis  1). Analyzing Multiple-Group Designs Comparing means: This is easy in a two-group design, where you look at the mean for Condition A compared to the mean for Condition B – Independent and correlated groups t-Tests are appropriate In multiple group designs, we are dealing with more than two means, so this entails a different statistical test – For multiple-group designs, we will focus a lot of attention on a type of test you should be familiar with … the ANOVA Statistical Analysis  1). Analyzing Multiple-Group Designs There are different ANOVAs for different kinds of studies! – In this Chapter 11, we will cover the One-Way ANOVA, or … One IV independent design ANOVA One IV correlated design ANOVA – (In Chapter 12, we will look at experiments with more than one IV) Statistical Analysis  1). Analyzing Multiple-Group Designs The ANOVA – Analysis of Variance – The One-Way ANOVA is a statistical test used to analyze data from an experimental design with one independent variable that has three or more groups (levels) The completely randomized ANOVA is appropriate for research that uses independent designs The repeated measures ANOVA is appropriate for research on correlated designs Statistical Analysis  2). Planning Your Experiment Recall the experiment we discussed in Chapter 10 (the same experiment discussed in your textbook) looking at the amount of time it takes for a salesclerk to approach a shopper dressed in either sloppy or dressy clothes. – Appropriately, we ran t-Tests for this study in Chapter 10, as we compared only two means Statistical Analysis  2). Planning Your Experiment Chapter 11 focuses on a three group design, so we add a third IV level: Sloppy clothes, Dressy clothes, and Casual clothes. – Keep in mind we could add other levels (naked shoppers, clown shoppers, etc.) Statistical Analysis  3). Rationale of the ANOVA Remember our formula for variability from Chapter 10 – When looking at Between Group Variability in the two-group design, we ask one basic question … Does Condition A differ from Condition B Statistical Analysis  3). Rationale of the ANOVA Remember our formula for variability from Chapter 10 – When thinking about Between Group Variability in multiplegroup designs (three or more), you ask three questions: Does Condition A differ from Condition B? Does Condition A differ from Condition C? Does Condition B differ from Condition C? Statistical Analysis  3). Rationale of the ANOVA Remember our formula for variability from Chapter 10 Consider data from Table 11:2 in your Smith and Davis text … Clothing & Salesmen Dressy 37 38 44 47 49 49 54 69 M = 48.38 Clothing Style Sloppy 50 46 62 52 74 69 77 76 M = 63.25 Casual 39 38 47 44 50 48 70 55 M = 48.88 Clothing & Salesmen Clothing Style Sloppy Casual 50 39 Between 46 You want 38 62 differences 47 Groups Dressy 37 38 44 variability 52 between the 44 47 49 74 dressy and 50 49 69 the sloppy 48 54 77 conditions 70 69 76 55 M = 48.38 M = 63.25 M = 48.88 Clothing & Salesmen Clothing Style Sloppy Casual 50 39 Between 46 You want 38 62 differences 47 Groups Dressy 37 38 44 variability 52 between the 44 47 49 74 sloppy and 50 the casual 48 49 69 54 77 conditions 70 69 76 55 M = 48.38 M = 63.25 M = 48.88 Clothing & Salesmen Clothing Style Sloppy Casual 50 39 Between 46 You want 38 62 differences 47 Groups Dressy 37 38 44 variability 52 between the 44 47 49 74 dressy and 50 the casual 48 49 69 54 77 conditions 70 69 76 55 M = 48.38 M = 63.25 M = 48.88 Statistical Analysis  3). Rationale of the ANOVA While we want high Between Group Variability, Error Variability should be minimized. This “bad” error variability … – may be due to characteristics of the participant – may be due to measurement error – may be due to recording errors Having control in the study and trying to reduce the influence of confounds / extraneous variables will help, but it is impossible to eliminate all error variability. Statistical Analysis  3). Rationale of the ANOVA Remember our formula for variability from Chapter 10 – As we move into multiple-group designs, we must consider that error variability is also present between different groups Clothing & Salesmen Dressy 37 38 44 47 49 49 54 69 M = 48.38 Clothing Style Sloppy Casual 50 39 46 IV variability 38 and error 62 47 52 74 69 77 76 M = 63.25 variability differ between conditions 44 50 48 70 55 M = 48.88 Statistical Analysis  3). Rationale of the ANOVA Remember our formula for variability from Chapter 10 At the same time, we also need to recognize that error occurs within each group as well. That is, even though all participants in a condition may be exposed to the same treatment, natural fluctuations exist even within that condition – Consider our table again. Notice how scores differ within the dressy condition (ranging from 37 to 69 seconds to help) Clothing & Salesmen Clothing Style Sloppy Casual 50 39 46 38 62 47 Within Error or Dressy 37 38 44 47 52 44 groups Subject variability 74 differences 50 49 49 69 48 54 77 70 69 76 55 M = 48.38 M = 63.25 M = 48.88 Clothing & Salesmen Clothing Style Sloppy Casual 50 39 46 38 62 47 Within Error or Dressy 37 38 44 47 52 44 groups Subject variability 74 differences 50 49 49 69 48 54 77 70 69 76 55 M = 48.38 M = 63.25 M = 48.88 Clothing & Salesmen Clothing Style Sloppy Casual 50 39 46 38 62 47 Within Error or Dressy 37 38 44 47 52 44 groups Subject variability 74 differences 50 49 49 69 48 54 77 70 69 76 55 M = 48.38 M = 63.25 M = 48.88 Statistical Analysis  3). Rationale of the ANOVA To sum up, we are dealing with several sources of variability in any data set, some due to error within groups and some due to error and good IV variability between groups – Participant characteristics (both within & between error) – Measurement / recording error (both within & between error) – Variability of independent variables (between variability) The ANOVA “partitions” this variability by dividing the treatment effect by the error. If the treatment effect is large (and the error low), your F statistic will be large (and significant!) Statistical Analysis  3). Rationale of the ANOVA Here is the basic Statistic formula we’ve used recently Now, below is the formula for the F test, which is similar to our usual formula, just a bit more precise F= Between-Groups Variability + Error Variability Error (Within-Groups) Variability Statistical Analysis  3). Rationale of the ANOVA The F-ratio statistic is the statistic computed in an ANOVA (our obtained value), which we compare to critical values of F – Similar to the t-Test, you can find the critical value for an F test (the value the obtained value F must be above to find it significant) by looking at an F table (Table A-2 in your book) IF significant, an ANOVA F requires further follow-up analyses (post hoc tests, like Tukey) to see if A differs from B, if A differs from C, or if B differs from C. Part Three Interpretation: Making Sense of Your Statistics Interpretation  Understanding the ANOVA is only half the battle. You also have to be able to interpret your results in SPSS. In this section, we will look at how to interpret statistical output, focusing on … 1). The One-Way ANOVA for Independent Samples 2). Translating Statistics into Words (Independent) 3). The One-Way ANOVA for Correlated Samples 4). Translating Statistics into Words (Correlated) 5). Comparing Independent and Correlated Designs Interpretation  1). One-Way ANOVA for Independent Samples Once again, recall the clothing study for a multi-group design, which asks the question, – “Do salespeople take longer to respond to customers who are dressed in sloppy, casual, or dressy clothes?” Let’s consider the data from Table 11-2) – Note: I strongly urge you to read section 11.3 in your Smith and Davis book to better understand the following slides … Table 11-2, page 250 Condition (IV) 1 = Dressy 2 = Sloppy 3 = Casual Time to help (in seconds – our DV) for eight participants per condition. Put it into SPSS as usual. Since we did this before, I will go through it quickly … Cond 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 Time (Sec) 37.00 38.00 44.00 47.00 49.00 49.00 54.00 69.00 50.00 46.00 62.00 52.00 74.00 69.00 77.00 76.00 39.00 38.00 47.00 44.00 50.00 48.00 70.00 55.00 Interpretation  1). One-Way ANOVA for Independent Samples Put the data into SPSS as we have done before, making sure to label everything we need Variable View Condition Labels Data View Interpretation  1). One-Way ANOVA for Independent Samples To run a One-Way ANOVA, go into the “Analyze” menu, scroll down to “Compare Means” and then click “One-Way ANOVA” – Move the IV to “Factor” and the DV to “Dependent List” – Click “Post Hoc” select “Tukey” – Click “Options” for “descriptives” Interpretation  1). One-Way ANOVA for Independent Samples The first table will be your descriptive statistics (which describe the data but do not let us know if they are significant). – The means and standard deviations below should look very familiar to you by now! Interpretation  1). One-Way ANOVA for Independent Samples Descriptive Statistics (Means, Standard Deviation) – Dressy Clothing: M = 48.38 sec, SD = 10.11 – Sloppy Clothing: M = 63.25 sec, SD = 12.54 – Casual Clothing: M = 48.88 sec, SD = 10.20 The means seem to differ between conditions, but our real question is “Do they differ significantly?” – To answer this question, we must look at the inferential statistics (One-Way ANOVA) Interpretation  1). One-Way ANOVA for Independent Samples The ANOVA source table refers to a table that contains the results of an ANOVA. Source refers to the source of different types of variation (within variation and between variation) – The “Sum of Squares” is the amount of variability in the DV attributable to each source (within + between). It’s tough to calculate by hand, so we will go with the SPSS numbers! Interpretation  1). One-Way ANOVA for Independent Samples The ANOVA source table refers to a table that contains the results of an ANOVA. Source refers to the source of different types of variation (within variation and between variation) – The between-groups df is the # of groups – 1, or 3 – 1 = 2 – Within groups df is # participants – # groups, or 24 – 3 = 21 – Total df is between groups + within groups, or 2 + 21 = 23 Pop Quiz – Quiz Yourself  When interpreting F(2, 27) = 8.80, p < .05, how many groups were examined? A). 30 B). 27 C). 3 D). 2 Pop Quiz – Quiz Yourself  When interpreting F(2, 27) = 8.80, p < .05, how many groups were examined? A). 30 B). 27 C). 3 (there are three groups initially. 3 – 1 = 2) D). 2 Pop Quiz – Quiz Yourself  When interpreting F(2, 27) = 8.80, p < .05, what is the total sample size? A). 30 B). 29 C). 3 D). 2 Pop Quiz – Quiz Yourself  When interpreting F(2, 27) = 8.80, p < .05, what is the total sample size? A). 30 (30 initial participants, minus 3 groups. 30 – 3 = 27) B). 29 C). 3 D). 2 Interpretation  1). One-Way ANOVA for Independent Samples The ANOVA source table refers to a table that contains the results of an ANOVA. Source refers to the source of different types of variation (within variation and between variation) – But we can ignore the “Total df” in our write up Interpretation  1). One-Way ANOVA for Independent Samples The ANOVA source table refers to a table that contains the results of an ANOVA. Source refers to the source of different types of variation (within variation and between variation) – The “Mean Square” is averaged variability for each source (divide Sum of squares by the df, or 1141.750 / 2 = 570.875 and 2546.25 / 21 = 121.25). This is essentially the variance Interpretation  1). One-Way ANOVA for Independent Samples The ANOVA source table refers to a table that contains the results of an ANOVA. Source refers to the source of different types of variation (within variation and between variation) – The F ratio is determined by dividing our variation sources Mean Square Between Groups 570.875 F= = = 4.708 Mean Square Within Groups 121.250 Interpretation  1). One-Way ANOVA for Independent Samples The ANOVA source table refers to a table that contains the results of an ANOVA. Source refers to the source of different types of variation (within variation and between variation) – Finally, we have our significance level. In this case, p < .05 Interpretation  1). One-Way ANOVA for Independent Samples The ANOVA source table refers to a table that contains the results of an ANOVA. Source refers to the source of different types of variation (within variation and between variation) – So, are we all done? Not yet! We have three means, so we need to figure out if mean A differs from B, if A differs from C, or if B differs from C. Time for some post-hoc tests! Interpretation  1). One-Way ANOVA for Independent Samples Post hoc tests refer to statistical comparisons made between groups means after finding a significant F ratio. As you saw when selecting a post hoc test, there are several you can use – Scheffe: This tends to be a very conservative test, and thus the IV has to be very strong in order to be significant – Tukey is more liberal than Scheffe, so finding significance is easier with Tukey than with Scheffe – LSD is even more liberal than Tukey Consider these three tests (and try others on your own!) … Interpretation  1). One-Way ANOVA for Independent Samples For the Dressy vs. Sloppy comparison, significance is .034 for Tukey, .044 for Scheffe (barely significant), and .013 for LSD. Although all are significant, let’s focus on Tukey … Interpretation  1). One-Way ANOVA for Independent Samples According to Tukey, Sloppy differs from Casual Interpretation  1). One-Way ANOVA for Independent Samples According to Tukey, Sloppy differs from Casual, Dressy differs from Sloppy Interpretation  1). One-Way ANOVA for Independent Samples According to Tukey, Sloppy differs from Casual, Dressy differs from Sloppy, but Dressy does NOT differ from Casual Interpretation  2). Translating Statistics into Words (Independent) Now we need to write it up. Again, this should be familiar! “The effect of different clothing on salesclerks’ response time was significant F(2, 21) = 4.71, p = .02. Tukey tests indicated that clerks waiting on customers dressed in sloppy clothes (M = 63.25, SD = 11.73) responded slower than clerks waiting on customers in both dressy clothes (M = 48.38, SD = 9.46) and casual clothes (M = 48.88, SD = 9.55). However, the response times of clerks waiting on customers in dressy and casual clothes did not differ from each other.” Pop Quiz – Quiz Yourself  Which is the correct way to write up the ANOVA output A. F(2, 35) = 85.75, p < .05 B. F(2, 33) = 21.97, p > .05 C. F(2, 33) = 3.90, p < .05 D. F(2, 35) = 3.90, p < .05 Pop Quiz – Quiz Yourself  Which is the correct way to write up the ANOVA output A. F(2, 35) = 85.75, p < .05 B. F(2, 33) = 21.97, p > .05 C. F(2, 33) = 3.90, p < .05 D. F(2, 35) = 3.90, p < .05 Pop Quiz – Quiz Yourself  Which is the correct way to write up the ANOVA output A. F(2, 35) = 85.75, p < .05 B. F(2, 33) = 21.97, p > .05 C. F(2, 33) = 3.90, p < .05 D. F(2, 35) = 3.90, p < .05 Your Turn – Discussion  Pause-Problem #3 (Multiple t-Tests) As you just saw, we used a Tukey post hoc test to look at the differences between the sloppy, dressy, and casual conditions. So why not just use three t-Tests (one comparing sloppy to dressy, one comparing sloppy to casual, and one comparing dressy to casual)? For this Pause-Problem, tell me why you wouldn’t want to run multiple t-Tests. #3 Interpretation  3). One-Way ANOVA for Correlated Samples The One-Way ANOVA for Correlated Samples – A correlated one-way ANOVA test is like a correlated t-Test but with three (or more!) matched participants (matched or natural set) or repeated measures (more than one score) Like the correlated t-Test, correlated One-Way ANOVAs are powerful, controlling for participant characteristics I want to focus on the repeated measures One-Way ANOVA, as it makes sense for our clothing study Interpretation  3). One-Way ANOVA for Correlated Samples The One-Way ANOVA for Correlated Samples – Again, recall our research question … “Do salespeople take longer to respond to customers dressed in sloppy clothes than those dressed in casual clothes or dressy clothes?” – Let’s reconsider this data for a repeated measures format. Imagine that researchers now approach the SAME salesclerk three times, but each approach they dress in a different style of clothing (dressy, sloppy, casual) Table 11-2, page 246 Condition (IV) T1 = Dressy T2 = Sloppy T3 = Casual Time to help (in seconds) for eight participants per condition (DV) Input these values into an SPSS file to follow along! P 1 2 3 4 5 6 7 8 T1 37 38 44 47 49 49 54 69 T2 50 46 62 52 74 69 77 76 T3 39 38 47 44 50 48 70 55 Notice we have just as many scores (24 scores) but fewer participants (only 8) Interpretation  3). One-Way ANOVA for Correlated Samples In SPSS, our file should look like this … Variable View Data View Interpretation  3). One-Way ANOVA for Correlated Samples The One-Way ANOVA for Correlated Samples – To run the ANOVA, go into Analyze, go to “General Linear Model”, and click on “Repeated Measures”. In the dialogue box, type in a variable name (for me, “Time” seemed like a good name), and then type in 3 Levels (for Dressy, Sloppy, and Casual). Then click “Add” and then “Define” – SPSS calls a repeated measures design “within subjects” Let’s see what this looks like in SPSS … Interpretation  3). One-Way ANOVA for Correlated Samples The One-Way ANOVA for Correlated Samples Interpretation  3). One-Way ANOVA for Correlated Samples Now, move your three variables (DressyTime, SloppyTime, and CasualTime) over to “Within-Subjects Variables” (Time) – Click options Select Descriptives Also, Compare Means for “Time” (use LSD) – Note we have Dressy as (1) Sloppy as (2), Casual as (3) This will be important! Interpretation  3). One-Way ANOVA for Correlated Samples Our first table includes descriptive statistics – Our data, of course, are similar to what we found before, but note that we have only 8 participants per condition – The means look different, but do they differ significantly? Interpretation  3). One-Way ANOVA for Correlated Samples There are a few other tables in the data display, including a Multivariate Tests table and Mauchly’s test of Sphericity – We’ll ignore these in Methods Two. We only have one DV, so the multivariate test is not relevant. Also, we don’t want a significant Sphericity test, as significance would violate OneWay ANOVA assumptions. Since our data does not violate Sphericity assumptions, we’ll ignore this table as well! What we are interested in is the “Tests of Within Subjects Effects” … Interpretation  3). One-Way ANOVA for Correlated Samples According to this ANOVA Table, we have significance in our data set, F(2, 14) = 19.71, p < .001 We assume Sphericity (as the prior slide noted), so use the first row. Note the df (for Time and for Error), F, and Sig Interpretation  3). One-Way ANOVA for Correlated Samples Next, look at the “Pairwise Comparison” table (“post hoc” test) This tells us that 1 differs from 2 (1 is dressy, 2 is sloppy) and 2 differs from 3 (2 is sloppy, 3 is casual). But 1 = 3 Interpretation  4). Translating Statistics into Words (Correlated) Now we need to write it up. Again, this should be familiar! “The effect of three different clothing styles on clerks’ response times was significant, F(2, 14) = 19.71, p < .001. Post hoc tests showed that clerks took longer to respond to customers dressed in sloppy clothes (M = 63.25, SD = 12.54) than to those in dressy clothes (M = 48.38, SD = 10.11) or those in casual clothes (M = 48.88, SD = 10.20). Response times did not differ between the clerks waiting on customers in dressy or casual clothing.” Interpretation  4). Translating Statistics into Words (Correlated) Now we need to write it up. Again, this should be familiar! “The effect of three different clothing styles on clerks’ response times was significant, F(2, 14) = 19.71, p < .001. Post hoc tests showed that clerks took longer to respond to customers dressed in sloppy clothes (M = 63.25, SD = 12.54) than to those in dressy clothes (M = 48.38, SD = 10.11) or those in casual clothes (M = 48.88, SD = 10.20). Response times did not differ between the clerks waiting on customers in dressy or casual clothing.” Please note the book standard deviations are wrong! (Page 259) Interpretation  5). Comparing Independent and Correlated Designs As we finish this section, keep in mind the differences between independent and correlated tests. For correlated tests … – 1). there are fewer degrees of freedom (fewer participants) – 2). the F value tends to be larger – 3). the probability chance (p value) is closer to zero – 4). the proportion of variance accounted for is larger – 5). post hoc comparisons (p values) are closer to zero Correlated tests control for variance, leading to some of these differences. Yup, lots of advantages in correlated designs! Part Four An Eye Toward The Future An Eye Toward The Future  Pause-Problem #4 (Pop Quiz) Yup, this slide again! #4 For your last Pause-Problem, I want YOU to write a multiple choice pop-quiz question based on the content of this chapter. I might use your question on a future pop quiz or actual course exam (though not this semester), so make it good! Make sure to include your correct answer and up to five possible answers! An Eye Toward The Future  An Eye Toward The Future In Chapter 11, we focused on one IV with more than two levels. But what if you need more than one IV to fully understand your topic? In Chapter 12, we will look at more complex designs, focusing on studies that have 2 or more IVs. But we will get to that in a few weeks For now … An Eye Toward The Future  An Eye Toward The Future For now I want to backtrack just a bit and look at Chapter 17 in Salkind (What to do when you’re not normal) – In our next lecture, we’ll discuss the Chi Square, a type of statistic we can use when we have dependent variables not based on the mean score – This chapter will be very important as you work on the Chi Square in your results section of your paper. An Eye Toward The Future  Finally, it is VERY, VERY, VERY important for you to read your lab presentation immediately. Since many of your papers are based on content covered in the lab, you need to know about that content sooner rather than later  So, here is your reminder to read that lab presentation immediately Chapter 17 (Salkind) What To Do When You’re Not Normal Overview of this Chapter  The Good News and the Bad News First up, the Bad News. Once again, we will look at statistics. Here, that means the Chi Square, a type of statistics we rely on when our scales are nominal or ordinal – The other Bad News is that this there are formulas and tables associated with this chapter. I know, ugh The Good News? Some of this might be a review! But you will need some of the new information here as you work on one statistical calculation for your research paper: The Chi Square Overview of this Chapter  In this chapter, we will focus on … Part One: Introduction To Non-Parametric Statistics Part Two (A): Introduction To The One-Sample Chi-Square – Part Two (B): Chi Square Test Of Independence Part Three: Computing The Chi-Square Statistic Part Four: Using The Computer To Perform A Chi-Square Test Part Five: Other Non-Parametric Tests You Should Know Part Six: An Eye Toward The Future Part One Introduction To Non-Parametric Statistics Introduction - Non-Parametric Statistics  Introduction To Non-Parametric Statistics Last semester in Research Methods and Design One (and last week in Chapter 9, Smith and Davis), we talked about normal curves and why we need normality in order to run ANOVAs, tTests, and other “parametric” tests. “Parametric tests” infer that the results obtained from a sample in the study easily applies to a population from which that sample was drawn. But such “normal” tests are based on a series of assumptions … Introduction - Non-Parametric Statistics  Introduction To Non-Parametric Statistics Four parametric test assumptions: – Assumption #1: Variances in each group are homogenous (that is, the two or more groups are similar in variability) – Assumption #2: The sample is large enough to adequately represent the population (e.g. it isn’t a biased sample) Introduction - Non-Parametric Statistics  Introduction To Non-Parametric Statistics Four parametric test assumptions: – Assumption #3: The statistical test uses interval or ratio scales of measurement (the I and R in NOIR) – Assumption #4: The characteristic under consideration is normally distributed (i.e. has a normal curve) Introduction - Non-Parametric Statistics  Introduction To Non-Parametric Statistics So what happens when/if a test violates these assumptions? – In some cases, t-Tests, ANOVAs, and other parametric tests are robust (e.g. strong enough) that the assumptions can be violated without too much hassle. Introduction - Non-Parametric Statistics  Introduction To Non-Parametric Statistics So what happens when/if a test violates these assumptions? – Non-parametric tests may be used when assumptions are violated “Non-parametric” statistics are essentially distributionfree, meaning they don’t follow the same rules as the parametric tests They don’t require homogeneity of variance and they can examine more than just interval and ratio data Introduction - Non-Parametric Statistics  Introduction To Non-Parametric Statistics So what happens when/if a test violates these assumptions? – Researchers often use non-parametric statistics used when the data set relies on frequencies or percentages (rather than scales), and we can test whether the percentages we see in a data set are what we would expect by chance alone – This takes us to one of the more common non-parametric tests, the chi square (something you’ll use for your first study this semester!) Introduction - Non-Parametric Statistics  Introduction To Non-Parametric Statistics Before we get too far into this chapter, I just want you to think about the concept of “expectations” Let’s say I go to a pet store to look at kittens, and there are dozens of them. Just looking at them from afar, what percent would you expect to be female? – About a 50 / 50 chance, right? Although we might “expect” this, we might be wrong. Chi Squares can help us see if our expectations match reality! Pop Quiz – Quiz Yourself  If you have 30 respondents identifying their political preference (i.e., Democrat, Republican, Independent), how many of each political affiliation would you expect? A). 10 B). 20 C). 30 D). 40 Pop Quiz – Quiz Yourself  If you have 30 respondents identifying their political preference (i.e., Democrat, Republican, Independent), how many of each political affiliation would you expect? A). 10 B). 20 C). 30 D). 40 Maybe, right? We SHOULD get 10 of each, but in reality there tend to be very few Independents (voters usually fall into either Democrat or Republican camps, so “10” might be too high for Independents!) Introduction - Non-Parametric Statistics  Introduction To Non-Parametric Statistics In the next part of this presentation, I want to tell you about two different types of chi squares we can run. We will split them up into two “flavors”: – Part Two (Section A): The One-Sample Chi Square This one is more FYI (though you will be tested on it! – Part Two (Section B): Chi Square Test of Independence This one is very important for your Paper II analysis! We’ll figure out how to compute each when we start Part Three Part Two (Section A) Introduction To The One-Sample Chi-Square Introduction: One-Sample Chi-Square  Introduction To One-Sample Chi-Square What is the one-sample chi-square all about? – The one sample chi-square is a non-parametric test that allows you to determine if what you observe in a frequency distribution of scores is what you would expect by chance, though this is limited to a single sample Introduction: One-Sample Chi-Square  Introduction To One-Sample Chi-Square What is the one-sample chi-square all about? – The one sample chi-square is a non-parametric test that allows you to determine if what you observe in a frequency distribution of scores is what you would expect by chance, though this is limited to a single sample – Consider ”year in college” as our “one sample” for students at FIU. For expectations, we ask, “What percent represents Freshmen, Sophomores, Juniors, and Seniors?” We can then compare our “expectations” to our “observations.” Introduction: One-Sample Chi-Square  Introduction To One-Sample Chi-Square What is the one-sample chi-square all about? – We would probably expect a few more Freshmen than other groups, right? After all, not all Freshmen will return for their senior year, and not all Sophomores return as Juniors, etc. – But generally, let’s say we expect around 25% of each class each year. If we look at the actual observations, would they be higher or lower than what we would “expect” by chance? Introduction: One-Sample Chi-Square  Introduction To One-Sample Chi-Square What is the one-sample chi-square all about? – That’s a question we can answer using the chi square At FIU, our total enrollment is around 50,000 So we might expect around 12,500 Freshmen (or one fourth of the total enrollment)? What if we found 15,000 Freshmen? Would that be outside the realm of expectation? FIU has a retention rate of 84% of Freshmen (84% of Freshmen return as Sophomores), which is high but still shows that some students are not “retained” Introduction: One-Sample Chi-Square  Introduction To One-Sample Chi-Square What is the one-sample chi-square all about? – That’s a question we can answer using the chi square The chi square tests the actual occurrences against the expected occurrences to see if they differ significantly This means that if there is no difference between what we observe and what we would expect by chance, our chi square will be close to zero Pop Quiz – Quiz Yourself  If you have 100 respondents identify their region of residence (i.e., north, south, east, or west), what would the expected frequency be for each category? A). 33 B). 50 C). 25 D). 100 Pop Quiz – Quiz Yourself  If you have 100 respondents identify their region of residence (i.e., north, south, east, or west), what would the expected frequency be for each category? A). 33 B). 50 C). 25 D). 100 But again, expectation and reality may differ a lot! Introduction: One-Sample Chi-Square  Introduction To One-Sample Chi-Square What is the one-sample chi-square all about? – As you can see, the one-sample chi square focuses on just one variable, or one sample Here, we looked at the number of students who fall into each year (Freshmen, Sophomore, Junior, or Senior) But what if we want to look at more than one variable? Well, that calls for a chi square test of independence … Part Two (Section B) The Chi-Square Test Of Independence The Chi-Square Test Of Independence  The Chi Square Test Of Independence As you just saw, we can see if the observed counts of a single variable match (or do not match) the counts we would expect by chance Often, though, you will also want to see if the observed counts across two variables match (or mismatch) the counts we would expect by chance. In this situation, you use a chi square test of independence (two samples) The Chi-Square Test Of Independence  The Chi Square Test Of Independence Go back to our Freshmen, Sophomore, Junior, and Seniors at FIU. Do you think there is a difference in terms of percentages of students in each year? – We could answer this using a one-sample chi square But do you think there might also be a difference for each of these classes between male and female students? – This question deals with two samples (year and gender), so we must answer it using a chi square test of independence The Chi-Square Test Of Independence  The Chi Square Test Of Independence Given four “years” (Freshmen, Sophomore, Junior, and Senior) and two “genders” (Male and Female), we might expect 12.5% of students to fall into each of our eight table cells: Gender Male Female Year in College Freshman Sophomore Junior 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% Senior 12.5% 12.5% Will our “observations” match our “expectations”? Let’s find out Pop Quiz – Quiz Yourself  A two-sample chi-square is also known as a ________. A). Goodness of fit test B). Test of independence C). Wilcoxon rank D). Mann-Whitney U Pop Quiz – Quiz Yourself  A two-sample chi-square is also known as a ________. A). Goodness of fit test B). Test of independence C). Wilcoxon rank D). Mann-Whitney U Part Three Computing The Chi-Square Test Statistic Computing The Chi-Square Test Statistic  Let’s focus on each test separately 1). Computing the one sample chi square test statistic 2). Computing the chi square of independence test statistic Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic The one sample chi square test compares what we observe with what we expect by chance. It uses this formula X2 =Σ (O – E )2 X2 is the chi-square value Σ is the summation sign O Is the observed frequency E is the expected frequency E Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Stastic Let’s say we get the following data from our enrollment rosters at FIU (including all online, live, MMC, and BBC students!) Freshmen Sophomores 15,000 13,500 Juniors 11,000 Seniors 10,500 Total 50,000 Time to walk through out eight research steps! I trust you recall all of these from Research Methods and Design One! Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step One: State the null and alternative research hypotheses – Our null hypothesis is that the four groups do not differ HO: PFresh = PSoph = PJunior = PSenior – Our research (alternative) hypothesis is there are differences in the proportion of occurrences in each “year” category H1: PFresh ≠ PSoph ≠ PJunior ≠ PSenior Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Two: State the level of risk – Similar to last semester, we get to set our own risk. We’ll go with the usual psychology suspects, either p < .05 or p < .01 Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Three: Select the appropriate test statistic – We are looking at categories for our one sample data set, or Freshmen, Sophomores, Juniors, and Seniors As such, we are dealing with a nominal variable, right! We need to use the mean if we want to run parametric tests like a t-Test or an ANOVA, but since we have a nominal variable, the mean is … meaningless here What would our mean even be? Something between a Freshman and a Sophomore. What is that, some kind of Freshomore? Makes no sense! Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Three: Select the appropriate test statistic – We are looking at categories for our one sample data set, or Freshmen, Sophomores, Juniors, and Seniors Given our nominal “year” variable, we have to use a nonparametric test here. The chi-square is perfect, as it can examine categorical (nominal) variables Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Four: Compute the test statistic – Consider our “year” data again (Note: I did make these up!) Freshmen Sophomores 15,000 13,500 Juniors 11,000 Seniors 10,500 Total 50,000 – To set up our chi-square calculations, we need to look at the observed frequency (tabled above), our expected frequency (there are four groups, so divide 50,000 by 4 to get 12,500 each). We need the difference, too, and some squaring! … Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Four: Compute the test statistic Year Observe Fresh. 15000 Soph. 13500 Junior 11000 Senior 10500 Total Expect 12500 12500 12500 12500 Difference (O – E)2 (O – E)2 / E Here are our observed and expected values Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Four: Compute the test statistic Year Observe Fresh. 15000 Soph. 13500 Junior 11000 Senior 10500 Total Expect 12500 12500 12500 12500 Difference 2500 1000 1500 2000 (O – E)2 (O – E)2 / E Subtract observed from the expected (ignore negative signs) Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Four: Compute the test statistic Year Observe Fresh. 15000 Soph. 13500 Junior 11000 Senior 10500 Total Expect 12500 12500 12500 12500 Difference 2500 1000 1500 2000 (O – E)2 (O – E)2 / E 6250000 1000000 2250000 4000000 Square each difference number (e.g. 2500 X 2500 = 6250000) Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Four: Compute the test statistic Year Observe Fresh. 15000 Soph. 13500 Junior 11000 Senior 10500 Total Expect 12500 12500 12500 12500 Difference 2500 1000 1500 2000 (O – E)2 (O – E)2 / E 6250000 500 1000000 80 2250000 180 4000000 320 Divide the square of each difference by its “expected” number Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Four: Compute the test statistic Year Observe Fresh. 15000 Soph. 13500 Junior 11000 Senior 10500 Total Expect 12500 12500 12500 12500 Difference 2500 1000 1500 2000 (O – E)2 (O – E)2 / E 6250000 500 1000000 80 2250000 180 4000000 320 1080 Our total chi square value is 500 + 80 + 180 + 300 = 1080 Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Five: Determine the value needed to reject the null – If you look in Appendix B (Salkind), you’ll see the chi-square table starting on page 380 – But we must first determine our degrees of freedom. For the one sample chi square, this is r – 1, where r is the # of rows In this case, we have four rows (four “years”), so r – 1 gives us 4 – 1, or 3 for our degrees of freedom Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Five: Determine the value needed to reject the null – Using df = 3, look up the critical value In this case, with a df of 3, we need to surpass a critical value of 7.82 for the p < .05 level and 11.34 to surpass the p < .01 level Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Six: Compare the obtained value and the critical value – We compare our obtained value of 1080 to our critical value of 7.82 (for p < .05) and 11.34 (for p < .01) – Is 1080 larger than either 7.82 or 11.34? – Well … Computing The Chi-Square Test Statistic  1). Computing The One Sample Chi-Square Test Statistic Step Seven / Eight: Make a decision – Since 1080 is clearly larger than our critical values, we can conclude that the null hypothesis cannot be accepted. Our observed values differ from our expected values – The “goodness of fit” (another name for the chi-square test) is not very “good” here. That is, our observed data does not “fit” the expected data Computing The Chi-Square Test Statistic  So How Do I Interpret X2(3) = 1080, p < .01 X2 represents the test statistic (Chi square) 3 is the number of degrees of freedom (r – 1, or 4 – 1 = 3) 1080 is the obtained value p < .01 indicates that the probability is less than 1% that the null hypothesis is correct across all categories by chance alone Computing The Chi-Square Test Statistic  How Would I Write Up This Result In A Results Section? “A chi-square goodness-of-fit test was performed to determine whether FIU students were equally distributed across the four years in college. Results showed that the students were not equally distributed, X2(3) = 1080, p < .01.” Pop Quiz – Quiz Yourself  If our degrees of freedom is 20, what critical value do we need to overcome to conclude that our obtained value is significant at the p < .01 level? A). 24.89 B). 31.41 C). 36.19 D). 37.57 E). 38.93 Pop Quiz – Quiz Yourself  If our degrees of freedom is 20, what critical value do we need to overcome to conclude that our obtained value is significant at the p < .01 level? A). 24.89 B). 31.41 C). 36.19 D). 37.57 E). 38.93 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic We just looked at a one sample chi square, but sometimes we have more than one variable that we may want to assess, all of which are nominal in nature For example, what if we want to see if there is a difference in “year” based on “gender” of the student. – We might get a table like this for our “expectations” for a population of 50,000 FIU students … Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Two group design Gender Males Females Freshmen 6250 6250 Sophs. 6250 6250 Juniors 6250 6250 Seniors 6250 6250 This includes 50,000 students total, or 25,000 males and 25,000 females (if you do the 50/50 split for gender). Divide 25,000 by four years, and you get 6250 per year (12.5% of 50,000 gets us to this 6250 as well!). Nice and easy, right! Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Two group design Gender Males Females Freshmen 6250 6250 Sophs. 6250 6250 Juniors 6250 6250 Seniors 6250 6250 Yeah, nothing is really easy in statistics. In fact, when you look at more than one variable, the simple “expectation” route is not really appropriate. In fact … Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Two group design Gender Males Females Freshmen 6250 6250 Sophs. 6250 6250 Juniors 6250 6250 Seniors 6250 6250 FORGET the scores above! The chi-square of independence uses a statistical calculation of the expectation, which is based on the expected value for one variable working in concert with the expected value for the second variable. Ugh. Calculations: Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Two group design – The “Real Expected” values Gender Males Females Freshmen Sophs. Juniors Seniors Do you want to know what the “Real Expected” values are? Well, here they are … Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Two group design – The “Real Expected” values Gender Males Females Freshmen 6975 8025 Sophs. 6277.5 7222.5 Juniors 5115 5885 Seniors 4882.5 5617.5 You’re probably scratching your head right now, wondering how I got these numbers. This is where some calculations come into play. Believe it or not, we need to begin with our “observed” values to calculate our “expected” values … Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Consider our “observed” values below, the values we actually observe. (Note: I made up the data below, but it is possible!) Gender Males Females Freshmen 7000 8000 Sophs. 6000 7500 Juniors 5250 5750 Seniors 5000 5500 What we need now are totals for the columns and rows … Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 ? ? ? ? ? ? ? Here’s a rearranged table that adds blank cells for each row (?) and each column (?) as well as a Column Total + Row Total (?) Let’s fill in the blank cells by doing some basic addition Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 Pretty easy, right. – Our male total is 7000 + 6000 + 5250 + 5000 = 23250 – Freshmen total is 7000 + 8000 = 15000, and so forth Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 Now multiply each row by each column and divide by total N, which will give us our expectation for each gender*year cell Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 For Freshman males, we have 15000*23250 / 50000 = 6975 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 That is, for Freshman males, our expected value is 6975! Thus we expect 6975 Freshman males. Let’s table that quickly … Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Here is our new “Expectation” (Mathematically Derived) Gender Males Females Freshmen 6975 Sophs. Juniors Seniors Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 For Soph. males, we have 13500*23250 / 50000 = 6277.5 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Here is our new “Expectation” (Mathematically Derived) Gender Males Females Freshmen 6975 And so on … Sophs. 6277.5 Juniors Seniors Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 For Junior males, we have 11000*23250 / 50000 = 5115 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 For Senior males, we have 10500*23250 / 50000 = 4882.5 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 For Freshman females, we have 15000*26750 / 50000 = 8025 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 For Soph. females, we have 13500*26750 / 50000 = 7222.5 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 For junior females, we have 11100*26750 / 50000 = 5885 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 For senior females, we have 10500*26750 / 50000 = 5617.5 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic So, this is our final set of “Expectation” data (familiar, right!) Gender Males Females Freshmen 6975 8025 Sophs. 6277.5 7222.5 Juniors 5115 5885 Seniors 4882.5 5617.5 Here is our “Observation” data. Time to calculate chi square! Gender Males Females Freshmen 7000 8000 Sophs. 6000 7500 Juniors 5250 5750 Seniors 5000 5500 Computing The Chi-Square Test Statistic G / Yr. Observe M. Fr. 7000 M. So. 6000 M. Jr. 5250 M. Sr. 5000 F. Fr. 8000 F. So. 7500 F. Jr. 5750 F. Sr. 5500 Total Expect 6975 6277.5 5115 4882.5 8025 7222.5 5885 5617.5 Difference (O – E)2 (O – E)2 / E Computing The Chi-Square Test Statistic G / Yr. Observe M. Fr. 7000 M. So. 6000 M. Jr. 5250 M. Sr. 5000 F. Fr. 8000 F. So. 7500 F. Jr. 5750 F. Sr. 5500 Total Expect 6975 6277.5 5115 4882.5 8025 7222.5 5885 5617.5 Difference 25 277.5 135 117.5 25 277.5 135 117.5 (O – E)2 (O – E)2 / E Computing The Chi-Square Test Statistic G / Yr. Observe M. Fr. 7000 M. So. 6000 M. Jr. 5250 M. Sr. 5000 F. Fr. 8000 F. So. 7500 F. Jr. 5750 F. Sr. 5500 Total Expect 6975 6277.5 5115 4882.5 8025 7222.5 5885 5617.5 Difference (O – E)2 (O – E)2 / E 25 625 277.5 77006.25 135 18225 117.5 13806 25 625 277.5 77006.25 135 18225 117.5 13806.25 Computing The Chi-Square Test Statistic G / Yr. Observe M. Fr. 7000 M. So. 6000 M. Jr. 5250 M. Sr. 5000 F. Fr. 8000 F. So. 7500 F. Jr. 5750 F. Sr. 5500 Total Expect 6975 6277.5 5115 4882.5 8025 7222.5 5885 5617.5 Difference (O – E)2 (O – E)2 / E 25 625 .089 277.5 77006.25 12.27 135 18225 3.56 117.5 13806 2.82 25 625 .078 277.5 77006.25 10.66 135 18225 3.10 117.5 13806.25 2.45 Computing The Chi-Square Test Statistic G / Yr. Observe M. Fr. 7000 M. So. 6000 M. Jr. 5250 M. Sr. 5000 F. Fr. 8000 F. So. 7500 F. Jr. 5750 F. Sr. 5500 Total Expect 6975 6277.5 5115 4882.5 8025 7222.5 5885 5617.5 Difference (O – E)2 (O – E)2 / E 25 625 .089 277.5 77006.25 12.27 135 18225 3.56 117.5 13806 2.82 25 625 .078 277.5 77006.25 10.66 135 18225 3.10 117.5 13806.25 2.45 35.042 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic So, our next step is to focus on the chi square table again to see if our obtained value of 35.042 is high enough to overcome the critical value Of course, we need to calculate the df once again. For the chi square of independence, the formula is df = (# of rows – 1) X (# of columns – 1) df = (2 – 1) X (4 – 1) df = 1 X 3 = 3 Computing The Chi-Square Test Statistic  2). Computing The Chi-Square Of Independence Test Statistic The df 3 critical value is 7.82 (p < .05) or 11.34 (p < .01) Our 35.042 is clearly over both, so we can say that our gender and year in school observations are significantly different than what we would expect by chance ay p < .01! In fact, it is significant at the p < .00001 level. How do I know that? Well, I cheated a bit and used a computer. I’ll show you that in a moment, I PROMISE! For now, some self-testing … Pop Quiz – Quiz Yourself  If we run a two factor chi square of independence looking at employment (employed versus unemployed) and parenthood (has children versus has no children), what df would we use? A). 1 B). 1 and 1 C). 2 D). 2 and 2 E). 3 Pop Quiz – Quiz Yourself  If we run a two factor chi square of independence looking at employment (employed versus unemployed) and parenthood (has children versus has no children), what df would we use? A). 1 employment (2 – 1) X parent (2 – 1) = 1 X 1 = 1 B). 1 and 1 C). 2 D). 2 and 2 E). 3 Pop Quiz – Quiz Yourself  If we run a two factor chi square of independence looking at employment (employed versus unemployed) and parenthood (has no children versus has one child versus two or more children), what df would we use? A). 1 B). 1 and 1 C). 2 D). 2 and 2 E). 3 Pop Quiz – Quiz Yourself  If we run a two factor chi square of independence looking at employment (employed versus unemployed) and parenthood (has no children versus has one child versus two or more children), what df would we use? A). 1 B). 1 and 1 C). 2 employment (2 – 1) X parent (3 – 1) = 1 X 2 = 2 D). 2 and 2 E). 3 Computing The Chi-Square Test Statistic  Pause-Problem #1 (Parametric v. Non-Parametric) Let’s see how much you have been paying attention. For your first Pause-Problem in this chapter, please tell me three things that differentiate parametric from non-parametric tests (Hint: There are actually four, so see if you can spot them all!) #1 Part Four Using The Computer To Perform A Chi-Square Test Part Four (A) 1). One Sample Chi-Square Using The Computer – The Chi Square  1). Using The Computer To Compute A Chi Square (One Sample) Let’s focus on the one sample chi square first Here, we look at only one variable (our variable is “year”, so we assess expected values for Freshmen, Sophomores, Juniors, and Seniors) – Forget about gender variable for now for this one sample chi square. We just want to see if the number of students in each year differs from what we would expect by chance. Using The Computer – The Chi Square  1). Using The Computer To Compute A Chi Square (One Sample) First, we need to enter our data into SPSS – Usually, we need one “year” cell for each student – Since we have 15,000 Freshmen, I would be entering the number 1 in the “Year” column 15,000 times! (1 = Freshmen) Sorry, I am not that crazy, so I am going to reduce this to 150 for our Freshman for these few slides … Using The Computer – The Chi Square  1). Using The Computer To Compute A Chi Square (One Sample) First, we need to enter our data into SPSS – So this SPSS data set is based on 150 Freshmen (15,000 originally), 135 Sophomores (13,500 originally), 110 juniors (11,000 originally), and 105 seniors (10,500 originally), or 500 total (50,000 originally). Using The Computer – The Chi Square  1). Using The Computer To Compute A Chi Square (One Sample) First, we need to enter our data into SPSS – Remember, this is a nominal variable, so 1 could be Seniors, 2 could be Juniors, 3 could be Freshmen, and 4 could be sophomores The actual “year” number is irrelevant and arbitrary. In fact, SPSS allows me to just look at the label if I want … Using The Computer – The Chi Square  1). Using The Computer To Compute A Chi Square (One Sample) First, we need to enter our data into SPSS – Remember, this is a nominal variable, so 1 could be Seniors, 2 could be Juniors, 3 could be Freshmen, and 4 could be sophomores The actual “year” number is irrelevant and arbitrary. In fact, SPSS allows me to just look at the label if I want … See! Using The Computer – The Chi Square  1). Using The Computer To Compute A Chi Square (One Sample) Second, we click analyze, find the “non-parametric test” option, and find the “Legacy Dialogs” option, which opens up the chi square (one sample) test. – Move your variable (“year”) to the “Test variable list” – Click “okay” Using The Computer – The Chi Square  1). Using The Computer To Compute A Chi Square (One Sample) This is the first table in our output. As you see, we get our “Observed N” for each year and our “Expected N” (Expected N is also 500 total, or 500 / 4 = 125 for each year). We also see residuals (Observed minus Expected) Using The Computer – The Chi Square  1). Using The Computer To Compute A Chi Square (One Sample) This is the second table in our output. Our df is still 3 (four years minus one, or 4 – 1 = 3) Our chi square is 10.800. Our hand calculation was 1080, but in SPSS we dealt with 500 students rather than 50,000, so just move the decimal here and you’ll see we duplicate our answer! Using The Computer – The Chi Square  1). Using The Computer To Compute A Chi Square (One Sample) How Would I Write Up This Result In A Results Section? – “A chi-square goodness-of-fit test was performed to determine whether FIU students were equally distributed across four years in college. Results showed that the students were not equally distribute, X2(3) = 10.80, p < .01.” Pop Quiz – Quiz Yourself  If there is no difference between what is observed and what is expected, your chi-square value will be: A). 0 B). 1 C). -1 D). Cannot be determined Pop Quiz – Quiz Yourself  If there is no difference between what is observed and what is expected, your chi-square value will be: A). 0 B). 1 C). -1 D). Cannot be determined Part Four (B) 2). Chi-Square Of Independence Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence What about our independent samples chi square? – IMPORTANT: In your first study (Paper II), you will compare two nominal variables (a nominal dependent variable and a nominal independent variable) to see if the observation differs by chance, so THIS test is the one to use … Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence I am going to get to the SPSS computer analysis in a moment for the chi square of independence, but I just want to mention that there are lots of free online statistical programs you can use for your data For the independent chi square analysis, I found a site that was very helpful in the calculations. – Note: The font of the numbers are small in these next tables, but they duplicate the tables we just went through! Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Lots of online programs will do this for you. For example, consider this website, where you enter your observed counts: http://www.socscistatistics.com/tests/chisquare2/Default2.aspx Blank “Starting” Screen Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Lots of online programs will do this for you. For example, consider this website, where you enter your observed counts: http://www.socscistatistics.com/tests/chisquare2/Default2.aspx Insert your gender and year (nominal variables) categories Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Lots of online programs will do this for you. For example, consider this website, where you enter your observed counts: http://www.socscistatistics.com/tests/chisquare2/Default2.aspx This is our original “observed” table for gender and year Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Lots of online programs will do this for you. For example, consider this website, where you enter your observed counts: http://www.socscistatistics.com/tests/chisquare2/Default2.aspx Ta da! Σ (E – O)2 / E to get your 35.04 chi square statistic! Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Lots of online programs will do this for you. For example, consider this website, where you enter your observed counts: http://www.socscistatistics.com/tests/chisquare2/Default2.aspx I promised to tell you how I got p = .00001? Promise kept! Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Unfortunately, dropping our sample from 50,000 down to 500 in SPSS actually impacts the chi square outcome for independent chi square tests when I run it in SPSS Since I don’t want to enter 50,000 participants into SPSS, let me show you the calculation for 500 students instead – But first, a friendly reminder of our 50,000 participants … Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Remember this table? Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 7000 8000 6000 7500 5250 5750 5000 5500 23250 26750 15000 13500 11000 10500 50000 It now becomes … Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Remember this table? Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 70 80 60 75 53 57 50 55 233 276 150 135 110 105 500 For the chi square, we multiply each row by each column / n Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Remember this table? Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 70 80 60 75 53 57 50 55 233 276 150 135 110 105 500 An N of 500 is much easier to enter into SPSS than 50,000! Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Remember this table? Gender Male Female Column Total Freshmen Sophs. Juniors Seniors Row Total 70 80 60 75 53 57 50 55 233 276 150 135 110 105 500 For Freshmen males, 150 X 233 = 34950 / 500 = 69.9 etc. Computing The Chi-Square Test Statistic  Computing The Chi-Square Of Independence Test Statistic Again, here is our “Expectation” data Gender Males Females Freshmen 69.90 80.10 Sophs. 62.91 72.09 Juniors 51.26 58.74 Seniors 48.93 56.07 Here is our “Observation”. Now, time to calculate chi square! Gender Males Females Freshmen 70 80 Sophs. 60 75 Juniors 53 57 Seniors 50 55 Using The Computer – The Chi Square G / Yr. Observe M. Fr. 70 M. So. 60 M. Jr. 53 M. Sr. 50 F. Fr. 80 F. So. 75 F. Jr. 57 F. Sr. 55 Total Expect 69.90 62.91 51.26 48.93 80.10 72.09 58.74 56.07 Difference .10 2.91 1.74 1.07 .10 2.91 1.74 2.07 (O – E)2 (O – E)2 / E .01 .00 8.47 .13 3.03 .06 1.44 .02 .01 .00 8.46 .12 3.02 .05 1.07 .02 .407 Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence What about our independent samples chi square? – Here, our chi square for independent samples gives us .407 (you can see this below from the online calculator as well as in the table in the previous slide) – Now, let’s see the SPSS version Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence What about our independent samples chi square? – In SPSS, we use a different procedure than the one sample chi square to look at a chi square test of independence. – In SPSS, first go into “Analyze”, then “Descriptive Statistics”, and find the “Crosstabs” statistical test – A BIG NOTE HERE: You will do this SPSS test for Paper II with your study one, so pay very close attention! Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence What about our independent samples chi square? – Move “Year” and “Gender” to the correct column – It doesn’t matter which goes where – Next, click the “Statistics” button Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence What about our independent samples chi square? – In “Statistics”, select the “Chi Square” as well as “Phi and Cramer’s V” – Then click continue and then okay Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence What about our independent samples chi square? – Our first table is the crosstabulation table. This simply tells us how many variables fall into each cell (70 male freshmen, 60 male sophomores, etc. Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence What about our independent samples chi square? – Our second table is more important: our Chi Square table – Focus on Pearson: It is NOT significant, with df = 3 and a value of .407 (p = .939) Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence What about our independent samples chi square? – So SPSS found the same thing as both our hand calculation for 500 students and our online calculator (some rounding is involved to get us to that .407, of course!) – The final table looks at phi. Phi is essentially a correlation, ranging from 0 to +1. A low phi (.029) means there is little correlation between our two nominal variables here Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Writing up our non-significant chi square test of independence – “There was no significant relationship between gender and year in school. X2(3) = .407, p > .05. The number of males and females did not differ from chance when taking into account their year in school.” – X2 is our chi square test and value – 3 is our degrees of freedom, or (2 – 1)*(4 – 1) = 1 X 3 = 3 – p < .05 is our significance level Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Writing up our non-significant chi square test of independence – Imagine there was significance (Pearson value was 18.26 and p = .0023). This is what that write-up would look like: “There was a significant relationship between gender and year in school. X2(3) = 18.26, p > .05.” – In your lab slideshow, I will show you an even more precise way of looking at percentages, but this is good for now Using The Computer – The Chi Square  2). Using The Computer To Compute Chi-Square of Independence Writing up our non-significant chi square test of independence – Imagine there was significance (Pearson value was 18.26 and p = .0023). This is what that write-up would look like: “There was a significant relationship between gender and year in school. X2(3) = 18.26, p > .05.” – Of course, I want to see if you can do a write-up similar to this on your own! Time for your second Pause-Problem … Using The Computer – The Chi Square  Pause-Problem #2 (Computer Output): Let’s say you design a study looking at cell phones. You ask participants what their current cell phone brand is and what brand of phone they would LIKE to have. You get this … #2 Now, consider your Chi Square table … Using The Computer – The Chi Square  Pause-Problem #2 (Computer Output): Chi Square Test of Independence #2 Using these two tables, write out the results as you would see them in a results section of an APA formatted journal article Part Five Other Nonparametric Tests You Should Know About Other Nonparametric Tests  Other Nonparametric Tests You Should Know About Sometimes we use nonparametric tests when we have nominal variables (as we saw here), but other times you might use such tests when … – 1). You do not have a normal curve (and thus you cannot use a t-Test or ANOVA, which rely on normal distributions) – 2). You have a sample size smaller than that required by either a t-Test or an ANOVA – 3). There are other violations of the assumptions underlying parametric tests Other Nonparametric Tests  Other Nonparametric Tests You Should Know About Table 17.1 in your Salkind textbook (page 360) lists several nonparametric tests, including tests like … – Categorical Data Tests (nominal data) : McNemar Test For Significance of Changes Fisher’s Exact Test Chi-Square One Sample Test (which we covered in this presentation) Other Nonparametric Tests  Other Nonparametric Tests You Should Know About Table 17.1 in your Salkind textbook (page 360) lists several nonparametric tests, including tests like … – Rank-Ordered Data Tests (ordinal data) Kologorov-Smirnov Test The Sign or Median Test Mann-Whitney U Test Wilcoxon Rank Test Other Nonparametric Tests  Other Nonparametric Tests You Should Know About Table 17.1 in your Salkind textbook (page 360) lists several nonparametric tests, including tests like … – Rank-Ordered Data Tests (ordinal data) Kruskal-Wallis One Way ANOVA Friedman Two Way ANOVA Spearmen Rank Correlation Coefficient Other Nonparametric Tests  Other Nonparametric Tests You Should Know About We are not going to cover these in this course, but just be aware that they exist (especially if you go on to become an academic!) Using The Computer – The Chi Square  Pause-Problem #3 (A Chi Square Study) Now that you have a better idea about what differentiates a one sample chi square from an independent samples (two factor) chi square, I want you to come up with one study idea that would use a one sample chi square and one study that would use an independent samples (two factor) chi square – One restriction here: You cannot use your lab study idea #2 Part Six An Eye Toward The Future An Eye Toward The Fut...