Fall 2020 Create a Biased Coin In this assignment, you will learn about NHST—Null Hypothesis Significance Testing —by trying to create a biased “coin” and then deciding how to prove that it’s biased. You will also learn about Bayesian statistics—and how sometimes it gives results that are very similar to the old-fashioned NHST approach, but not always. The “coins” in question are anything you can find that is coin-like: actual coins, plastic checkers, the metal lid of a glass jar... any round, flat disk that can be flipped or spun. It’s your job to bias this coin (or “coin”). Then you’ll have to explain how you proved that it is biased. Background The first section of this class covered descriptive statistics, which simply try to describe the numerical features of a set of data. Starting after the first exam, we began inferential statistics, which tries to make inferences (logical deductions) from one set of data to another. Typically, we want to know something about a large population of some kind, but we can’t study the entire population. Instead, we can only examine a smaller sample taken from the larger population. Then we use a series of steps (called the steps of hypothesis testing) to figure out whatever we can about the population, based on our results from the sample. This process is called NHST (Null Hypothesis Significance Testing). One of the crucial steps of NHST is to calculate a result’s level of statistical significance, or p-value, which is the probability that our result from the sample is just a chance variation. If we got what we got by chance, then our results tell us little or nothing about the larger population that we didn’t already know. NHST depends on a weird idea that is truly statistical: the null hypothesis (which is, obviously, the NH of NHST). In general, a hypothesis is a possible fact that, for the purposes of debate, is accepted as true. (Perhaps it will turn out to be really true, perhaps false, but for the moment, everyone will pretend that it’s true, and then discuss what that implies.) A null hypothesis is a particular type of statistical hypothesis that proposes a particularly boring idea: namely, that nothing interesting is going on. That’s so weird that it’s worth talking about more. In this homework, the null hypothesis is that the coins we’re creating are unbiased— that is, when flipped, shaken, spun, or toppled, they are equally likely to turn up heads and tails. That is the ordinary assumption when someone flips a coin. Another word for unbiased is fair. People think, rightly or wrongly, that it is a fair way to decide which team gets to defend which end of the sports field, for example. In statistics, a fair coin is a mythical ideal device that, when flipped, turns up heads 50% of the time and tails 50% of the time (on average)—totally randomly, unpredictably, and fairly, in the long run. In real life, a fair coin is a disk-shaped object that, when tossed or spun, lands on heads or tails close enough to 50/50 to be worth betting on. That is, if you bet on heads—or on tails—consistently, you will in the long run neither win or lose money (you’ll break even, or pretty close to it). In this homework, your job is to create a real-life unfair coin—and then to figure out how you would prove to someone that it really is unfair. Once we figure out how to do this, we will be able to figure how to reveal the opposite trait (the other side of the coin, ahem): namely, how you would prove to someone that a real-life coin is fair. How can you prove that a coin really is unbiased? How do we use a coin to make a random decision? In the real world, there are four procedures that we might use: 1. Flip it: Flick it with your thumb as you toss it into the air. On its upward and downward path, the coin spins rapidly, in a way that shows headstails-heads-tails at a high rate of speed, spinning around a horizontal axis. Let it fall to the ground, or your hand, then see whether heads or tails is facing up. 2.Shake it: Put the coin inside a medium-sized container such as a shoe box. Shake the box vigorously so that the coin bounces around many times inside. Stop shaking and allow the coin to settle flat on the bottom of the box. Open the box and report whether heads or tails is facing up. 3. Spin it: Hold the coin vertically on a hard, level surface and spin it on a vertical axis so that the coin, supported by the hard surface, shows heads-tails-heads-tails at a high rate of speed. After the spinning slows down, the coin falls flat; then report whether heads or tails is facing up. 4.Topple it: Balance the coin on its edge on a hard surface such as a table. Once the coin is stable, slap the table so that the coin topples over (falls flat). Report whether the coin comes to rest with heads or tails facing up. Your assignment: Try to bias your coin strongly enough that you can detect the bias using at least one of the four methods above. You will choose to flip (#1) or shake (#2) your coin 20 times and write down, in order, the results of Heads versus Tails. (You’ll do the same thing again in a moment, but using spin #3 or topple #4.) Look carefully at your coin (or “coin”). Is the main body curved (one side is convex and the other is concave)? Some are; some are not. Does that affect flipping, shaking, spinning, or toppling? Test your unmodified coin to find out. If you find that it’s biased, then try to bias it even more, or try to make it unbiased. Next, think about how you could prove that it is unbiased. Try to evaluate whether your coin is biased according to #3, spin it, or #4, topple it (your choice) 20 more times. Record your thoughts and your results on the data sheet at the end of this PDF. Bring to class both your coin and your completed data sheet. In class, we will discuss everyone’s results, and then perform two tests of statistical significance on the results. Name: ________________________ Class: ____________________ Specific instructions Find a coin or coin-like object. Feel free to modify it in any way you wish. Add weight to one side or the other. Bevel the edge (shave it at an angle). Find a way to bend it without breaking it. Shave part of it off. Melt it into a weird shape. Be creative! Test your coin twice. In each case, do it 20 times. In one column below, record the results (H and T, or whatever names you want to use for the two sides). For the first column, decide whether to Flip it or Shake it; circle your choice. Then do that 20 times and write down your results. Next, decide whether to Spin it or Topple it; circle your choice. Then do that 20 times, and write down your results. Flip it or Shake it Spin it or Topple it (circle one) (circle one) We’ll answer these in lecture. We’ll answer these in lecture. Binomial test results n = _____ Binomial test results n = _____ Heads ___ out of ___ Heads ___ out of ___ NHST Statistical Significance NHST Statistical Significance p = 0.______ p = 0.______ Is p ≤ .05? Yes No Is this statistically significant? Yes No Can you reject the null hypothesis? Yes No Is p ≤ .05? Yes No Is this statistically significant? Yes No Can you reject the null hypothesis? Yes No Directionality Directionality Non-directional p = .______ (two-tailed) Non-directional p = .______ (two-tailed) Directional p = .______ (one-tailed) Directional p = .______ (one-tailed) Bayesian test BF10 = _______ Bayesian test BF10 = _______ Is this > 3? Yes No Is this > 10? Yes No Is it < 1/3? Yes No Is it < 1/10? Yes No Is this > 3? Yes No Is this > 10? Yes No Is it < 1/3? Yes No Is it < 1/10? Yes No Although this is an online class, I will use ConferZoom to conduct a classroom-like experience. To prepare for that session, complete the coin-biasing as described above, and fill in your data in the two middle columns of the previous page. We will fill in the blanks in the two outside columns during the class. If you miss the live class, you can watch the recorded session and use it as a guide for completing your page. Try to answer the following questions, which we will also discuss at the lecture. Write your answers below, so you’ll be sure to have something to contribute to the discussion. Answering these questions is not what you will be graded on; this is just to help you think about the class discussion questions in advance. 1. What is the null hypothesis for this experiment? (Hint: See page 1.) 2. What is the alternative hypothesis? (Hint: The opposite of answer 1.) 3. At the end of the math (which in this case means filling in the “Binomial Test” results), should we perform a directional test or a non-directional test? Why? 4. Is it possible to prove, absolutely positively 100%, that a coin is biased? 5. Instead of absolute 100% proof, is it possible to show that it is extremely likely that a coin is biased? 6. Now consider the converse question: how to prove that a given coin is unbiased. Suppose you test it for bias and it passes the first test. Does that prove that it’s unbiased? What if you test it a second time, and it passes the second test, too? 7. It is very common for wagers (bets) to be decided by a flip of a coin. Given that a coin can be biased, why are people willing to trust this? Why don’t we fear that the flipper is using a biased coin? When you created a biased coin, were you able to detect the bias in the flipping you performed? 8. It is not very common for wagers to be decided by spinning a coin. When you created a biased coin, were you able to detect a bias by the spinning you did for this homework? Might that explain why people don’t spin coins very often? 9. Mostly we performed tests using NHST. Nowadays, many statisticians prefer Bayesian methods. In this experiment, would you have reached different conclusions between the NHST p-value (p < .05) and the Bayes Factor (BF10 > 3 or > 10)? 10. NHST can reject the null hypothesis, but if there’s not enough evidence to do that, it can only say that “we failed to reject the null hypothesis”. But Bayesian statistics can classify its results into three categories: (1) Alternative hypothesis is supported. (2) Null hypothesis is supported. (3) Not enough evidence; neither is supported strongly. Which of the following matches the data you gathered? (1) Support alternative: Bayes Factor (BF10 > 3 or > 10) (2) Support null: Bayes Factor (BF10 < 0.33 or < 0.10) (one-third or one-tenth) (3) Support neither: Bayes Factor (1/3 < BF10 < 3)