10 SO FAR. 1 have been addressing you rather as if you were a pirate with a yen for instruction in the finer points of cutlass work. In this concluding chapter 111 drop that literary device. 111 face up to the serious purpose that I like to think lurks just beneath the surface of this book: explaining how to look a phony statistic in the eye and face it down; alJd lJO less imporlant. bow to n:cognize sound and usable data in that wilderness of fraud to which the previous chapters have been largely devoted. Not aU the statistical in£onnation that you may come upon can be tested with the sureness of chemical analysis or of what goes on in an assayer'$lahoratory. Rut you can prod the stuff with five Simple questions. and by finding the answers avoid learning a remarkable lot that isn't so. HOW TO TALX BACK TO A STATISTIC About the first thing to look for is bias-the laboratory with something to prove for the sake of a theory. a reputation, or a fee; the newspa?er whose aim is a good story; labor or management with a wage level at stake. Look for conscious bias. The method may be direct misstatement or it may be ambiguous statement that serves as well and cannot be convicted. It may be selection of favorahle data and snppression of unfavorable. Units of measurement may be shifted, as with the practice of using one year for one comparison and sliding over to a more favorable year for another. An improper measure may be used: ~ lncan where a median would be more infonnative (perhaps all too infonnative), with the trickery covered by the unqualified word "average.Look sharply for unconscious bias. It is often more dangerous. In the charts and predictions of many statisticians and economists in 1928 it operated to produce remarkable things. The cracks in the economic structure were joyonsly overlooked, and all sorts of evidence was adduced and statistically supported to show that we had no more than entered the stream of prosperity. It may take at least a second look to find out who-saysso. The who may be hidden by what Stephen Potter, the Lif8tno:tl8hip man. would probably call the "O.K. name." Anything smacking of the medical professiotl is an O.K. D8JD.e. Scientilic laboratories have O.K. names. So do BOW TO LIE WITH STATISTICS ~--a:.-,; BiG NO$S ........ 0 .. THR~A.T MAN 0 .. ~-----~ HOW TO TALK BACK TO A STATISTIC 125 colleges, especially universities, more especially ones eminent in technical work. The writer who proved a few chapters back that higher education jeopardizes a girl's chance to marry made good use of the O.K. name of Cornell. Please note that while the data came from Cornell, the conclusions were entirely the writer's own. But the O.K. name helps you carry away a misimpression of "Cornell University says .. :' When an O.K. name is cited, make sure that the authority stands behind the infonnation, not merely somewhere alongside it. You may have read a proud announcement by the Chicago Journal of Commerce. That publication had made a survey. Of 169 corporations that replied to a poll on pricEll 60uging and hoarding, two-thirds declared that they were ahsorbing price increases produced by the Korean war. "The survey shows," said the Journal (look sharp whenever you meet those words!), "that corporations have done exactly the opposite of what the enemies of the American business system have charged." This is an obvious place to ask, "Who says so?" since the Journal of Commerce mighl be regarded as an interested party. It is also a splendid place to ask our second test question: It turns out that the Journal had begun by sending its questionnaires to 1,200 large companies. Only fourteen HOW TO LIE WITH STATISTICS per cent had replied. Eighty-six per cent had not cared to say anything in public on whether they were hoarding or price gouging. The Journal had put a remarkably good face On things, but the fact remains that there was little to brag about. It came down to this: Of 1,200 companies polled, nine per cent said they had not raised prices, five per cent said they had, and eighty-six per cent wouldn't say. Those that had replied constituted a sample in which bias might be suspected. Wat~h Ollt for evidence of 11 !>iased sample, one that has been selected improperly or-as with this one-has selected itself. Ask the question we dealt with in an early chapter: Is the sample large enough to pennit any reliable conclusion? Similarly with a reported correlation: Is it big enough to mean anything? Are there cnough cases to add up to any sign!ficance? You cannot, as a casual reader, apply tests of significance Or come to exact conclusions as to the adequacy of a sample. On a good many of the things you see reported, however, you will be able to tell at a glance -a good long glance, perhaps-that there just weren't enough cases to convince any reasoning person of anything. BOW TO TALI: BACK TO A STATISTIC 'IZJ 'kI/ud'4. MJUi",? You won't al~'3Ys be told how many cases. The absence of such a figure. particularly when the source is an interested one, is enough to throw suspicion on the whole thing. Similarly a correlation given without a measure of reliability (probable error, standard error) is not to be taken very seriously. Watch out for an average. variety unspecified, in any matter where mean and median might be expected to WHee substantially. Many figures lose meaning because a comparison is missing. An article in Look magazine says. in connection HOW TO LIE WITH STATISTICS with Mongolism, that "one study shows that in 2,800 cases. Over haH of the mothers were 35 or over," Getting any meaning from this depends upon your knowing something about the ages at which women in general produce babies. Few of us know things like that. Here is an exb"act from the New Yor'ker magazine's "Letter from London" of January 31,1953. The Ministry of Health's recently published figures showing that in the week of the great fog the death rate for Greater London jumped by twenty-eight hundred were a shock to the public, which is used to regarding Britain's unpleasant climatic effects as nuisances rather than as killers... , The extraordinary lethal properties of this winter's prize visitation" , But how lethal was the visitation? Was it exceptional for the death rate to be that much higher than usual in a week? All such things do vary. And what about ensuing weeks? Did the death rate drop below average, indicating that if the fog killed people they were largely those who would have died shortly anyway? The figure sounds impressive, but the absence of other figures takes away most of its meaning. Sometimes it is percentages that are given and raw figures that are missing, and this can be deceptive too. Long ago, when Johns Hopkins University had Just begun to admit WOmen students, someone not particularly enamored of coeducation reported a real shocker: Thirtythree and one-third per cent of the women at Hopkins had married faculty membersl The raw figures gave a clearer picture. There were three women enrolled at the BOW TO TALIC BACK: TO A STATISTIC I~ time. and one of them had married a faculty man. A couple of years ago the Boston Chamber of Commerce ehose its American Women of Achievement. Of the sixteen among them who were also in Who's Who, it was announced that they had "sixty academic degrees and eighteen children:' That sounds like an infonnative picture of the group 1D1tn you discover that among the women were Dean Virginia Gildersleeve and Mrs. Lillian M. Gilbreth. Those two had a full third of the degrees between them. And Mrs. Gilbre~ ")f course, supplied two-thirds of the childrell. A corporation was able to announce that its stock was held by 3,003 persons, who had an average of 660 shares each. This was true. It was also true that of the two mnlion shares of stock in the corporation three men held three-quarters and three thousand persons held the other one-fourth among them. If you are handed an index, you may ask what's missing there. It may be the base, a base chosen to give a distorted picture. A national labor organization once showed that indexes of profits and production had risen much more :rapidly after the depression than an index of wages had 13° HOW TO LIE WITH STATISTICS As an argument for wage increases this demonstration lost its potency when someone dug out the missing figures. It could be seen then that profits had been almost bound to rise more rapidly in percentage than wages simply because profits had reached a lower point, giving a smaller base. Sometimes what is missing is the factor that caused a change to occur. This omission leaves the implication that some other, more desired, factor is responsible. Figures published one year attempted to show that business was on the upgrade by pointing out that April retail sales were greater than in the year before. What was mlssing was the fact that Easter had come in March in the earlier year and in April in the later year. A report of a great increase in deaths from cancer in the last quarter-century is misleading unless you know how much of it is a product of such extraneous factors as these: Cancer is often listed now where "causes unlmown" was fonnerly used; autopsies are more frequent, giving surer diagnoses; reporting and compiling of medical statistics are more complete; and people more frequently reach the most susceptible ages now. And if you are looking at total deaths rather than the death rate, don't neglect the fact that there are more people now than there used to be. HOW TO TALK BACK TO A STATISTIC 131 When assaying a statistic, watch out for a switch som~ where between the raw figure and the conclusion. One thing is all too often reported as another. /" ~ just indicated, more reported cases of a disease are not always the same thing as more cases of the disease. A straw-vote victory for a candidate is not always negoti· able at the polIs. An expressed preference by a ··cross section" of a magazine's readers for arlicles on world affairs is no final proof that they would read the articles if they were published. Encephalitis cases reported in the central valley of California in 1952 were triple the figure for the worst previous year. Many alarmed residents shipped their ~hi1rlren away. But when the reckoning was in, there had been no great increase in deaths from sleeping sickness. What had happened was that state and federal health people had come in in great numbers to tackle a long-time prob. lem; as a result of their efforts a great many low-grade cases were recorded that in other years would have been overlooked, possibly not even recognized. It is all reminiscent of the way that Lincoln Steffens and Jacob A. Riis, as New York newspapennen, once created a crime wave. Crime cases in the papers reached such proportions, both ill nUll1b~n and in space and big type given to them. that the public demanded action. Theodore Roosevelt, as president of the reform Police Board, was HOW TO LIE WITH STATISTICS seriously embarrassed. He put an end to the crime wave simply by asking Steffens and Riis to layoff. It had all corne about Simply because the reporters, led by those two, had got into competition as to who could dig up the most burglaries and whatnot. The official police record showed no increase at all. "The British male over 5 years of age soaks himself in a hot tub on an average of 1.7 times a week in the winter and 2.1 times in the summer," says a newspaper story. "British women average 1.5 baths a week in the winter and 2.0 in the 8ummer:" The source is a Ministry of Works hot-water survey of 4
Purchase answer to see full attachment