Chapter 9 •
How Good Is the Evidence 111
Clues for Evaluating Research Studies
Apply the following questions to research findings to determine whether the
findings are dependable evidence.
1. What is the quality of the source of the report? Usually, the most dependable
is not accepted until it has been reviewed by a series of relevant experts.
reports are those published in peer-reviewed journals, those in which a study
Usually—but not always the more reputable the source, the better designed
the study. So, try to find out all you can about the reputation of the source.
2. Other than the quality of the source, are there other clues included in the
communication suggesting the research was well done? For example, does the
report detail any special strengths of the research? Unfortunately, most reports
of research findings encountered in popular magazines, newspapers, television
reports, and blogs fail to provide sufficient detail about the research to warrant
our judgment of the research quality.
3. How recently was the research conducted, and are there any reasons to believe
that the findings might have changed over time? Many research conclusions
ange over time. For example,
the causes of depression, crime, or heart dis-
e in 1980 may be quite different from those in 2014.
ve the study's findings been replicated by other studies? When an association
is repeatedly and consistently found in well-designed studies-for example,
the link between smoking and cancer-then there is reason to believe it, at
least until those who disagree can provide persuasive evidence for their point
of view.
5. How selective has the communicator been in choosing studies? For exam-
ple, have relevant studies with contradictory results been omitted? Has the
researcher selected only those studies that support his point?
6. Is there any evidence of strong-sense critical thinking? Has the speaker or writer
showed a critical attitude toward earlier research that was supportive of her
point of view? Most conclusions from research need to be qualified because
of research limitations. Has the communicator demonstrated a willingness to
qualify?
7. Is there any reason for someone to have distorted the research? We need to be
wary of situations in which the researchers need to find certain kinds of results.
8. Are conditions in the research artificial and therefore distorted? Always ask,
"How similar are the conditions under which the research study was con-
ducted to the situation the researcher is generalizing about?"
9. How far can we generalize, given the research sample? Because this is such an
important issue, we discuss it in depth in our next section.
10. Are there any biases or distortions in the surveys, questionnaires, ratings, or
other measurements that the researcher uses? We need to have confidence that
the researcher has measured accurately what she has wanted to measure. The
problem of biased surveys and questionnaires is so pervasive in research that
we discuss it in more detail in a later section.
96
Chapter 8 • How Good Is the Evidence
EXHIBIT 8-1 Major kinds of Evidence
✓ intuition
✓ personal experiences
✓ case examples
✓ testimonials
✓ appeals to authorities or experts
personal observations
✓ research studies
✓ analogies
“Does an author's evidence provide dependable support for her claim?” Thus,
we begin to evaluate evidence by asking, “How good is the evidence?" Always
keep in the back of your mind that no evidence will be a slam dunk that gets
the job done conclusively. You are looking for better evidence; searching for
oltarether wonderful evidence will be frustrating.
this chapter and in Chapter 9, we examine the kinds of questions
ask of each type of evidence to help us decide its quality. Kinds of
ce examined in this chapter are intuition, personal experiences, case
uples, testimonials, and appeals to authority.
INTUITION AS EVIDENCE
"I just sense that Janette is the right girl for me, even though my
friends think we're a bad match."
"I just have this feeling that Senator Ramirez will surprise the poll-
sters and win the election."
"I can tell immediately that this slot machine is going to be a winner
for me today."
When we use intuition to support a claim, we rely on “common sense," or on
our "gut feelings," or on hunches. Listen to Jewell celebrating intuition as a
source of understanding:
Follow your heart
Your intuition
It will lead you in the right direction
Let go of your mind
Your Intuition
It's easy to find
--Jewel, "Intuition"
258 | THIS CHANGES EVERYTHING
C
ate in depth, and each presents its own particular governing challenges
for the next three days
, we will zero in on the geoengineering methods the
scientists here consider most plausible and promising. These involve vari
ous means of injecting particles into the atmosphere in order to reflect more
sunlight back to space, thereby reducing the amount of heat that reaches
the earth. In geoengineering lingo, this is known as Solar Radiation Man
agement (SRM)-since these methods would be attempting to literally
"manage the amount of sunlight that reaches earth.
There are various possible sun-dimming approaches. The most gleefully
sci-fi is space mirrors, which is quickly dismissed out of hand. Another is
"cloud brightening": spraying seawater into the sky (whether from fleets
of boats or from towers on shore) to create more cloud cover or to make
clouds more reflective and longer lasting. The most frequently discussed
option involves spraying sulfate aerosols into the stratosphere, whether via
specially retrofitted airplanes or a very long hose suspended by helium bal-
loons (some have even suggested using cannons).
The choice to focus exclusively on SRM is somewhat arbitrary given
that ocean fertilization experiments have been conducted on several oc-
casions, including a heavily reported "rogue" test off the coast of British
Columbia in 2012. But SRM is attracting the lion's share of serious scien-
tific interest: sun blocking has been the subject of over one hundred peet-
reviewed papers, and several high-level research teams are poised to run
open-air field trials, which would test the mechanics of these schemes using
ships, planes, and very long hoses. If rules and guidelines aren't developed
soon (including, as some are suggesting, banning field tests outright), we
could end up with a research Wild West.
H
fe
PA
ch
to
te
w
Spraying sulfate into the stratosphere is often referred to as "the Pinatubo
Option," after the 1991 eruption of Mount Pinatubo in the Philippines
Most volcanic eruptions send ash and gases into the lower atmosphere
where sulfuric acid droplets are formed that simply fall down to earth. (That
was the case, for instance, with the 2010 Icelandic volcano that grounded
co
he
many European flights.) But certain, much rarer eruptions Mount Pa
ha
tubo among them-send high volumes of sulfur dioxide all the way up to
When that happens, the sulfuric acid droplets don't fall back down: they
remain in the stratosphere, and within weeks can circulate to surround the
CC
the stratosphere.
SC
DIMMING THE SUN 1 257
governed? What rules should researchers follow? What bodies, if any, will
regulate these experiments? National governments? The United Nations?
What constitutes "good governance" of geoengineering? To answer these
questions and others, the society has teamed up with two cosponsors for
the retreat: the World Academy of Sciences based in Italy, which focuses
on promoting scientific opportunities in the developing world, and the
Environmental Defense Fund, which has described geoengineering as a
ming.com
to
ives, 2008
les, 1895
"bridging tool” (much as it has described natural gas). That makes this
conference both the most international gathering about geoengineering to
date, and the first time a major green group has publicly offered its blessing
the exploration of radical interventions into the earth's climate system
as a response to global warming.
The venue for this futuristic discussion is an immaculately restored
sixty-two-room redbrick Georgian mansion called Chicheley Hall, once
a set in a BBC production of Pride and Prejudice, and the Royal Society's
newly acquired retreat center. The effect is wildly anachronistic: the es-
tate's sprawling bright green lawns, framed by elaborately sculpted hedges,
seem to cry out for women in corseted silk gowns and parasols discussing
their suitors-not disheveled scientists discussing a parasol for the planet.
And yet geoengineering has always had a distinctly retro quality, not quite
steampunk, but it definitely harkens back to more confident times, when
taking control over the weather seemed like the next exciting frontier of
scientific innovation—not a last-ditch attempt to save ourselves from in-
cineration.
After dinner, consumed under towering oil paintings of plump-faced
men in silver wigs, the delegates are invited to the wood-paneled library.
bout gever
a
and
torals
among
t scientit
reduction
There, about thirty scientists, lawyers, environmentalists, and policy wonks
gather for the opening “technical briefing" on the different geoengineering
an B. In
to devot
Ods might
schemes under consideration. A Royal Society scientist takes us through a
slide show that includes “fertilizing” oceans with iron to pull carbon out of
Cale ene
maybe
vent of
the atmosphere; covering deserts with vast white sheets in order to reflect
sunlight back to space; and building fleets of machines like the ones com-
for Richard Branson's Earth Challenge that would suck carbon out
peting
of the air.
The scientist explains that there are too many such schemes to evalu-
us: How
nent
8
gover
regula
What
quest
DIMMING THE SUN
che re
The Solution to Pollution Is ... Pollution?
on pi
Envir
"brid
confe
date,
"Geoengineering holds forth the promise of addressing global warming com-
cerns for just a few billion dollars a year.”
to th
as a r
T
-Newt Gingrich, former speaker of the U.S. House of Representatives, 2008-
sixty-
a set
newly
tate's
"Our science is a drop, our ignorance a sea."
-William James, 18952
seem
their
And
steam
It's March 2011 and I have just arrived at a three-day retreat about gever
gineering in the Buckinghamshire countryside, about an hour and a ba?
northwest of London. The meeting has been convened by the Royal Soc
ety, Britain's legendary academy of science, which has counted among is
takin:
scient
cinera
fellows Isaac Newton, Charles Darwin, and Stephen Hawking.
А
men
There
gathe
scher
slide
In recent years, the society has become the most prominent scientit
organization to argue that, given the lack of progress on emission reduction
the time has come for governments to prepare a technological Plan B. Ini
report published in 2009, it called upon the British government to dever
significant resources to researching which geoengineering methods maha
prove most effective. Two years later it declared that planetary-scale en
neering interventions that would block a portion of the sun's rays "mark
the only option for reducing global temperatures quickly in the event of a
narrow focus: How
should research into geoengineering, as well as eventual deployment,
the a
sunlig
petin
of the
TH
k
climate emergency."
The retreat in Buckinghamshire has a
Purchase answer to see full
attachment