Research Methods in Psychology
Research Methods in Psychology
3rd American Edition
Paul C. Price, Rajiv S. Jhangiani, I-Chant A. Chiang, Dana C. Leighton, and
Carrie Cuttler
Copyright:2017 by Paul C. Price, Rajiv Jhangiani, I-Chant A. Chiang, Dana C. Leighton, & Carrie Cuttler.
This textbook is an adaptation of one written by Paul C. Price (California State University, Fresno) and adapted by The Saylor
Foundation under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License without attribution as requested by the
work’s original creator or licensee. The original text is available here: http://www.saylor.org/site/textbooks/
This adaptation constitutes the second American edition, and incorporates the second Canadian edition by Rajiv S. Jhangiani
(Kwantlen Polytechnic University) and I-Chant A. Chiang (Quest University Canada) and is licensed under a Creative Commons
Attribution-NonCommercial-ShareAlike 4.0 International License.
The second U.S. edition was authored by Dana C. Leighton (Southern Arkansas University) and is licensed under a Creative
Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Revisions in the current edition include:
Throughout: Reversion of spelling from Canadian English to U.S. English
Reversion of Canadian ethics chapter to the original U.S. chapter
Cover photo: “Great Wave off Kanagawa” after Katsushika Hokusai (葛飾北斎) is public domain.
The third U.S. edition was authored by Carrie Cuttler (Washington State University) in 2017 and is licensed under a Creative
Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Revisions in the current edition include general
reorganization, language revision, spelling, formatting, additional video links, and examples throughout. More specifically, the
overall model section was moved from Chapter 1 to Chapter 2, new sections were added to Chapter 1 on methods of knowing and
goals of science, and a link on the replication crisis in psychology was added to Chapter 1. Chapter 2 was also reorganized by
moving the section on reviewing the research literature to earlier in the chapter and taking sections from Chapter 4 (on theories
and hypotheses), moving them to Chapter 2, and cutting the remainder of Chapter 4. Sections of Chapter 2 on correlation were
also moved to Chapter 6. New sections on characteristics of good research questions, an overview of experimental vs. nonexperimental research, a description of field vs. lab studies, and making conclusions were also added to Chapter 2. Chapter 3 was
expanded by adding a definition of anonymity, elaborating on the Belmont Report (the principles of respect for persons and
beneficence were added), and adding a link to a clip dispelling the myth that vaccines cause autism. Sections from Chapter 4 (on
defining theories and hypotheses) were moved to Chapter 2 and the remainder of the previous Chapter 4 (on phenomenon,
theories, and hypotheses) was cut. Chapter 5 was reorganized by moving the sections on four types of validity, manipulation
checks, and placebo effects to later in the chapter. Descriptions of single factor two-level designs, single factor multi-level
designs, matched-groups designs, order effects, and random counterbalancing were added to Chapter 5 and the concept of
statistical validity was expanded upon. Chapter 6 was also reorganized by moving sections describing correlation coefficients from
Chapters 2 and 12 to Chapter 6. The section of the book on complex correlation was also moved to Chapter 6 and the section on
quasi-experiments was moved from Chapter 6 to its own chapter (Chapter 8). The categories of non-experimental research
described in Chapter 6 were change to cross-sectional, correlational, and observational research. Chapter 6 was further expanded
to describe cross-sectional studies, partial correlation, simple regression, the use of regression to make predictions, case studies,
participant observation, disguised and undisguised observation, and structured observation. The terms independent variable and
dependent variable as used in the context of regression were changed to predictor variable and outcome/criterion variable
respectively. A distinction between proportionate stratified sampling and disproportionate stratified sampling was added to
Chapter 7. The section on quasi-experimental designs was moved to its own chapter (Chapter 8) and was elaborated upon to
include instrumentation and testing as threats to internal validity of one-group pretest-posttest designs, and to include sections
describing the one-group posttest only design, pretest-posttest nonequivalent groups design, interrupted time-series with
nonequivalent groups design, pretest-posttest design with switching replication, and switching replication with treatment removal
designs. The section of Chapter 9 on factorial designs was split into two sections and the remainder of the chapter was moved or
cut. Further, examples of everyday interactions were added and a description of simple effects was added to Chapter 9. The
section on case studies that appeared in Chapter 10 was edited and moved to Chapter 6. Further, labels were added to multiplebaseline across behaviors, settings, and participants designs, and a concluding paragraph on converging evidence was added to
Chapter 10. Only minor edits were made to the remaining chapters (Chapters 11, 12, and 13).
Contents
About This Book .................................................................................................................................................... i
Acknowledgements .............................................................................................................................................. iii
Preface ................................................................................................................................................................. iv
Chapter 1: The Science of Psychology .................................................................................................................. 5
1 1.1 Methods of Knowing .................................................................................................................................... 6
2 1.2 Understanding Science ............................................................................................................................... 8
3 1.3 Goals of Science ........................................................................................................................................ 12
4 1.4 Science and Common Sense ...................................................................................................................... 14
5 1.5 Experimental and Clinical Psychologists ................................................................................................... 17
Chapter 2: Overview of the Scientific Method ................................................................................................... 20
6 2.1 A Model of Scientific Research in Psychology .......................................................................................... 21
7 2.2 Finding a Research Topic .......................................................................................................................... 23
8 2.3 Generating Good Research Questions ....................................................................................................... 29
9 2.4 Developing a Hypothesis ........................................................................................................................... 32
10 2.5 Designing a Research Study .................................................................................................................... 36
11 2.6 Analyzing the Data .................................................................................................................................. 39
12 2.7 Drawing Conclusions and Reporting the Results .................................................................................... 41
Chapter 3: Research Ethics ................................................................................................................................... 43
13 3.1 Moral Foundations of Ethical Research .................................................................................................. 44
14 3.2 From Moral Principles to Ethics Codes ................................................................................................... 49
15 3.3 Putting Ethics Into Practice .................................................................................................................... 56
Chapter 4: Psychological Measurement .............................................................................................................
16 4.1 Understanding Psychological Measurement ...........................................................................................
17 4.2 Reliability and Validity of Measurement .................................................................................................
18 4.3 Practical Strategies for Psychological Measurement ..............................................................................
60
61
67
73
Chapter 5: Experimental Research ......................................................................................................................
19 5.1 Experiment Basics ...................................................................................................................................
20 5.2 Experimental Design ...............................................................................................................................
21 5.3 Experimentation and Validity ..................................................................................................................
22 5.4 Practical Considerations .........................................................................................................................
77
79
83
89
93
Chapter 6: Nonexperimental Research .............................................................................................................
23 6.1 Overview of Non-Experimental Research .............................................................................................
24 6.2 Correlational Research ..........................................................................................................................
25 6.3 Complex Correlation .............................................................................................................................
26 6.4 Qualitative Research .............................................................................................................................
27 6.5 Observational Research ........................................................................................................................
Research Methods in Psychology
101
102
107
113
117
121
Chapter 7: Survey Research ................................................................................................................................ 128
28 7.1 Overview of Survey Research ................................................................................................................ 129
29 7.2 Constructing Surveys ............................................................................................................................ 132
30 7.3 Conducting Surveys .............................................................................................................................. 139
Chapter 8: Quasi-Experimental Research ......................................................................................................... 144
31 8.1 One-Group Designs ............................................................................................................................... 145
32 8.2 Non-Equivalent Groups Designs ............................................................................................................ 150
Chapter 9: Factorial Designs .............................................................................................................................. 154
33 9.1 Setting Up a Factorial Experiment ........................................................................................................ 155
34 9.2 Interpreting the Results of a Factorial Experiment .............................................................................. 160
Chapter 10: Single-Subject Research ................................................................................................................
35 10.1 Overview of Single-Subject Research .................................................................................................
36 10.2 Single-Subject Research Designs ........................................................................................................
37 10.3 The Single-Subject Versus Group “Debate” ........................................................................................
168
169
172
180
Chapter 11: Presenting Your Research .............................................................................................................
38 11.1 American Psychological Association (APA) Style ................................................................................
39 11.2 Writing a Research Report in American Psychological Association (APA) Style ................................
40 11.3 Other Presentation Formats ................................................................................................................
184
185
192
204
Chapter 12: Descriptive Statistics .....................................................................................................................
41 12.1 Describing Single Variables ................................................................................................................
42 12.2 Describing Statistical Relationships ....................................................................................................
43 12.3 Expressing Your Results ......................................................................................................................
44 12.4 Conducting Your Analyses ...................................................................................................................
210
211
220
229
237
Chapter 13: Inferential Statistics ......................................................................................................................
45 13.1 Understanding Null Hypothesis Testing .............................................................................................
46 13.2 Some Basic Null Hypothesis Tests ......................................................................................................
47 13.3 Additional Considerations ...................................................................................................................
48 13.4 From the “Replicability Crisis” to Open Science Practices .................................................................
241
242
248
261
268
Research Methods in Psychology
Research Methods in Psychology
1
About This Book
This textbook is an adaptation of one written by Paul C. Price (California State University, Fresno) and adapted by
The Saylor Foundation under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License without
attribution as requested by the work’s original creator or licensee. The original text is available here:
http://www.saylor.org/site/textbooks/
The first Canadian edition (published in 2013) was authored by Rajiv S. Jhangiani (Kwantlen Polytechnic University)
and was licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License. Revisions included
the addition of a table of contents, changes to Chapter 3 (Research Ethics) to include a contemporary example of an
ethical breach and to reflect Canadian ethical guidelines and privacy laws, additional information regarding online
data collection in Chapter 9 (Survey Research), corrections of errors in the text and formulae, spelling changes from
US to Canadian conventions, the addition of a cover page, and other necessary formatting adjustments.
The second Canadian edition was co-authored by Rajiv S. Jhangiani (Kwantlen Polytechnic University) and I-Chant A.
Chiang (Quest University Canada) and is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike
4.0 International License. Revisions included: (throughout) language revision, spelling & formatting, additional video
links and website links, interactive visualizations, figures, tables, and examples; (Chapter 1) the Many Labs
Replication Project; (Chapter 2) double-blind peer review, contemporary literature databases, how to read academic
papers; (Chapter 3) Canadian ethics; (Chapter 4) laws, effects, theory; (Chapter 5) fuller description of the MMPI,
removal of IAT, validity descriptions; (Chapter 6) validity & realism descriptions, Latin Square design; (Chapter 7)
Mixed-design studies, qualitative-quantitative debate; (Chapter 8) 2 × 2 factorial exercise; (Chapter 9) Canadian
Election Studies, order and open-ended questions; (Chapter 13) p-curve and BASP announcement about banning pvalues; “replicability crisis” in psychology; (Glossary) added key terms.
The second U.S. edition was authored by Dana C. Leighton (Southern Arkansas University) and is licensed under a
Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Revisions in the current edition
include:
Throughout: Reversion of spelling from Canadian English to U.S. English
Cover photo: “Great Wave off Kanagawa” after Katsushika Hokusai (葛飾北斎) is public domain.
Year of Publication: 2017
The third U.S. edition was authored by Carrie Cuttler (Washington State University) in 2017 and is licensed under a
Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. Revisions in the current edition
include general reorganization, language revision, spelling, formatting, additional video links, and examples
throughout. More specifically, the overall model section was moved from Chapter 1 to Chapter 2, new sections were
added to Chapter 1 on methods of knowing and goals of science, and a link on the replication crisis in psychology
was added to Chapter 1. Chapter 2 was also reorganized by moving the section on reviewing the research literature
to earlier in the chapter and taking sections from Chapter 4 (on theories and hypotheses), moving them to Chapter 2,
and cutting the remainder of Chapter 4. Sections of Chapter 2 on correlation were also moved to Chapter 6. New
sections on characteristics of good research questions, an overview of experimental vs. non-experimental research, a
description of field vs. lab studies, and making conclusions were also added to Chapter 2. Chapter 3 was expanded
by adding a definition of anonymity, elaborating on the Belmont Report (the principles of respect for persons and
beneficence were added), and adding a link to a clip dispelling the myth that vaccines cause autism. Sections from
Chapter 4 (on defining theories and hypotheses) were moved to Chapter 2 and the remainder of the previous
Chapter 4 (on phenomenon, theories, and hypotheses) was cut. Chapter 5 was reorganized by moving the sections
on four types of validity, manipulation checks, and placebo effects to later in the chapter. Descriptions of single
factor two-level designs, single factor multi-level designs, matched-groups designs, order effects, and random
counterbalancing were added to Chapter 5 and the concept of statistical validity was expanded upon. Chapter 6 was
also reorganized by moving sections describing correlation coefficients from Chapters 2 and 12 to Chapter 6. The
section of the book on complex correlation was also moved to Chapter 6 and the section on quasi-experiments was
i
Research Methods in Psychology
moved from Chapter 6 to its own chapter (Chapter 8). The categories of non-experimental research described in
Chapter 6 were change to cross-sectional, correlational, and observational research. Chapter 6 was further expanded
to describe cross-sectional studies, partial correlation, simple regression, the use of regression to make predictions,
case studies, participant observation, disguised and undisguised observation, and structured observation. The terms
independent variable and dependent variable as used in the context of regression were changed to predictor
variable and outcome/criterion variable respectively. A distinction between proportionate stratified sampling and
disproportionate stratified sampling was added to Chapter 7. The section on quasi-experimental designs was moved
to its own chapter (Chapter 8) and was elaborated upon to include instrumentation and testing as threats to internal
validity of one-group pretest-posttest designs, and to include sections describing the one-group posttest only design,
pretest-posttest nonequivalent groups design, interrupted time-series with nonequivalent groups design, pretestposttest design with switching replication, and switching replication with treatment removal designs. The section of
Chapter 9 on factorial designs was split into two sections and the remainder of the chapter was moved or cut.
Further, examples of everyday interactions were added and a description of simple effects was added to Chapter 9.
The section on case studies that appeared in Chapter 10 was edited and moved to Chapter 6. Further, labels were
added to multiple-baseline across behaviors, settings, and participants designs, and a concluding paragraph on
converging evidence was added to Chapter 10. Only minor edits were made to the remaining chapters (Chapters 11,
12, and 13).
Research Methods in Psychology
ii
Acknowledgements
I would like to thank Washington State University’s Academic Outreach and Innovation for funding and supporting
the revision of this textbook so that students at WSU would have free and open access to this textbook.
— Carrie Cuttler
iii
Research Methods in Psychology
Preface
Psychology, like most other sciences, has its own set of tools to investigate the important research questions of its
field. Unlike other sciences that are older and more mature, psychology is a relatively new field and, like an
adolescent, is learning and changing rapidly. Psychology researchers are learning and changing along with the
emerging science. This textbook introduces students to the fundamental principles of what it is like to think like a
psychology researcher in the contemporary world of psychology research.
Historically, psychology developed practices and methods based on the established sciences. Unlike physical
sciences, psychology had to grapple with the inherent variation among its subjects: people. To better accept for this,
we developed some practices and statistical methods that we (naïvely) considered to be foolproof. Over time we
established a foundation of research findings that we considered solid.
In recent years, psychology’s conversation has shifted to an introspective one, looking inward and re-examining the
knowledge that we considered foundational. We began to find that some of that unshakable foundation was not as
strong as we thought; some of the bedrock findings in psychology were being questioned and failed to be upheld in
fuller scrutiny. As many introspective conversations do, this one caused a crisis of faith.
Psychologists are now questioning if we really know what we thought we knew or if we simply got lucky. We are
struggling to understand how what we choose to publish and not publish, what we choose to report and not report,
and how we train our students as researchers is having an effect on what we call “knowledge” in psychology. We are
beginning to question whether that knowledge represents real behavior and mental processes in human beings, or
simply represents the effects of our choice of methods. This has started a firestorm among psychology researchers,
but it is one that needs to play out. For a book aimed at novice psychology undergraduates, it is tempting to gloss
over these issues and proclaim that our “knowledge” is “truth.” That would be a disservice to our students though,
who need to be critical questioners of research. Instead of shying away from this controversy, this textbook invites
the reader to step right into the middle of it.
With every step of the way, the research process in psychology is fraught with decisions, trade-offs, and uncertainty.
We decide to study one variable and not another; we balance the costs of research against its benefits; we are
uncertain whether our results will replicate. Every step is a decision that takes us in a different direction and closer to
or further from the truth. Research is not an easy route to traverse, but we hope this textbook will be a hiking map
that can at least inspire the direction students can take, and provide some absolute routes to begin traveling.
As we wrote at the beginning of this preface, psychology is a young science. Like any adolescent, psychology is
grappling with its identity as a science, learning to use better tools, understanding the importance of transparency,
and is having more open conversations to improve its understanding of human behavior. We will grow up and mature
together. It is an exciting time to be part of that growth as psychology becomes a more mature science.
Research Methods in Psychology
iv
Chapter 1: The Science of Psychology
Many people believe that women tend to talk more than men—with some even suggesting that this difference has a
biological basis. One widely cited estimate is that women speak 20,000 words per day on average and men speak
only 7,000. This claim seems plausible, but is it true? A group of psychologists led by Matthias Mehl decided to find
out. They checked to see if anyone had actually tried to count the daily number of words spoken by women and men.
No one had. So these researchers conducted a study in which female and male college students (369 in all) wore
audio recorders while they went about their lives. The result? The women spoke an average of 16,215 words per day
and the men spoke an average of 15,669—an extremely small difference that could easily be explained by chance. In
an article in the journal Science, these researchers summed up their findings as follows: “We therefore conclude, on
the basis of available empirical evidence, that the widespread and highly publicized stereotype about female
[1]
talkativeness is unfounded” (Mehl, Vazire, Ramirez-Esparza, Slatcher, & Pennebaker, 2007, p. 82) .
Psychology is usually defined as the scientific study of human behavior and mental processes, and this example
illustrates the features that make it scientific. In this chapter, we look closely at these features, review the goals of
psychology, and address several basic questions that students often have about it. Who conducts scientific research
in psychology? Why? Does scientific psychology tell us anything that common sense does not? Why should I bother
to learn the scientific approach—especially if I want to be a clinical psychologist and not a researcher? These are
extremely good questions, and answering them now will provide a solid foundation for learning the rest of the
material in your course.
Mehl, M. R., Vazire, S., Ramirez-Esparza, N., Slatcher, R. B., & Pennebaker, J. W. (2007). Are women really
more talkative than men? Science, 317, 82. ↵
5
Research Methods in Psychology
1
1.1 Methods of Knowing
Learning Objectives
Describe the 5 methods of acquiring knowledge
Understand the benefits and problems with each.
Take a minute to ponder some of what you know and how you acquired that knowledge. Perhaps you know that you
should make your bed in the morning because your mother or father told you this is what you should do, perhaps
you know that swans are white because all of the swans you have seen are white, or perhaps you know that your
friend is lying to you because she is acting strange and won’t look you in the eye. But should we trust knowledge
from these sources? The methods of acquiring knowledge can be broken down into five categories each with its own
strengths and weaknesses.
Intuition
The first method of knowing is intuition. When we use our intuition, we are relying on our guts, our emotions, and/or
our instincts to guide us. Rather than examining facts or using rational thought, intuition involves believing what
feels true. The problem with relying on intuition is that our intuitions can be wrong because they are driven by
cognitive and motivational biases rather than logical reasoning or scientific evidence. While the strange behavior of
your friend may lead you to think s/he is lying to you it may just be that s/he is holding in a bit of gas or is
preoccupied with some other issue that is irrelevant to you. However, weighing alternatives and thinking of all the
different possibilities can be paralyzing for some people and sometimes decisions based on intuition are actually
superior to those based on analysis (people interested in this idea should read Malcolm Gladwell’s book Blink)[1].
Authority
Perhaps one of the most common methods of acquiring knowledge is through authority. This method involves
accepting new ideas because some authority figure states that they are true. These authorities include parents, the
media, doctors, Priests and other religious authorities, the government, and professors. While in an ideal world we
should be able to trust authority figures, history has taught us otherwise and many instances of atrocities against
humanity are a consequence of people unquestioningly following authority (e.g., Salem Witch Trials, Nazi War
Crimes). On a more benign level, while your parents may have told you that you should make your bed in the
morning, making your bed provides the warm damp environment in which mites thrive. Keeping the sheets open
provides a less hospitable environment for mites. These examples illustrate that the problem with using authority to
obtain knowledge is that they may be wrong, they may just be using their intuition to arrive at their conclusions, and
they may have their own reasons to mislead you. Nevertheless, much of the information we acquire is through
authority because we don’t have time to question and independently research every piece of knowledge we learn
through authority. But we can learn to evaluate the credentials of authority figures, to evaluate the methods they
used to arrive at their conclusions, and evaluate whether they have any reasons to mislead us.
Rationalism
Rationalism involves using logic and reasoning to acquire new knowledge. Using this method premises are stated
and logical rules are followed to arrive at sound conclusions. For instance, if I am given the premise that all swans
are white and the premise that this is a swan then I can come to the rational conclusion that this swan is white
Research Methods in Psychology
6
without actually seeing the swan. The problem with this method is that if the premises are wrong or there is an error
in logic then the conclusion will not be valid. For instance, the premise that all swans are white is incorrect; there are
black swans in Australia. Also, unless formally trained in the rules of logic it is easy to make an error. Nevertheless, if
the premises are correct and logical rules are followed appropriately then this is sound means of acquiring
knowledge.
Empiricism
Empiricism involves acquiring knowledge through observation and experience. Once again many of you may have
believed that all swans are white because you have only ever seen white swans. For centuries people believed the
world is flat because it appears to be flat. These examples and the many visual illusions that trick our senses
illustrate the problems with relying on empiricism alone to derive knowledge. We are limited in what we can
experience and observe and our senses can deceive us. Moreover, our prior experiences can alter the way we
perceive events. Nevertheless, empiricism is at the heart of the scientific method. Science relies on observations. But
not just any observations, science relies on structured observations which is known as systematic empiricism.
The Scientific Method
The scientific method is a process of systematically collecting and evaluating evidence to test ideas and answer
questions. While scientists may use intuition, authority, rationalism, and empiricism to generate new ideas they don’t
stop there. Scientists go a step further by using systematic empiricism to make careful observations under various
controlled conditions in order to test their ideas and they use rationalism to arrive at valid conclusions. While the
scientific method is the most likely of all of the methods to produce valid knowledge, like all methods of acquiring
knowledge it also has its drawbacks. One major problem is that it is not always feasible to use the scientific method;
this method can require considerable time and resources. Another problem with the scientific method is that it
cannot be used to answer all questions. As described in the following section, the scientific method can only be used
to address empirical questions. This book and your research methods course are designed to provide you with an indepth examination of how psychologists use the scientific method to advance our understanding of human behavior
and the mind.
Gladwell, M. E. (2007). Blink: The power of thinking without thinking. How to think straight about psychology
(9th ed.). New York: Little, Brown & Company. ↵
7
Research Methods in Psychology
2
1.2 Understanding Science
Learning Objectives
Define science.
Describe the three fundamental features of science.
Explain why psychology is a science.
Define pseudoscience and give some examples.
What Is Science?
Some people are surprised to learn that psychology is a science. They generally agree that astronomy, biology, and
chemistry are sciences but wonder what psychology has in common with these other fields. Before answering this
question, however, it is worth reflecting on what astronomy, biology, and chemistry have in common with each
other. It is clearly not their subject matter. Astronomers study celestial bodies, biologists study living organisms, and
chemists study matter and its properties. It is also not the equipment and techniques that they use. Few biologists
would know what to do with a radio telescope, for example, and few chemists would know how to track a moose
population in the wild. For these and other reasons, philosophers and scientists who have thought deeply about this
question have concluded that what the sciences have in common is a general approach to understanding the natural
world. Psychology is a science because it takes this same general approach to understanding one aspect of the
natural world: human behavior.
Features of Science
The general scientific approach has three fundamental features (Stanovich, 2010)[1]. The first is systematic
empiricism. Empiricism refers to learning based on observation, and scientists learn about the natural world
systematically, by carefully planning, making, recording, and analyzing observations of it. As we will see, logical
reasoning and even creativity play important roles in science too, but scientists are unique in their insistence on
checking their ideas about the way the world is against their systematic observations. Notice, for example, that Mehl
and his colleagues did not trust other people’s stereotypes or even their own informal observations. Instead, they
systematically recorded, counted, and compared the number of words spoken by a large sample of women and men.
Furthermore, when their systematic observations turned out to conflict with people’s stereotypes, they trusted their
systematic observations.
The second feature of the scientific approach—which follows in a straightforward way from the first—is that it is
concerned with empirical questions. These are questions about the way the world actually is and, therefore, can
be answered by systematically observing it. The question of whether women talk more than men is empirical in this
way. Either women really do talk more than men or they do not, and this can be determined by systematically
observing how much women and men actually talk. Having said this, there are many interesting and important
questions that are not empirically testable and that science is not in a position to answer. Among these are questions
about values—whether things are good or bad, just or unjust, or beautiful or ugly, and how the world ought to be. So
although the question of whether a stereotype is accurate or inaccurate is an empirically testable one that science
can answer, the question—or, rather, the value judgment—of whether it is wrong for people to hold inaccurate
stereotypes is not. Similarly, the question of whether criminal behavior has a genetic basis is an empirical question,
but the question of what actions ought to be considered illegal is not. It is especially important for researchers in
psychology to be mindful of this distinction.
Research Methods in Psychology
8
The third feature of science is that it creates public knowledge. After asking their empirical questions, making their
systematic observations, and drawing their conclusions, scientists publish their work. This usually means writing an
article for publication in a professional journal, in which they put their research question in the context of previous
research, describe in detail the methods they used to answer their question, and clearly present their results and
conclusions. Increasingly, scientists are opting to publish their work in open access journals, in which the articles are
freely available to all – scientists and nonscientists alike. This important choice allows publicly-funded research to
create knowledge that is truly public.
Publication is an essential feature of science for two reasons. One is that science is a social process—a large-scale
collaboration among many researchers distributed across both time and space. Our current scientific knowledge of
most topics is based on many different studies conducted by many different researchers who have shared their work
publicly over many years. The second is that publication allows science to be self-correcting. Individual scientists
understand that, despite their best efforts, their methods can be flawed and their conclusions incorrect. Publication
allows others in the scientific community to detect and correct these errors so that, over time, scientific knowledge
increasingly reflects the way the world actually is.
A good example of the self-correcting nature of science is the “Many Labs Replication Project” – a large and
coordinated effort by prominent psychological scientists around the world to attempt to replicate findings from 13
classic and contemporary studies (Klein et al., 2013)[2]. One of the findings selected by these researchers for
replication was the fascinating effect, first reported by Simone Schnall and her colleagues at the University of
Plymouth, that washing one’s hands leads people to view moral transgressions—ranging from keeping money inside
a found wallet to using a kitten for sexual arousal—as less wrong (Schnall, Benton, & Harvey, 2008)[3]. If reliable, this
effect might help explain why so many religious traditions associate physical cleanliness with moral purity. However,
despite using the same materials and nearly identical procedures with a much larger sample, the “Many Labs”
[4]
researchers were unable to replicate the original finding (Johnson, Cheung, & Donnellan, 2013) , suggesting that the
original finding may have stemmed from the relatively small sample size (which can lead to unreliable results) used
in the original study. To be clear, at this stage we are still unable to definitively conclude that the handwashing effect
does not exist; however, the effort that has gone into testing its reliability certainly demonstrates the collaborative
and cautious nature of scientific progress.
For
more
on
the
replication
crisis
in
http://nobaproject.com/modules/the-replication-crisis-in-psychology
psychology
see:
Science Versus Pseudoscience
Pseudoscience refers to activities and beliefs that are claimed to be scientific by their proponents—and may
appear to be scientific at first glance—but are not. Consider the theory of biorhythms (not to be confused with sleep
cycles or circadian rhythms that do have a scientific basis). The idea is that people’s physical, intellectual, and
emotional abilities run in cycles that begin when they are born and continue until they die. Allegedly, the physical
cycle has a period of 23 days, the intellectual cycle a period of 33 days, and the emotional cycle a period of 28 days.
So, for example, if you had the option of when to schedule an exam, you would want to schedule it for a time when
your intellectual cycle will be at a high point. The theory of biorhythms has been around for more than 100 years,
and you can find numerous popular books and websites about biorhythms, often containing impressive and scientificsounding terms like sinusoidal wave and bioelectricity. The problem with biorhythms, however, is that scientific
evidence indicates they do not exist (Hines, 1998)[5].
A set of beliefs or activities can be said to be pseudoscientific if (a) its adherents claim or imply that it is scientific but
(b) it lacks one or more of the three features of science. For instance, it might lack systematic empiricism. Either
there is no relevant scientific research or, as in the case of biorhythms, there is relevant scientific research but it is
ignored. It might also lack public knowledge. People who promote the beliefs or activities might claim to have
conducted scientific research but never publish that research in a way that allows others to evaluate it.
A set of beliefs and activities might also be pseudoscientific because it does not address empirical questions. The
philosopher Karl Popper was especially concerned with this idea (Popper, 2002)[6]. He argued more specifically that
any scientific claim must be expressed in such a way that there are observations that would—if they were
made—count as evidence against the claim. In other words, scientific claims must be falsifiable. The claim that
women talk more than men is falsifiable because systematic observations could reveal either that they do talk more
9
Research Methods in Psychology
than men or that they do not. As an example of an unfalsifiable claim, consider that many people who believe in
extrasensory perception (ESP) and other psychic powers claim that such powers can disappear when they are
observed too closely. This makes it so that no possible observation would count as evidence against ESP. If a careful
test of a self-proclaimed psychic showed that she predicted the future at better-than-chance levels, this would be
consistent with the claim that she had psychic powers. But if she failed to predict the future at better-than-chance
levels, this would also be consistent with the claim because her powers can supposedly disappear when they are
observed too closely.
Why should we concern ourselves with pseudoscience? There are at least three reasons. One is that learning about
pseudoscience helps bring the fundamental features of science—and their importance—into sharper focus. A second
is that biorhythms, psychic powers, astrology, and many other pseudoscientific beliefs are widely held and are
promoted on the Internet, on television, and in books and magazines. Far from being harmless, the promotion of
these beliefs often results in great personal toll as, for example, believers in pseudoscience opt for “treatments”
such as homeopathy for serious medical conditions instead of empirically-supported treatments. Learning what
makes them pseudoscientific can help us to identify and evaluate such beliefs and practices when we encounter
them. A third reason is that many pseudoscience’s purport to explain some aspect of human behavior and mental
processes, including biorhythms, astrology, graphology (handwriting analysis), and magnet therapy for pain control.
It is important for students of psychology to distinguish their own field clearly from this “pseudo psychology.”
The Skeptic’s Dictionary
An excellent source for information on pseudoscience is The Skeptic’s Dictionary (http://www.skepdic.com).
Among the pseudoscientific beliefs and practices you can learn about are the following:
Cryptozoology. The study of “hidden” creatures like Bigfoot, the Loch Ness monster, and the
chupacabra.
Pseudoscientific psychotherapies. Past-life regression, rebirthing therapy, and bioscream therapy,
among others.
Homeopathy. The treatment of medical conditions using natural substances that have been diluted
sometimes to the point of no longer being present.
Pyramidology. Odd theories about the origin and function of the Egyptian pyramids (e.g., that they
were built by extraterrestrials) and the idea that pyramids in general have healing and other special
powers.
Another excellent online resource is Neurobonkers (http://neurobonkers.com), which regularly posts articles
that investigate claims that pertain specifically to psychological science.
Key Takeaways
Science is a general way of understanding the natural world. Its three fundamental features are
systematic empiricism, empirical questions, and public knowledge.
Psychology is a science because it takes the scientific approach to understanding human behavior.
Pseudoscience refers to beliefs and activities that are claimed to be scientific but lack one or more of
the three features of science. It is important to distinguish the scientific approach to understanding
human behavior from the many pseudoscientific approaches.
Exercises
Practice: List three empirical questions about human behavior. List three nonempirical questions about
human behavior.
Discussion: Consider the following psychological claim. “People’s choice of spouse is strongly
influenced by their perception of their own parents. Some choose a spouse who is similar in some way
Research Methods in Psychology
10
to one of their parents. Others choose a spouse who is different from one of their parents.” Is this claim
falsifiable? Why or why not?
Discussion: People sometimes suggest that psychology cannot be a science because either (a) human
behavior cannot be predicted with perfect accuracy or (b) much of its subject matter (e.g., thoughts
and feelings) cannot be observed directly. Do you agree or disagree with each of these ideas? Why?
Watch the following video by PHD Comics for an overview of open access publishing and why it
matters:
https://www.youtube.com/watch?v=L5rVH1KGBCY
Reading in print? Scan this QR
code to view the video on
your mobile device. Or go to
https://youtu.be/L5rVH1KGBC
Y
Stanovich, K. E. (2010). How to think straight about psychology (9th ed.). Boston, MA: Allyn & Bacon. ↵
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, S., Bernstein, M. J., . . . Nosek, B. A. (2013).
Investigating variation in replicability: A “many labs” replication project. Social Psychology, 45(3), 142-152.
doi: 10.1027/1864-9335/a000178 ↵
Schnall, S., Benton, J., & Harvey, S. (2008). With a clean conscience: Cleanliness reduces the severity of moral
judgments. Psychological Science, 19(12), 1219-1222. doi: 10.1111/j.1467-9280.2008.02227.x ↵
Johnson, D. J., Cheung, F., & Donnellan, M. B. (2013). Does cleanliness influence moral judgments? A direct
replication of Schnall, Benton, and Harvey (2008). Social Psychology, 45(3), 209-215. doi:
10.1027/1864-9335/a000186 ↵
Hines, T. M. (1998). Comprehensive review of biorhythm theory. Psychological Reports, 83, 19–64. ↵
Popper, K. R. (2002). Conjectures and refutations: The growth of scientific knowledge. New York, NY:
Routledge. ↵
11
Research Methods in Psychology
3
1.3 Goals of Science
Learning Objectives
Describe the three goals of science and give an example for each.
Distinguish between basic research and applied research.
The Broader Purposes of Scientific Research in Psychology
People have always been curious about the natural world, including themselves and their behavior (in fact, this is
probably why you are studying psychology in the first place). Science grew out of this natural curiosity and has
become the best way to achieve detailed and accurate knowledge. Keep in mind that most of the phenomena and
theories that fill psychology textbooks are the products of scientific research. In a typical introductory psychology
textbook, for example, one can learn about specific cortical areas for language and perception, principles of classical
and operant conditioning, biases in reasoning and judgment, and people’s surprising tendency to obey those in
positions of authority. And scientific research continues because what we know right now only scratches the surface
of what we can know.
The Three Goals of Science
The first and most basic goal of science is to describe. This goal is achieved by making careful observations. As an
example, perhaps I am interested in better understanding the medical conditions that medical marijuana patients
use marijuana to treat. In this case, I could try to access records at several large medical marijuana licensing centers
to see which conditions people are getting licensed to use medical marijuana. Or I could survey a large sample of
medical marijuana patients and ask them to report which medical conditions they use marijuana to treat or manage.
Indeed, research involving surveys of medical marijuana patients has been conducted and has found that the
primary symptom medical marijuana patients use marijuana to treat is pain, followed by anxiety and depression
(Sexton, Cuttler, Finnell, & Mischley, 2016).[1].
The second goal of science is to predict. Once we have observed with some regularity that two behaviors or events
are systematically related to one another we can use that information to predict whether an event or behavior will
occur in a certain situation. Once I know that most medical marijuana patients use marijuana to treat pain I can use
that information to predict that an individual who uses medical marijuana likely experiences pain. Of course, my
predictions will not be 100% accurate but if the relationship between medical marijuana use and pain is strong then
my predictions will have greater than chance accuracy.
The third and ultimate goal of science is to explain. This goal involves determining the causes of behavior. For
example, researchers might try to understand the mechanisms through which marijuana reduces pain. Does
marijuana reduce inflammation which in turn reduces pain? Or does marijuana simply reduce the distress associated
with pain rather than reducing pain itself? As you can see these questions tap at the underlying mechanisms and
causal relationships.
Basic versus Applied Research
Scientific research is often classified as being either basic or applied. Basic research in psychology is conducted
primarily for the sake of achieving a more detailed and accurate understanding of human behavior, without
necessarily trying to address any particular practical problem. The research of Mehl and his colleagues falls into this
category. Applied research is conducted primarily to address some practical problem. Research on the effects of
Research Methods in Psychology
12
cell phone use on driving, for example, was prompted by safety concerns and has led to the enactment of laws to
limit this practice. Although the distinction between basic and applied research is convenient, it is not always clearcut. For example, basic research on sex differences in talkativeness could eventually have an effect on how marriage
therapy is practiced, and applied research on the effect of cell phone use on driving could produce new insights into
basic processes of perception, attention, and action.
Key Takeaways
Psychologists conduct research in order to describe basic phenomenon, to make predictions about
future behaviors, and to explain the causes of behavior.
Basic research is conducted to learn about human behavior for its own sake, and applied research is
conducted to solve some practical problem. Both are valuable, and the distinction between the two is
not always clear-cut.
Exercises
Try to generate different research questions to describe, predict, and explain.
Practice: Based on your own experience or on things you have already learned about psychology, list
three basic research questions and three applied research questions of interest to you.
Sexton, M., Cuttler, C., Finnell, J., & Mischley, L (2016). A cross-sectional survey f medical cannabis users:
Patterns of use and perceived efficacy. Cannabis and Cannabinoid Research, 1, 131-138. doi:
10.1089/can.2016.0007. ↵
13
Research Methods in Psychology
4
1.4 Science and Common Sense
Learning Objectives
Explain the limitations of common sense when it comes to achieving a detailed and accurate
understanding of human behavior.
Give several examples of common sense or folk psychology that are incorrect.
Define skepticism and its role in scientific psychology.
Can We Rely on Common Sense?
Some people wonder whether the scientific approach to psychology is necessary. Can we not reach the same
conclusions based on common sense or intuition? Certainly we all have intuitive beliefs about people’s behavior,
thoughts, and feelings—and these beliefs are collectively referred to as folk psychology. Although much of our folk
psychology is probably reasonably accurate, it is clear that much of it is not. For example, most people believe that
anger can be relieved by “letting it out”—perhaps by punching something or screaming loudly. Scientific research,
however, has shown that this approach tends to leave people feeling more angry, not less (Bushman, 2002) [1].
Likewise, most people believe that no one would confess to a crime that he or she had not committed, unless
perhaps that person was being physically tortured. But again, extensive empirical research has shown that false
confessions are surprisingly common and occur for a variety of reasons (Kassin & Gudjonsson, 2004)[2].
Some Great Myths
In 50 Great Myths of Popular Psychology, psychologist Scott Lilienfeld and colleagues discuss several widely
held commonsense beliefs about human behavior that scientific research has shown to be incorrect
(Lilienfeld, Lynn, Ruscio, & Beyerstein, 2010)[3]. Here is a short list:
“People use only 10% of their brain power.”
“Most people experience a midlife crisis in their 40’s or 50’s.”
“Students learn best when teaching styles are matched to their learning styles.”
“Low self-esteem is a major cause of psychological problems.”
“Psychiatric admissions and crimes increase during full moons.”
How Could We Be So Wrong?
How can so many of our intuitive beliefs about human behavior be so wrong? Notice that this is an empirical
question, and it just so happens that psychologists have conducted scientific research on it and identified many
contributing factors (Gilovich, 1991) [4] . One is that forming detailed and accurate beliefs requires powers of
observation, memory, and analysis to an extent that we do not naturally possess. It would be nearly impossible to
count the number of words spoken by the women and men we happen to encounter, estimate the number of words
they spoke per day, average these numbers for both groups, and compare them—all in our heads. This is why we
tend to rely on mental shortcuts (what psychologists refer to as heuristics) in forming and maintaining our beliefs.
For example, if a belief is widely shared—especially if it is endorsed by “experts”—and it makes intuitive sense, we
tend to assume it is true. This is compounded by the fact that we then tend to focus on cases that confirm our
Research Methods in Psychology
14
intuitive beliefs and not on cases that dis-confirm them. This is called confirmation bias. For example, once we
begin to believe that women are more talkative than men, we tend to notice and remember talkative women and
silent men but ignore or forget silent women and talkative men. We also hold incorrect beliefs in part because it
would be nice if they were true. For example, many people believe that calorie-reducing diets are an effective longterm treatment for obesity, yet a thorough review of the scientific evidence has shown that they are not (Mann et al.,
[5]
2007) . People may continue to believe in the effectiveness of dieting in part because it gives them hope for losing
weight if they are obese or makes them feel good about their own “self-control” if they are not.
Scientists—especially psychologists—understand that they are just as susceptible as anyone else to intuitive but
incorrect beliefs. This is why they cultivate an attitude of skepticism. Being skeptical does not mean being cynical
or distrustful, nor does it mean questioning every belief or claim one comes across (which would be impossible
anyway). Instead, it means pausing to consider alternatives and to search for evidence—especially systematically
collected empirical evidence—when there is enough at stake to justify doing so. For example, imagine that you read
a magazine article that claims that giving children a weekly allowance is a good way to help them develop financial
responsibility. This is an interesting and potentially important claim (especially if you have children of your own).
Taking an attitude of skepticism, however, would mean pausing to ask whether it might be instead that receiving an
allowance merely teaches children to spend money—perhaps even to be more materialistic. Taking an attitude of
skepticism would also mean asking what evidence supports the original claim. Is the author a scientific researcher? Is
any scientific evidence cited? If the issue was important enough, it might also mean turning to the research literature
to see if anyone else had studied it.
Because there is often not enough evidence to fully evaluate a belief or claim, scientists also cultivate a tolerance
for uncertainty. They accept that there are many things that they simply do not know. For example, it turns out
that there is no scientific evidence that receiving an allowance causes children to be more financially responsible,
nor is there any scientific evidence that it causes them to be materialistic. Although this kind of uncertainty can be
problematic from a practical perspective—for example, making it difficult to decide what to do when our children ask
for an allowance—it is exciting from a scientific perspective. If we do not know the answer to an interesting and
empirically testable question, science, and perhaps even you as a researcher, may be able to provide the answer.
Key Takeaways
People’s intuitions about human behavior, also known as folk psychology, often turn out to be wrong.
This is one primary reason that psychology relies on science rather than common sense.
Researchers in psychology cultivate certain critical-thinking attitudes. One is skepticism. They search
for evidence and consider alternatives before accepting a claim about human behavior as true. Another
is tolerance for uncertainty. They withhold judgment about whether a claim is true or not when there is
insufficient evidence to decide.
Exercises
Practice: For each of the following intuitive beliefs about human behavior, list three reasons that it
might be true and three reasons that it might not be true:
You cannot truly love another person unless you love yourself.
People who receive “crisis counseling” immediately after experiencing a traumatic event are
better able to cope with that trauma in the long term.
Studying is most effective when it is always done in the same location.
Watch the following video, in which psychologist Scott Lilienfeld talks about confirmation bias, tunnel
vision, and using evidence to evaluate the world around us:
15
Research Methods in Psychology
Reading in print? Scan this QR
code to view the video on
your mobile device. Or go to
https://youtu.be/Eut8jMfSA_k
Bushman, B. J. (2002). Does venting anger feed or extinguish the flame? Catharsis, rumination, distraction,
anger, and aggressive responding. Personality and Social Psychology Bulletin, 28, 724–731. ↵
Kassin, S. M., & Gudjonsson, G. H. (2004). The psychology of confession evidence: A review of the literature
and issues. Psychological Science in the Public Interest, 5, 33–67. ↵
Lilienfeld, S. O., Lynn, S. J., Ruscio, J., & Beyerstein, B. L. (2010). 50 great myths of popular psychology.
Malden, MA: Wiley-Blackwell. ↵
Gilovich, T. (1991). How we know what isn’t so: The fallibility of human reason in everyday life. New York, NY:
Free Press. ↵
Mann, T., Tomiyama, A. J., Westling, E., Lew, A., Samuels, B., & Chatman, J. (2007). Medicare’s search for
effective obesity treatments: Diets are not the answer. American Psychologist, 62, 220–233. ↵
Research Methods in Psychology
16
5
1.5 Experimental and Clinical Psychologists
Learning Objectives
Define the clinical practice of psychology and distinguish it from experimental psychology.
Explain how science is relevant to clinical practice.
Define the concept of an empirically supported treatment and give some examples.
Who Conducts Scientific Research in Psychology?
Experimental Psychologists
Scientific research in psychology is generally conducted by people with doctoral degrees (usually the doctor of
philosophy [Ph.D.]) and master’s degrees in psychology and related fields, often supported by research assistants
with bachelor’s degrees or other relevant training. Some of them work for government agencies (e.g., the Mental
Health Commission of Canada), national associations (e.g., the American Psychological Association), non-profit
organizations (e.g., the Canadian Mental Health Association), or in the private sector (e.g., in product development).
However, the majority of them are college and university faculty, who often collaborate with their graduate and
undergraduate students. Although some researchers are trained and licensed as clinicians—especially those who
conduct research in clinical psychology—the majority are not. Instead, they have expertise in one or more of the
many other subfields of psychology: behavioral neuroscience, cognitive psychology, developmental psychology,
personality psychology, social psychology, and so on. Doctoral-level researchers might be employed to conduct
research full-time or, like many college and university faculty members, to conduct research in addition to teaching
classes and serving their institution and community in other ways.
Of course, people also conduct research in psychology because they enjoy the intellectual and technical challenges
involved and the satisfaction of contributing to scientific knowledge of human behavior. You might find that you
enjoy the process too. If so, your college or university might offer opportunities to get involved in ongoing research
as either a research assistant or a participant. Of course, you might find that you do not enjoy the process of
conducting scientific research in psychology. But at least you will have a better understanding of where scientific
knowledge in psychology comes from, an appreciation of its strengths and limitations, and an awareness of how it
can be applied to solve practical problems in psychology and everyday life.
Scientific Psychology Blogs
A fun and easy way to follow current scientific research in psychology is to read any of the many excellent
blogs devoted to summarizing and commenting on new findings. Among them are the following:
Brain Blogger, http://brainblogger.com/
Mind Hacks, http://mindhacks.com/
Research Digest, http://digest.bps.org.uk/
Talk Psych, http://www.talkpsych.com/
PsyBlog, http://www.spring.org.uk
Social Psychology Eye, http://socialpsychologyeye.wordpress.com
We’re Only Human, http://www.psychologicalscience.org/onlyhuman
You can also browse to http://www.researchblogging.org, select psychology as your topic, and read entries
from a wide variety of blogs.
17
Research Methods in Psychology
Clinical Psychologists
Psychology is the scientific study of behavior and mental processes. But it is also the application of scientific
research to “help people, organizations, and communities function better” (American Psychological Association,
[1]
2011) . By far the most common and widely known application is the clinical practice of psychology—the
diagnosis and treatment of psychological disorders and related problems. Let us use the term clinical practice
broadly to refer to the activities of clinical and counseling psychologists, school psychologists, marriage and family
therapists, licensed clinical social workers, and others who work with people individually or in small groups to identify
and help address their psychological problems. It is important to consider the relationship between scientific
research and clinical practice because many students are especially interested in clinical practice, perhaps even as a
career.
The main point is that psychological disorders and other behavioral problems are part of the natural world. This
means that questions about their nature, causes, and consequences are empirically testable and therefore subject to
scientific study. As with other questions about human behavior, we cannot rely on our intuition or common sense for
detailed and accurate answers. Consider, for example, that dozens of popular books and thousands of websites claim
that adult children of alcoholics have a distinct personality profile, including low self-esteem, feelings of
powerlessness, and difficulties with intimacy. Although this sounds plausible, scientific research has demonstrated
that adult children of alcoholics are no more likely to have these problems than anybody else (Lilienfeld et al.,
2010)[2]. Similarly, questions about whether a particular psychotherapy is effective are empirically testable questions
that can be answered by scientific research. If a new psychotherapy is an effective treatment for depression, then
systematic observation should reveal that depressed people who receive this psychotherapy improve more than a
similar group of depressed people who do not receive this psychotherapy (or who receive some alternative
treatment). Treatments that have been shown to work in this way are called empirically supported treatments.
Empirically Supported Treatments
An empirically supported treatment is one that has been studied scientifically and shown to result in greater
improvement than no treatment, a placebo, or some alternative treatment. These include many forms of
psychotherapy, which can be as effective as standard drug therapies. Among the forms of psychotherapy with
strong empirical support are the following:
Cognitive behavioral therapy. For depression, panic disorder, bulimia nervosa, and post-traumatic
stress disorder.
Exposure therapy. For post-traumatic stress disorder.
Behavioral therapy. For depression.
Behavioral couples therapy. For alcoholism and substance abuse.
Exposure therapy with response prevention. For obsessive-compulsive disorder.
Family therapy. For schizophrenia.
For a more complete list, see the following website, which is maintained by Division 12 of the American
Psychological Association, the Society for Clinical Psychology: http://www.div12.org/psychological-treatments
Many in the clinical psychology community have argued that their field has not paid enough attention to scientific
research—for example, by failing to use empirically supported treatments—and have suggested a variety of changes
in the way clinicians are trained and treatments are evaluated and put into practice. Others believe that these claims
are exaggerated and the suggested changes are unnecessary (Norcross, Beutler, & Levant, 2005)[3]. On both sides of
the debate, however, there is agreement that a scientific approach to clinical psychology is essential if the goal is to
diagnose and treat psychological problems based on detailed and accurate knowledge about those problems and the
most effective treatments for them. So not only is it important for scientific research in clinical psychology to
continue, but it is also important for clinicians who never conduct a scientific study themselves to be scientifically
literate so that they can read and evaluate new research and make treatment decisions based on the best available
evidence.
Research Methods in Psychology
18
Key Takeaways
Scientific research in psychology is conducted mainly by people with doctoral degrees in psychology
and related fields, most of whom are college and university faculty members. They do so for
professional and for personal reasons, as well as to contribute to scientific knowledge about human
behavior.Most psychologists are experimental psychologists and they conduct research.
The clinical practice of psychology—the diagnosis and treatment of psychological problems—is one
important application of the scientific discipline of psychology.
Scientific research is relevant to clinical practice because it provides detailed and accurate knowledge
about psychological problems and establishes whether treatments are effective.
Exercises
Discussion: Some clinicians argue that what they do is an “art form” based on intuition and personal
experience and therefore cannot be evaluated scientifically. Write a paragraph about how satisfied you
would be with such a clinician and why from each of three perspectives:
a potential client of the clinician
a judge who must decide whether to allow the clinician to testify as an expert witness in a child
abuse case
an insurance company representative who must decide whether to reimburse the clinician for
his or her services
Practice: Create a short list of questions that a client could ask a clinician to determine whether he or
she pays sufficient attention to scientific research.
American Psychological Association. (2011). About APA. Retrieved from http://www.apa.org/about ↵
Lilienfeld, S. O., Lynn, S. J., Ruscio, J., & Beyerstein, B. L. (2010). 50 great myths of popular psychology.
Malden, MA: Wiley-Blackwell. ↵
Norcross, J. C., Beutler, L. E., & Levant, R. F. (Eds.). (2005). Evidence-based practices in mental health: Debate
and dialogue on the fundamental questions. Washington, DC: American Psychological Association. ↵
19
Research Methods in Psychology
Chapter 2: Overview of the Scientific Method
Here is the abstract of a 2014 article in the journal Psychological Science.
Taking notes on laptops rather than in longhand is increasingly common. Many researchers have
suggested that laptop note taking is less effective than longhand note taking for learning. Prior studies
have primarily focused on students’ capacity for multitasking and distraction when using laptops. The
present research suggests that even when laptops are used solely to take notes, they may still be
impairing learning because their use results in shallower processing. In three studies, we found that
students who took notes on laptops performed worse on conceptual questions than students who took
notes longhand. We show that whereas taking more notes can be beneficial, laptop note takers’
tendency to transcribe lectures verbatim rather than processing information and reframing it in their
[1]
own words is detrimental to learning. (Mueler & Oppenheimer, 2014, p. 1159)
In this abstract, the researcher has identified a research question—about the effect of taking notes on a laptop on
learning—and identified why it is worthy of investigation—because the practice is ubiquitous and may be harmful to
learning. In this chapter, we give you a broad overview of the various stages of the research process. These include
finding a topic of investigation, reviewing the literature, refining your research question and generating a hypothesis,
designing and conducting a study, analyzing the data, coming to conclusions, and reporting the results.
Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand
over laptop note taking. Psychological Science, 25(6), 1159-1168. ↵
Research Methods in Psychology
20
6
2.1 A Model of Scientific Research in Psychology
Learning Objectives
Review a general model of scientific research in psychology.
Figure 2.1 presents a simple model of scientific research in psychology. The researcher (who more often than not is
really a small group of researchers) formulates a research question, conducts a study designed to answer the
question, analyzes the resulting data, draws conclusions about the answer to the question, and publishes the results
so that they become part of the research literature. Because the research literature is one of the primary sources of
new research questions, this process can be thought of as a cycle. New research leads to new questions, which lead
to new research, and so on. Figure 2.1 also indicates that research questions can originate outside of this cycle
either with informal observations or with practical problems that need to be solved. But even in these cases, the
researcher would start by checking the research literature to see if the question had already been answered and to
refine it based on what previous research had already found.
Figure 2.1 A Simple Model of Scientific Research in Psychology
The research by Mehl and his colleagues is described nicely by this model. Their question—whether women are more
talkative than men—was suggested to them both by people’s stereotypes and by published claims about the relative
talkativeness of women and men. When they checked the research literature, however, they found that this question
had not been adequately addressed in scientific studies. They then conducted a careful empirical study, analyzed the
results (finding very little difference between women and men), and published their work so that it became part of
the research literature. The publication of their article is not the end of the story, however, because their work
suggests many new questions (about the reliability of the result, about potential cultural differences, etc.) that will
likely be taken up by them and by other researchers inspired by their work.
21
Research Methods in Psychology
https://www.youtube.com/watch?v=XToWVxS_9lA
As another example, consider that as cell phones became more widespread during the
1990s, people began to wonder whether, and to what extent, cell phone use had a
negative effect on driving. Many psychologists decided to tackle this question scientifically
[1]
(Collet, Guillot, & Petit, 2010) . It was clear from previously published research that
engaging in a simple verbal task impairs performance on a perceptual or motor task
carried out at the same time, but no one had studied the effect specifically of cell phone
use on driving. Under carefully controlled conditions, these researchers compared
people’s driving performance while using a cell phone with their performance while not
using a cell phone, both in the lab and on the road. They found that people’s ability to
detect road hazards, reaction time, and maintain control of the vehicle were all impairedReading in print? Scan this QR
by cell phone use. Each new study was published and became part of the growingcode to view the video on
your mobile device. Or go to
research literature on this topic.
youtu.be/XToWVxS_9lA
Key Takeaways
Research in psychology can be described by a simple cyclical model. A research
question based on the research literature leads to an empirical study, the
results of which are published and become part of the research literature.
Exercises
Practice: Find a description of an empirical study in a professional journal or in one of the scientific
psychology blogs. Then write a brief description of the research in terms of the cyclical model
presented here. One or two sentences for each part of the cycle should suffice.
Watch the following TED Ed video, in which David H. Schwartz provides an introduction to two types of
empirical studies along with some methods that scientists use to increase the reliability of their results:
Reading in print? Scan this QR
code to view the video on
your mobile device. Or go to
https://youtu.be/GUpd2HJHUt8
Collet, C., Guillot, A., & Petit, C. (2010). Phoning while driving I: A review of epidemiological, psychological,
behavioral and physiological studies. Ergonomics, 53, 589–601. ↵
Research Methods in Psychology
22
7
2.2 Finding a Research Topic
Learning Objectives
Learn some common sources of research ideas.
Define the research literature in psychology and give examples of sources that are part of the research
literature and sources that are not.
Describe and use several methods for finding previous research on a particular research idea or
question.
Good research must begin with a good research question. Yet coming up with good research questions is something
that novice researchers often find difficult and stressful. One reason is that this is a creative process that can appear
mysterious—even magical—with experienced researchers seeming to pull interesting research questions out of thin
air. However, psychological research on creativity has shown that it is neither as mysterious nor as magical as it
[1]
appears. It is largely the product of ordinary thinking strategies and persistence (Weisberg, 1993) . This section
covers some fairly simple strategies for finding general research ideas, turning those ideas into empirically testable
research questions, and finally evaluating those questions in terms of how interesting they are and how feasible they
would be to answer.
Finding Inspiration
Research questions often begin as more general research ideas—usually focusing on some behavior or psychological
characteristic: talkativeness, learning, depression, bungee jumping, and so on. Before looking at how to turn such
ideas into empirically testable research questions, it is worth looking at where such ideas come from in the first
place. Three of the most common sources of inspiration are informal observations, practical problems, and previous
research.
Informal observations include direct observations of our own and others’ behavior as well as secondhand
observations from non-scientific sources such as newspapers, books, blogs, and so on. For example, you might notice
that you always seem to be in the slowest moving line at the grocery store. Could it be that most people think the
same thing? Or you might read in a local newspaper about people donating money and food to a local family whose
house has burned down and begin to wonder about who makes such donations and why. Some of the most famous
research in psychology has been inspired by informal observations. Stanley Milgram’s famous research on obedience
to authority, for example, was inspired in part by journalistic reports of the trials of accused Nazi war
criminals—many of whom claimed that they were only obeying orders. This led him to wonder about the extent to
which ordinary people will commit immoral acts simply because they are ordered to do so by an authority figure
(Milgram, 1963)[2].
Practical problems can also inspire research ideas, leading directly to applied research in such domains as law,
health, education, and sports. Does taking lecture notes by hand improve students’ exam performance? How
effective is psychotherapy for depression compared to drug therapy? To what extent do cell phones impair people’s
driving ability? How can we teach children to read more efficiently? What is the best mental preparation for running a
marathon?
23
Research Methods in Psychology
Probably the most common inspiration for new research ideas, however, is previous
research. Recall that science is a kind of large-scale collaboration in which many different
researchers read and evaluate each other’s work and conduct new studies to build on it.
Of course, experienced researchers are familiar with previous research in their area of
expertise and probably have a long list of ideas. This suggests that novice researchers can
find inspiration by consulting with a more experienced researcher (e.g., students can
consult a faculty member). But they can also find inspiration by picking up a copy of
almost any professional journal and reading the titles and abstracts. In one typical issue of
Psychological Science, for example, you can find articles on the perception of shapes, antiSemitism, police lineups, the meaning of death, second-language learning, people who
seek negative emotional experiences, and many other topics. If you can narrow yourReading in print? Scan this QR
code to view the video on
interests down to a particular topic (e.g., memory) or domain (e.g., health care), you can
your mobile device. Or go to
also look through more specific journals, such as Memory & Cognition or Healthhttps://youtu.be/nXNztCLYgxc
Psychology.
Reviewing the Research Literature
Once again, one of the most common sources of inspiration is previous research. Therefore, it is important to review
the literature early in the research process. Reviewing the research literature means finding, reading, and
summarizing the published research relevant to your topic of interest. In addition to helping you discover new
research questions, reviewing the literature early in the research process can help you in several other ways.
It can tell you if a research question has already been answered.
It can help you evaluate the interestingness of a research question.
It can give you ideas for how to conduct your own study.
It can tell you how your study fits into the research literature.
The research literature in any field is all the published research in that field. The research literature in psychology
is enormous—including millions of scholarly articles and books dating to the beginning of the field—and it continues
to grow. Although its boundaries are somewhat fuzzy, the research literature definitely does not include self-help and
other pop psychology books, dictionary and encyclopedia entries, websites, and similar sources that are intended
mainly for the general public. These are considered unreliable because they are not reviewed by other researchers
and are often based on little more than common sense or personal experience. Wikipedia contains much valuable
information, but the fact that its authors are anonymous and may not have any formal training or expertise in that
subject area, and its content continually changes makes it unsuitable as a basis of sound scientific research. For our
purposes, it helps to define the research literature as consisting almost entirely of two types of sources: articles in
professional journals, and scholarly books in psychology and related fields.
Professional Journals
Professional journals are periodicals that publish original research articles. There are thousands of professional
journals that publish research in psychology and related fields. They are usually published monthly or quarterly in
individual issues, each of which contains several articles. The issues are organized into volumes, which usually
consist of all the issues for a calendar year. Some journals are published in hard copy only, others in both hard copy
and electronic form, and still others in electronic form only.
Most articles in professional journals are one of two basic types: empirical research reports and review articles.
Empirical research reports describe one or more new empirical studies conducted by the authors. They introduce
a research question, explain why it is interesting, review previous research, describe their method and results, and
draw their conclusions. Review articles summarize previously published research on a topic and usually present
new ways to organize or explain the results. When a review article is devoted primarily to presenting a new theory, it
is often referred to as a theoretical article.
Research Methods in Psychology
24
Figure 2.2 Small Sample of the Thousands of Professional Journals That Publish
Research in Psychology and Related Fields
Most professional journals in psychology undergo a process of double-blind peer review. Researchers who want to
publish their work in the journal submit a manuscript to the editor—who is generally an established researcher
too—who in turn sends it to two or three experts on the topic. Each reviewer reads the manuscript, writes a critical
but constructive review, and sends the review back to the editor along with his or her recommendations. The editor
then decides whether to accept the article for publication, ask the authors to make changes and resubmit it for
further consideration, or reject it outright. In any case, the editor forwards the reviewers’ written comments to the
researchers so that they can revise their manuscript accordingly. This entire process is double-blind, as the reviewers
do not know the identity of the researcher(s) and vice versa. Double-blind peer review is helpful because it ensures
that the work meets basic standards of the field before it can enter the research literature. However, in order to
increase transparency and accountability, some newer open access journals (e.g., Frontiers in Psychology) utilize an
open peer review process wherein the identities of the reviewers (which remain concealed during the peer review
process) are published alongside the journal article.
Scholarly Books
Scholarly books are books written by researchers and practitioners mainly for use by other researchers and
practitioners. A monograph is written by a single author or a small group of authors and usually, gives a coherent
presentation of a topic much like an extended review article. Edited volumes have an editor or a small group of
editors who recruit many authors to write separate chapters on different aspects of the same topic. Although edited
volumes can also give a coherent presentation of the topic, it is not unusual for each chapter to take a different
perspective or even for the authors of different chapters to openly disagree with each other. In general, scholarly
books undergo a peer review process similar to that used by professional journals.
25
Research Methods in Psychology
Literature Search Strategies
Using PsycINFO and Other Databases
The primary method used to search the research literature involves using one or more electronic databases. These
include Academic Search Premier, JSTOR, and ProQuest for all academic disciplines, ERIC for education, and PubMed
for medicine and related fields. The most important for our purposes, however, is PsycINFO, which is produced by the
American Psychological Association (APA). PsycINFO is so comprehensive—covering thousands of professional
journals and scholarly books going back more than 100 years—that for most purposes its content is synonymous
with the research literature in psychology. Like most such databases, PsycINFO is usually available through your
university library.
PsycINFO consists of individual records for each article, book chapter, or book in the database. Each record includes
basic publication information, an abstract or summary of the work (like the one presented at the start of this
chapter), and a list of other works cited by that work. A computer interface allows entering one or more search terms
and returns any records that contain those search terms. (These interfaces are provided by different vendors and
therefore can look somewhat different depending on the library you use.) Each record also contains lists of keywords
that describe the content of the work and also a list of index terms. The index terms are especially helpful because
they are standardized. Research on differences between women and men, for example, is always indexed under
“Human Sex Differences.” Research on note-taking is always indexed under the term “Learning Strategies.” If you do
not know the appropriate index terms, PsycINFO includes a thesaurus that can help you find them.
Given that there are nearly four million records in PsycINFO, you may have to try a variety of search terms in
different combinations and at different levels of specificity before you find what you are looking for. Imagine, for
example, that you are interested in the question of whether women and men differ in terms of their ability to recall
experiences from when they were very young. If you were to enter “memory for early experiences” as your search
term, PsycINFO would return only six records, most of which are not particularly relevant to your question. However,
if you were to enter the search term “memory,” it would return 149,777 records—far too many to look through
individually. This is where the thesaurus helps. Entering “memory” into the thesaurus provides several more specific
index terms—one of which is “early memories.” While searching for “early memories” among the index terms returns
1,446 records—still too many to look through individually—combining it with “human sex differences” as a second
search term returns 37 articles, many of which are highly relevant to the topic.
Depending on the vendor that provides the interface to PsycINFO, you may be able to
save, print, or e-mail the relevant PsycINFO records. The records might even contain links
to full-text copies of the works themselves. (PsycARTICLES is a database that provides fulltext access to articles in all journals published by the APA.) If not, and you want a copy of
the work, you will have to find out if your library carries the journal or has the book and
the hard copy on the library shelves. Be sure to ask a librarian if you need help.
Using Other Search Techniques
Reading in print? Scan this QR
In addition to entering search terms into PsycINFO and other
code to view the video on
databases, there are several other techniques you can use to
your mobile device. Or go to
search the research literature. First, if you have one good
https://youtu.be/fhhctbaVXvk
article or book chapter on your topic—a recent review article is
best—you can look through the reference list of that article for
other relevant articles, books, and book chapters. In fact, you
should do this with any relevant article or book chapter you
find. You can also start with a classic article or book chapter on
your topic, find its record in PsycINFO (by entering the author’s
name or article’s title as a search term), and link from there to
a list of other works in PsycINFO that cite that classic article.Reading in print? Scan this QR
code to view the video on
This works because other researchers working on your topic areyour mobile device. Or go to
likely to be aware of the classic article and cite it in their ownhttps://youtu.be/t1ZwgDeX2e
work. You can also do a general Internet search using searchQ
terms related to your topic or the name of a researcher who
conducts research on your topic. This might lead you directly to
Research Methods in Psychology
26
works that are part of the research literature (e.g., articles in
open-access journals or posted on researchers’ own websites).
The search engine Google Scholar is especially useful for this
purpose. A general Internet search might also lead you to
websites that are not part of the research literature but might
provide references to works that are. Finally, you can talk to
people (e.g., your instructor or other faculty members in
psychology) who know something about your topic and can
suggest relevant articles and book chapters.
What to Search For
When you do a literature review, you need to be selective. Not every article, book chapter, and book that relates to
your research idea or question will be worth obtaining, reading, and integrating into your review. Instead, you want
to focus on sources that help you do four basic things: (a) refine your research question, (b) identify appropriate
research methods, (c) place your research in the context of previous research, and (d) write an effective research
report. Several basic principles can help you find the most useful sources.
First, it is best to focus on recent research, keeping in mind that what counts as recent depends on the topic. For
newer topics that are actively being studied, “recent” might mean published in the past year or two. For older topics
that are receiving less attention right now, “recent” might mean within the past 10 years. You will get a feel for what
counts as recent for your topic when you start your literature search. A good general rule, however, is to start with
sources published in the past five years. The main exception to this rule would be classic articles that turn up in the
reference list of nearly every other source. If other researchers think that this work is important, even though it is
old, then, by all means, you should include it in your review.
Second, you should look for review articles on your topic because they will provide a useful overview of it—often
discussing important definitions, results, theories, trends, and controversies—giving you a good sense of where your
own research fits into the literature. You should also look for empirical research reports addressing your question or
similar questions, which can give you ideas about how to operationally define your variables and collect your data.
As a general rule, it is good to use methods that others have already used successfully unless you have good reasons
not to. Finally, you should look for sources that provide information that can help you argue for the interestingness of
your research question. For a study on the effects of cell phone use on driving ability, for example, you might look for
information about how widespread cell phone use is, how frequent and costly motor vehicle crashes are, and so on.
How many sources are enough for your literature review? This is a difficult question because it depends on how
extensively your topic has been studied and also on your own goals. One study found that across a variety of
professional journals in psychology, the average number of sources cited per article was about 50 (Adair & Vohra,
[3]
2003) . This gives a rough idea of what professional researchers consider to be adequate. As a student, you might
be assigned a much lower minimum number of references to include, but the principles for selecting the most useful
ones remain the same.
Key Takeaways
The research literature in psychology is all the published research in psychology, consisting primarily of
articles in professional journals and scholarly books.
Early in the research process, it is important to conduct a review of the research literature on your
topic to refine your research question, identify appropriate research methods, place your question in
the context of other research, and prepare to write an effective research report.
There are several strategies for finding previous research on your topic. Among the best is using
PsycINFO, a computer database that catalogs millions of articles, books, and book chapters in
psychology and related fields.
27
Research Methods in Psychology
Exercise
Practice: Use the techniques discussed in this section to find 10 journal articles and book chapters on
one of the following research ideas: memory for smells, aggressive driving, the causes of narcissistic
personality disorder, the functions of the intraparietal sulcus, or prejudice against the physically
handicapped.
Watch the following video clip produced by UBCiSchool about how to read an academic paper (without
losing your mind):
Reading in print? Scan this QR
code to view the video on
your mobile device. Or go to
https://youtu.be/SKxm2HF_-k0
Weisberg, R. W. (1993). Creativity: Beyond the myth of genius. New York, NY: Freeman. ↵
Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371–378. ↵
Adair, J. G., & Vohra, N. (2003). The explosion of knowledge, references, and citations: Psychology’s unique
response to a crisis. American Psychologist, 58, 15–23. ↵
Research Methods in Psychology
28
8
2.3 Generating Good Research Questions
Learning Objectives
Describe some techniques for turning research ideas into empirical research questions and use those
techniques to generate questions.
Explain what makes a research question interesting and evaluate research questions in terms of their
interestingness.
Generating Empirically Testable Research Questions
Once you have a research idea, you need to use it to generate one or more empirically testable research questions,
that is, questions expressed in terms of a single variable or relationship between variables. One way to do this is to
look closely at the discussion section in a recent research article on the topic. This is the last major section of the
article, in which the researchers summarize their results, interpret them in the context of past research, and suggest
directions for future research. These suggestions often take the form of specific research questions, which you can
then try to answer with additional research. This can be a good strategy because it is likely that the suggested
questions have already been identified as interesting and important by experienced researchers.
But you may also want to generate your own research questions. How can you do this? First, if you have a particular
behavior or psychological characteristic in mind, you can simply conceptualize it as a variable and ask how frequent
or intense it is. How many words on average do people speak per day? How accurate are our memories of traumatic
events? What percentage of people have sought professional help for depression? If the question has never been
studied scientifically—which is something that you will learn in your literature review—then it might be interesting
and worth pursuing.
If scientific research has already answered the question of how frequent or intense the behavior or characteristic is,
then you should consider turning it into a question about a relationship between that behavior or characteristic and
some other variable. One way to do this is to ask yourself the following series of more general questions and write
down all the answers you can think of.
What are some possible causes of the behavior or characteristic?
What are some possible effects of the behavior or characteristic?
What types of people might exhibit more or less of the behavior or characteristic?
What types of situations might elicit more or less of the behavior or characteristic?
In general, each answer you write down can be conceptualized as a second variable, suggesting a question about a
relationship. If you were interested in talkativeness, for example, it might occur to you that a possible cause of this
psychological characteristic is family size. Is there a relationship be...
Purchase answer to see full
attachment