A Presentation by Experts
[Your Name]
[Date]
PSY 496
[Provide
a brief, general explanation of the
topic chosen in Week One (2-3 sentences).]
[The topic is…]
[The reason it is important]
[Issues and controversies]
[At least 2 key questions about the topic]
[Replace the pictures with others of your own choosing.]
[Name, Degrees,
Profession]
Expert
1
[Background/Brief
Biography: 1-2 sentences]
[Name, Degrees,
Profession]
Expert
2
[Background/Brief
Biography: 1-2 sentences]
[You may use up to 2 slides for this section.]
Describe
her or his evaluation of the
contributions of psychological research on
the chosen topic in an applied context.
Be sure to include information from at
least one peer-reviewed, journal article
or chapter from a an edited scholarly
book of your choice, published within the
last 5 years, to substantiate your experts’
claims.
Include a website recommended by the
expert.
[You may use up to 2 slides for this section.]
Describe
her or his evaluation of the
contributions of psychological research on
the chosen topic in an applied context.
Be sure to include information from at least
one peer-reviewed, journal article or chapter
from a an edited scholarly book of your
choice, published within the last 5 years, to
substantiate your experts’ claims. This
article should be different from the article
you used with Expert 1.
Include a website recommended by the
expert.
Expert 1’s Name
1. Point 1
2. Point 2
3. Point 3
[Use 1-2 words to
describe each
point.]
At least
one idea
both have
in common
Expert 2’s Name
1. Point 1
2. Point 2
3. Point 3
[Use 1-2 words
to describe
each point.]
[List all references used in alphabetical order in accordance with APA guidelines.]
[Peer-reviewed journal article 1, in APA format]
[Peer-reviewed journal article 2, in APA format]
Piotrowski, C. (2012). Research areas of emphasis in professional
psychology: Past and current trends. Journal of Instructional
Psychology, 39(2), 131-135. Retrieved from the ProQuest
database.
[Website 1, in APA format]
[Website 2, in APA format]
4
Associated Press
Quasi-Experimental Designs
Chapter Learning Outcomes
After reading and studying this chapter, students should be able to:
• understand the important role that quasi-experimental designs play in answering research
questions of interest when participants in a research study cannot be randomly assigned to
conditions.
• articulate the appropriate research scenarios for a number of quasi-experimental design
approaches, including nonequivalent control groups, time series, cohort/panel, and regression discontinuity designs, as well as the applications of program evaluation.
• appreciate the practical applications and limitations of observational designs such as case
studies and naturalistic observation.
• understand the contexts in which archival research approaches, such as content analysis and
meta-analysis, can answer interesting questions based on the availability of pre-existing data.
lan66845_04_c04_p101-130.indd 101
4/20/12 2:47 PM
CHAPTER 4
Introduction
Introduction
I
n the ideal world, psychology researchers would like to be able to understand causeand-effect relationships as they pertain to attitudes and behaviors. Cause-and-effect
conclusions are powerful, because they provide insight as to how positive behaviors
can be promoted and how negative behaviors can be lessened. Not all research situations
lend themselves to laboratory controls and randomization of participants to conditions.
So outside of the laboratory (and even sometimes in the laboratory), we apply research
methods as best we can, but without random assignment. Much of the research conducted
in the social sciences is conducted using quasi-experimental designs. In fact, there are
times where random assignment is inappropriate, such as withholding health care services from children for the sake of an experimental study (Bawden & Sonenstein, n.d.).
Say that I wanted to study how effective the use of clickers (student response systems)
might be in the classroom. I’m teaching two sections of introductory psychology, and I
decide that I will use clickers in one of the sections, and I won’t use clickers in the other
section. At the end of the semester, I want to see if students earned more points in the section with clickers than in the section without clickers. This would be a quasi-experimental
design, because I am unable to randomly assign students to sections (students usually
like selecting their own classes). In fact, using X and O, here’s what the design would look
like (remember, X represents the independent variable manipulation and O represents the
dependent variable measure):
X clickers O points
O points
If I had been able to randomly assign participants to conditions, there would have been an
R (for randomization) at the beginning of each line. This design is called a nonequivalent
control groups posttest only design. Although it is a good idea to have a control group,
because of the lack of randomization, we are less confident that the groups were equivalent
even before the start of the study—thus, we label the design as nonequivalent. This is one
example of the types of designs that are collectively known as quasi-experimental designs.
In this chapter, we’ll investigate the advantages and disadvantages of quasi-experimental
designs, observational designs, and archival research.
Voices from the Workplace
Your name: James K.
Your age: 47
Your gender: Male
Your primary job title: Co-Director of Community Living
Your current employer: Village Northwest Unlimited
How long have you been employed in your present position?
27 years (continued)
102
lan66845_04_c04_p101-130.indd 102
4/20/12 2:47 PM
CHAPTER 4
Introduction
Voices from the Workplace (continued)
What year did you graduate with your bachelor’s degree in psychology?
1982
Describe your major job duties and responsibilities.
I am responsible for coordinating services for nearly 60 clients. Our clients are all adults with physical or mental disabilities or both. I also help oversee the 60 staff that provides care and services for
these individuals, including meeting with them on a quarterly basis to review the care programs being
provided, conducting performance reviews. I help develop new training goals, and provide guidance
to staff and clients on how to effectively complete their work and grow. I conduct Quality Assurance
on documentation samples that are written by our staff, and perform follow-up reports to the various
staff teams that we supervise. One of our goals is to encourage and support our staff, and we make
this a priority in our work.
What elements of your undergraduate training in psychology do you use in your work?
I would say the training we received in counseling and helping techniques is probably the most beneficial and frequently used education from my undergraduate training.
What do you like most about your job?
The flexibility of schedule.
What do you like least about your job?
Bureaucracy and poor communication with and from the various funding streams that we work with
and those hired to oversee this. Poor wages for staff.
What is the compensation package for an entry-level position in your occupation?
It would be a range of $19,000–$25,000 for a middle management person, $15,000–$18,000 for a fulltime direct care staff.
What benefits (e.g., health insurance, pension, etc.) are typically available for someone in your
profession?
Major Medical Health Insurance, Life Insurance, various perks offered by our agency such as reduced
costs for various items like newspaper subscriptions, wellness or fitness memberships.
What are the key skills necessary for you to succeed in your career?
Patience, good listening abilities, creativity understanding, integrity, honesty, and willingness to go the
extra mile.
Thinking back to your undergraduate career, what courses would you recommend that you believe are
key to success in your type of career?
Abnormal Psychology, Theories of Counseling, Developmental Psychology.
As an undergraduate, do you wish you had done anything differently? If so, what?
I don’t think so, I was adequately prepared in my education for my job.
What advice would you give to someone who was thinking about entering the field you are in?
Be focused and dedicated to the people you are serving or trying to help. They are someone else’s
son or daughter and have tremendous value. Try to instill ways of making your job FUN and enjoyable.
Encourage and support those that work with you. Create a climate of participative decision-making so
that everyone feels invested in what is being decided. Show honor and respect to those above you and
to those you come into contact with each day, make them feel valued and important. (continued)
103
lan66845_04_c04_p101-130.indd 103
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
Voices from the Workplace (continued)
If you were choosing a career and occupation all over again, what (if anything) would you do differently?
I can’t think of anything that I would choose to do differently.
Copyright . 2009 by the American Psychological Association. Reproduced with permission. The official
citation that should be used in referencing this material is R. Eric Landrum, Finding Jobs With a Psychology Bachelor’s Degree: Expert Advice for Launching Your Career, American Psychological Association,
2009. The use of this information does not imply endorsement by the publisher. No further reproduction
or distribution is permitted without written permission from the American Psychological Association.
4.1 Quasi-Experimental Design Types
A
quasi-experiment is “a design that manipulates the presumed case and measures
the presumed outcome but does not randomly assign participants to conditions”
(Shadish & Cook, 2009, p. 619). The example with the clickers is a nonequivalent control groups design, but sometimes there isn’t even a control group (these types of
designs are sometimes referred to as pre-experimental designs; Morgan, Gliner, & Harmon, 2000). Using our X’s and O’s, here is an example:
X O
The above is a posttest only design (commonly used) but yields very little information
that can be generalized beyond the participants being tested. A common example would
be the teaching evaluations you provide at the end of the semester. The instructor taught
the class (X), and you provided an evaluation (O) at the end of the semester. Although
helpful to the instructor, there is not much we can say about whether this instructor is
effective as compared to other instructors, if one class learned better than another class,
and so on. Another pre-experimental design is the one group pretest-posttest design:
O1 X O2
In this case, we can observe if there was change from pre to post, but little else. Staying in
class, perhaps at the beginning of a math course you are given a comprehensive pretest
(O1), and at the end of the course (X) you are given the same test again (O2). This design
will allow us to detect change over time, but that may not provide much useful information to the instructor. What if you were a math wizard coming into the course, and over
the semester you didn’t learn much? That might make it look like the instructor didn’t do a
very good job, when it was your excellent preparation that explains why your scores didn’t
increase much over time. Note that in discussions about K–12 teacher salaries and meritbased pay, the ability to demonstrate student learning over time is a huge issue. So this
pre-experimental design does allow us to inquire about change over time, but it provides
little insight as to why changes may have occurred. We could also add a pretest to this nonequivalent control groups design, making it a pretest-posttest design, as depicted below:
O X O
O
O
104
lan66845_04_c04_p101-130.indd 104
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
Although random assignment is not achieved with this design, the existence of a pretest
does allow us to explore whether the groups were different at the start on the variables
that were measured. This is not the same as randomization, but at least if the groups were
roughly equivalent prior to the introduction of the independent variable, then our confidence increases as to what the possible implications of the results mean.
Nonequivalent Control Groups
The nonequivalent control groups design is quite common throughout the social sciences and psychology, and here we’ll discuss just a sampling of practical applications of
this design. For example, to measure the impact of a computer-based training program
for nurses, Hart et al. (2008) administered a pretest questionnaire, delivered information
about evidence-based practice, and then administered a posttest questionnaire. This is the
classic pretest-posttest design, and here’s what it would look like graphically:
Oevidence-based practice pretest Xcomputer based education program Oevidence-based practice posttest
This type of design lets the researchers know if the participants changed over time. However, it’s hard to gauge the effectiveness of the intervention (X) without a control group.
Sometimes the constraints of the situation make random assignment impractical. For
example, a medical school decided to implement a new form of ethics training for its students based on small-group ethics teaching (Goldie, Schwartz, McConnachie, & Morrison,
2001) but wanted to compare this new approach with the previous lecture-style largegroup ethics instruction. Rather than randomly assign students to different instructional
conditions, new incoming medical students received the new curriculum, and students
from the previous year were utilized as the control condition. The experimental design
would look like this:
Osurvey score Xnew curriculum Osurvey score (experimental group)
Osurvey score Xold curriculum Osurvey score (control group)
Luckily, under the old curriculum, an ethics and health care survey had been administered both pretest and posttest. These same instruments were utilized with the new small
group ethics discussion sections. Goldie et al. (2001) found that the new curriculum led
to greater consensus in considering ethical situations and concluded that “small-group
ethics teaching, in an integrated medical curriculum, had a positive impact on first-year
students’ potential ethical behavior. It was more effective than a lecture and a large-group
seminar-based course in developing students’ normative identification with the profession of medicine” (p. 295). Even though a true experiment was not conducted here, you
can see the benefit of the outcomes of the quasi-experimental design—we can learn much
from these types of designs, even if we cannot draw a cause-and-effect conclusion.
Time Series Design
In its simplest form, quasi-experimental research using a time series design “. . . is simply a set of repeated observations of a variable on some entity or unit, where the number of repetitions is relatively large” (Mark, Reichardt, & Sanna, 2000, p. 353). For example (Garson, 2008), the monthly calculation of the national unemployment index by the
105
lan66845_04_c04_p101-130.indd 105
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
Bureau of Labor Statistics would be considered a simple time series design. In essence,
you can think of this as an extended sequence of dependent variable measurements (O).
A simple time series would look like this:
O O O O O O O O O O O O
The above 12 observations could be the monthly reporting of the unemployment index, for
example. As you can imagine, the time series design allows for the assessment of change
over time—trends—but it can do much more than that (Mark et al., 2000). A time series
design can also be used for forecasting. For example, if an economist is tracking unemployment rates, he or she may use this data to try to predict what will happen six months
from now, based on the data accumulated leading up to this point in time.
Time series designs can become more complex as we introduce independent variables (X)
in to the mix, such as a particular treatment or intervention. These types of designs are
sometimes called interrupted time series designs (Cook & Campbell, 1979; Mark et al.,
2000) because of the interruption (X) over the series of observations (O’s). This type of
design might look like this:
O O O O O O X O O O O O O
Note the independent variable manipulation in the middle of the sequence of observations. This interrupted time series design is often used to measure the impact of legislation and public policy, such as
the implementation of a mandatory seat belt law or a ban
on cigarette smoking on a college campus. Let’s say you were
interested in determining the
impact of a smoking ban on
your college campus. You might
utilize a research design where
observers are trained to collect
data at various points on campus during different days of the
week at different times of day.
The dependent variable of interest (O) is the number of smokers
observed. You implement this
data collection program well
If you wanted to study the impact of a smoking ban on a college
before the announcement of the
campus, you could collect data by observing the number of
new smoking ban on campus
people smoking throughout the day.
that begins on July 1. Here’s
iStockphoto/Thinkstock what that design might look like:
OJan OFeb OMar OApr OMay OJun XJul OAug OSep OOct ONov ODec OJan
Notice that there are six observations before and after the ban takes effect. The goal of the
smoking ban would be to reduce the number of smokers, at least the number of smoking
incidents observed on campus. One of the benefits of this type of design is that it tracks
106
lan66845_04_c04_p101-130.indd 106
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
changes over time. So after the ban is publicized and implemented July 1, there might be
an immediate decrease in observed smokers (maybe they stopped smoking, or are hiding it better). But after a while, say a couple of months, the numbers of observed smokers
might increase. (Ever received a speeding ticket? Did you decrease your speeds for the
short term, only to return to your regular habits a short time later?) So you can see the benefit of the interrupted time series design, to assess the impact of an intervention. But the
drawback of the quasi-experimental design is that we cannot be overly confident about
causality—a decrease in observed smokers could mean many things—some stopped
smoking, some hid their smoking better, some switched to chewing tobacco, and so on.
Time series designs can be expanded to examine the relationship between multiple levels of
an independent variable manipulation (Mark et al., 2000). This design might look like this:
O O O O O O X O O O O O O
O O O O O O O O O O O O
In fact, this is very similar to the non-equivalent control groups pretest-posttest design,
but it includes multiple pretests and multiple posttests. An example of this type of design
is the work of Ivancevich (1974) looking at the impact of a management-by-objectives
(MBO) approach as well as reinforcement schedules in the performance of a manufacturing corporation with multiple plant sites. Ivancevich measured multiple dependent variables, such as the quantity of output, quality of output, grievance rates by employees, and
absenteeism. Three different plants were utilized, as indicated below.
Plant 1 O XMBO O O O O
Plant 2 O XMBO O O O XReinf. O
Plant 3 O O O O O
From this research, Ivancevich concluded that the benefit of the reinforcement observed
in Plant 2 tended to overshadow the MBO effects.
In considering time series designs, there are three factors to keep in mind (Garson, 2008a):
age, period, and cohort. In the time series design, variables are repeatedly measured over
time. Given that people can naturally change over time without any exposure to an independent variable, sometimes it will be difficult to disentangle a change due to the independent variable, or just the passage of time. There are also period effects as well, meaning that
individuals from a particular historical period may be impacted similarly. Garson suggested
that those individuals who lived through the Great Depression, the Challenger explosion,
or 9/11 may have similar beliefs and behaviors. Thus, recording times series data over long
periods of time (e.g., decades), the researcher must understand that historical events may be
impacting the change or lack of change seen in a time series design. Finally, cohort effects
should be considered. A cohort effect occurs when people from a particular age range are
impacted differentially by a historical event. For instance, Garson (2008) suggested that after
World War II, young adults at this time (a cohort) reacted differently than past generations
to the challenges of the later Vietnam War. Thus, the idea of a cohort effect is the intersection
of a group of people in time with a historical event that impacts that group; the post–World
War II cohort appeared to be more suspicious of the U.S. government and its policies as compared to earlier cohorts. Some quasi-experimental studies are specifically designed to examine cohort effects, and these types of designs are called cohort designs and panel studies.
107
lan66845_04_c04_p101-130.indd 107
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
Cohort Designs and Panel Studies
As Alwin, McCammon, and Hofer (2006) pointed out, cohorts can take on a number of
identities. Sometimes researchers think of cohorts as generations, such as baby boomers or
millennials. The term birth cohort is used to identify a group of people who were all born
in the same year. But a general definition of cohort is a group of people who have shared
critical experiences over the same span of time; however, a cohort may not be equivalent
to a generation, nor may a collection of adjacent cohorts necessarily be considered a generation (Alwin et al., 2006).
In thinking about the three keys to time series (age, period, cohort), we can construct a
standard cohort table that depicts these factors simultaneously (Alwin et al., 2006; Garson,
2008). In the cohort study, individuals are randomly sampled for each of the “cells,” thus a
measure of net change over time (period), age, and cohort can be acquired (see Figure 4.1).
Figure 4.1: Standard cohort approach for a longitudinal study
Year of Study (Period)
Age
1980
1990
2000
2010
20–29
30–39
40–49
50–59
60–69
70–79
80–89
This figure depicts the complexity of a standard cohort approach over time. In 1980, there are random
samples of individuals from each of the age groupings on the leftmost column. Ten years later, additional
random samples are taken for each of the age groupings depicted in the leftmost column. Note that this
study would take 30 years to complete.
Source: Wuench (2003)
As you can see from above, age is depicted across the rows, period is depicted down the
columns, and the cohorts are identified by the diagonal lines crossing individual cells. For
just a moment, think about the complexity of this design. First, your study takes 30 years
to complete (launched in 1980, completed in 2010). Second, you recruit many people from
many age ranges, from 20-year-olds to 89-year-olds. In the typical cohort study, participant selection is random; that is, different people participate at the different periods of time. To
follow a cohort, you would follow the diagonal line as time passes and participants age.
You should know that not all quasi-experimental cohort designs involve 10-year time periods where the study spans 30 years. Many, if not most, cohort designs are based on shorter
108
lan66845_04_c04_p101-130.indd 108
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
periods. In fact, relatively short (6 months to 1 year) cohort studies have been used to
examine the effectiveness of new therapeutic approaches to help couples (Epstein et al.,
2007), minor depressive disorders, and miscarriage (Klier, Geller, & Neugebauer, 2000) and
to monitor the development of social skills in kindergartners (Hall, Jones, & Claxton, 2008).
Panel studies, also called prospective cohort studies, add an interesting twist to the standard cohort study—they follow the same individuals over time. In a typical cohort study,
like the one depicted previously, different individuals are recruited and studied in 1980,
1990, 2000, and 2010. In the panel study/prospective cohort study, using this same structural example, the same people would be tracked for 30 years, which would take enormous
effort, expense, and organization. These types of studies are relatively rare, yet particularly
informative. To illustrate a panel study, the chart has been modified to add start points and
arrows to indicate that the same individuals are being studied over time (see Figure 4.2).
Figure 4.2: Panel study or prospective cohort study for longitudinal research
Year of Study (Period)
Age
1980
1990
2000
2010
20–29
30–39
40–49
50–59
60–69
70–79
80–89
This figure depicts the complexity of a panel study approach over time. In 1980, there are samples of
individuals from each of the age groupings on the leftmost column selected. These same individuals are
tracked over time and tested 10, 20, and 30 years later. Thus one age grouping is followed over time.
Note that this study would also take 30 years to complete, but is much more complicated than a standard
cohort table design because in the panel study the same individuals are tracked and tested over time.
Source: Wuench (2003)
An impressive example of this type of study was conducted by Wilson and Widom (2008),
who studied individuals who had been victims of abuse and neglect during childhood.
These researchers conducted a 30-year follow up with the original participants using a
prospective cohort design. Although the results are complex, the major finding was that
individuals who were maltreated as children were more likely to report sexual contact
before age 15, participate in prostitution by their young adult years, and test positive for
HIV during middle adulthood as compared to individuals who were not maltreated as
children. These types of studies are invaluable in helping psychologists understand the
role of childhood experiences and how these experiences may shape choices and behaviors in various stages of adulthood.
109
lan66845_04_c04_p101-130.indd 109
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
Regression Discontinuity Designs
The regression discontinuity design (RDD) is a unique member of the quasi-experimental
design family in that it leads to conclusions that are similar in strength to randomized
controlled trials (Lesik, 2006; Rutter, 2007). Regression discontinuity designs are sometimes called cutoff-based designs because assignment into the treatment group or the
control group is based on a predetermined cutoff (Shadish & Cook, 2009). From a population of individuals, a cutoff score is set before the study begins, and then the trends
(regression lines) are followed to see if they are continuous or not. Think about it this
way—an RDD is a good choice if you are working in a situation where some remediation may be helpful to a subgroup of the population (Lesik, 2006), such as students who
need extra help with math, or employees who may be facing layoffs in a weak economy.
Rather than randomly assigning participants to conditions, such as with a coin toss, the
RDD sets a cutoff score, and scores above the cutoff are assigned to one condition of
the study, while scores below the cutoff are assigned to the other condition of the study
(Lesik, 2006).
Figure 4.3: The first part of a regression discontinuity design
22
C
C
20
C
18
C
C C
C
Posttest
C
C
C
C
C
T T
T
T
T
T
14
C
T
C
T
C
C
C
T T
T
C C
C
T
12
T
T
TT
10
Cutoff
T
8
2
3
4
5
6
7
8
9
10
11
Pretest
This graphic depicts the first part of a regression discontinuity design, where the relationship between two
variables (here: pretest and posttest) is identified. Now, an intervention is about to be applied to the T
(treatment) individuals.
Source: Lesik (2006)
110
lan66845_04_c04_p101-130.indd 110
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
An example with pictures might be helpful here (Wuensch, 2003). Look at the graph in
Figure 4.3. On the x-axis is the pretest, and on the y-axis is the posttest score. The T’s
and C’s represent individual people, graphically depicting their placement with regard to
their pretest and posttest scores. The solid blue line is the regression line—this is the line
of best fit, the line that best captures the linear relationship between the pretest and posttest scores. As you can see, there is no “discontinuity”—the line is solid and continuous.
The vertical black line just below the pretest score of 6 represents the cutoff. Thus every
person scoring just below a 6 is placed into the treatment group (depicted by T’s), and
everyone scoring above the cutoff is a control participant (C’s). If, after the study was
complete, the results looked like this graph, this would tell us that there was no effect
of the intervention/independent variable on the treatment group—in other words, the
regression line shows continuity.
But what if there had been an effect? What would that look like? Well, as in most research,
we are looking for the independent variable to influence the dependent variable. In RDD
we have that evidence when we see a regression discontinuity—a break in line (and/
or different line slopes) on either side of the cutoff. See the example in Figure 4.4 from
Wuensch (2003).
Figure 4.4: The second part of a regression discontinuity design
22
C
20
C
Posttest
C
C
C
C
18
T T
T
T
16
CC C C
CC C
CCC
T
T
CC
T
14
C
C
T T
T
C
T
T
T
T
C
T
12
T
Cutoff
10
0
2
4
6
8
10
12
Pretest
This is a hypothetical example of a regression discontinuity design where the intervention (the treatment,
T) was effective. Rather than a linear (straight-line) relationship between the pretest and the posttest, the
lines are now broken (i.e., discontinuous).
Source: Wuensch (2003)
111
lan66845_04_c04_p101-130.indd 111
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
The dashed line to the left of the cutoff reminds us about the original continuous regression line, and the break in the solid lines on either side of the cutoff demonstrates a
regression discontinuity. In this case, the treatment worked—posttest scores are higher
(the new red line) for the lowest-scoring pretest participants as compared to where we
would have expected them to be. You can see where RDD could be very useful when
scores are used to determine membership in programs or receipt of benefits, such as
need-based programs for scholarships, or scores that are used as cutoffs to determine
which children are enrolled in gifted and talented programs in elementary schools. One
key consideration to remember is that the cutoff score must be determined prior to the
beginning of the study (Lesik, 2006; Rutter, 2007; Shadish & Cook, 2009). RDD adds
another useful approach to studying behavior outside the laboratory in the use of quasiexperimental designs.
Program Evaluation
Quasi-experimental approaches are often employed in program evaluation efforts because
of the emphasis on evaluating a program in its context, not in the laboratory. The basic
idea of program evaluation is to assess the impact of a program based on its pre-stated
goals (Rose & Fiore, 1999). But what do we mean by a program? A program tends to
serve a known population, is long-lasting, and attempts to solve a limited set of problems (Greene, 2003). Examples of programs that would be suitable for program evaluation
would be a local Head Start program, the McNair Scholarship program, or perhaps the
ongoing performance of your Department of Psychology. Greene (2003) provided excellent examples of what program evaluation is not: it is not a single methodology or data
collection instrument, not data collection that occurs at the end of a program cycle, and
not decisions made about the value or worth of a program.
Program evaluation efforts tend to follow in three distinct phases, as outlined by Greene
(2003). In the generation phase, a needs assessment may be conducted prior to the implementation of a program, to see if the need justifies the implementation of a program,
followed by program development and refinement. In the implementation phase, the
program begins, provides services to clients and the community, and hopefully has a
positive impact. In the causation phase, the program evaluator attempts to demonstrate
and measure the impact of the program, such as the delivery of services to clients, clients’
evaluation of services, and the impact felt by other agencies or the community in general.
For example, a legislator might be concerned about reports of unplanned teenage pregnancy in his or her legislative district. Following a program evaluation approach, the first
step would be to generate possible programs that could address the concern of unwanted
teenage pregnancies, using local or national programs that promote abstinence, contraception, or both strategies. The needs analysis conducted prior to program implementation would provide baseline data as to the true nature and scope of the issue—is the teen
pregnancy rate on the rise, or are there just some well-publicized recent events that bring
this topic to light?
Following a needs analysis, a public relations (PR) campaign may be implemented to try
to change teenagers’ attitudes and opinions about sexual activity, with the ultimate goal
to influence behavior. This might be implemented, for example, by a state agency such
as a Health and Welfare District, which might provide workshops for middle school and
112
lan66845_04_c04_p101-130.indd 112
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
high school sex educators to offer curriculum material for teaching teenagers strategies
to avoid unwanted pregnancy. Throughout the program, evaluations may be conducted,
asking local sex education teachers about their perception of how the program is working. Additionally, teens might be surveyed about program perceptions and any possible
behavior changes. In addition, the program evaluator might examine epidemiological
data from the county or legislative district to examine if the rate of teen pregnancy did
decline over the period of the implementation of the program. Although it continues to
be difficult to draw causality conclusions when using a quasi-experimental design (Rose
& Fiore, 1999), this convergence of data might be used by citizens and legislators alike
to evaluate the effectiveness of the program and determine whether the program should
continue. In fact, program evaluation and other quasi-experimental approaches are often
used to help answer public policy questions, as well as to generate new ideas on how to
solve complex social issues.
Case Study: Answering Public Policy Questions
Although naturally implied, public policy questions should hopefully lead to public policy answers,
and this is just one of the domains of program evaluation and the use of experimental and quasiexperimental designs. The need for increased program evaluation services comes, in part, with the
necessity for agencies to be held accountable for government-sponsored human services programs.
According to Hosie (1994), “program evaluation plays an extremely important role in policy-making.
Outcome data is [sic] provided to decision makers who determine the value and worth of programs
based on outcome and available financial resources” (p. 349). McCartney and Weiss (2007) took this
notion (at least) one step further when they stated “policies designed to improve the life chances of
children and families often live or die on the basis of the findings from evaluation research” (p. 59).
It seems clear that program evaluations are used to affect federal, state, and local agency policy decisions (Hosie, 1994). Given this weight, it should be obvious how important it is to get appropriate
answers in the course of addressing public policy questions.
Historically, psychologists have had some success in helping to answer public policy questions, but
as you’ll see, psychology can do much more. Lodzinski, Motomura, and Schneider (2005) suggested
that the best public policy influence in psychology was the role psychological science played in
1954 in influencing the Brown v. Board of Education Supreme Court ruling, which essentially made
racial segregation in public schools unconstitutional. The contributions of psychologists, especially
Kenneth B. Clark, made the argument that segregating White and African-American students led to
poorer self-esteem for African-American students and that segregation was a source of interracial
prejudice.
In the U.S. Supreme Court ruling, psychological research was openly credited with influencing the decision, and this case has been credited with forever connecting psychology and public policy (Lodzinski
et al., 2005).
Critical Thinking Questions
1. Quasi-experimental designs are complex yet allow for research conclusions to be drawn outside
of the laboratory. Thinking about your current or future work environment, can you think of one
real-world situation where you could use a nonequivalent control groups design, times series
design, cohort design or panel study, and regression discontinuity design?
2. The research methods presented in this chapter provide you with impressive additions to your
toolkit or Swiss Army knife regarding how research methods can allow for the study of challenging, real-world questions. However, good science communicates outcomes, especially if the
goal is to influence public policy. How would you present one of the methodologies (continued)
113
lan66845_04_c04_p101-130.indd 113
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
Case Study: Answering Public Policy Questions (continued)
discussed so far, say regression discontinuity design, to the general public? That is, how would
you translate the benefits of this design approach to actual data such that others can see the
benefit and comprehend the impact? Note: This is a valuable skill—if you want to see if you
understand a concept, attempt to teach that concept to someone else.
3. Large social issues, such as public schooling, unemployment, teen pregnancy, universal health
care, poverty, and homelessness, are complex on many levels. How might different aspects of
research methods (that is, different features in your ever-growing Swiss Army knife of available
tools) be used to address such large, complex societal concerns?
There are some fundamental lessons that evaluators and public policy makers can keep in
mind when examining the effectiveness of a program evaluation (McCartney & Weiss, 2007):
•
•
•
•
•
Use mixed method designs—in other words, use different approaches to attempt
to capture the totality of the program. If true experiments (e.g., randomized
controlled trials) are not possible, then use quasi-experimental methods to help
measure the effectiveness of programs and public policy implementations.
Interpret effect size in a research context—when conducting statistical analyses
on the data from program evaluation research, one must go beyond the typical
inferential/null hypothesis significance testing and look at effect sizes. This helps
researchers distinguish between statistically significant artifacts and conditions
where the results indicate practical significance.
Synthesize all research findings—this chapter ends with a presentation of metaanalysis, which is a methodological procedure that allows researchers to combine
the results of many studies into broad recommendations and findings.
Adopt fair and reasonable scientific expectations—scientific data may go only so
far in helping to answer public policy questions. Researchers must be careful not
to overstate the relative impact or contribution of a set of data.
Encourage peer and public critique of the data—we need to invite our peers and
the public to scrutinize our work and the resulting impact that our work has in
impacting the research community. Ultimately, the success that psychologists
have in influencing public policy decisions may heavily rest on the public’s ability to interpret what we do.
In the day-to-day functioning of a scientist, data are paramount, but policy makers and
the public may not similarly appreciate a data-driven approach. McCartney and Weiss
(2007) reminded us that in addition to empirical data, policy makers and citizens consider
anecdotal evidence and testimonies, newspaper features, politics as usual, and so forth.
Add to this mix the influence of blogs and cable news programs that feature partisan
approaches, and there are many voices that contribute to the complexity of decisions to
be made. Public policies, whether we like it or not, are political entities, and the success
or failure of public policy can lead to the success or failure of those in government. Psychology needs to inform the public on what it can provide to public policy table, for the
potential contributions are formidable. In 2002, Philip Zimbardo, president of the American Psychological Association, made this statement not long after the tragic events of 9/11
regarding relevance of psychology to the shaping of public policy:
114
lan66845_04_c04_p101-130.indd 114
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
Even before the September 11th tragedy, most of the major problems facing
the United States were psychological in cause, correlates, or consequence:
for example, AIDS and sexually transmitted diseases; drug addiction, as
well as addictions to smoking, gambling, alcohol, and food; prejudice and
discrimination; delinquency; violent crimes; educational failures for too
many minority youth; and the full range of physical illnesses that are influenced by lifestyle and behavioral functioning. Psychologists have much
to say about more effective ways of dealing with these problems at both
individual and community levels of action. Psychologists need to be heard
and to be at the table of influential leaders and policymakers because psychologists have more to say about these issues than do members of any
other discipline (p. 432).
Classic Studies in Psychology: Work Decisions and Absenteeism
(Lawler & Hackman, 1969)
If you had the chance to develop a program at work that could add to your paycheck, would you take
that opportunity, or would you be skeptical and cynical about what management was up to? If the
plan were designed by coworkers at another location but imposed on you, would you gladly accept
the opportunity to earn more, or would you not invest your efforts into the program, since you had
no hand in designing it? These are some of the real questions that Edward Lawler and Richard Hackman began to answer in 1969.
These researchers worked with a large service that
cleaned buildings in the evenings, and there were various groups of employees who did not work in similar
locations but performed similar tasks. Essentially,
there were three groups in this quasi-experimental
design. It should be noted that Lawler and Hackman
(1969) purposely chose this design because conducting the study in the field would increase the probability that the results of this research could be generalized to other settings (i.e., enhance external validity).
The first group in the study is labeled the “participaFlirt/SuperStock
tive” group, and these workers designed their own
pay-for-performance incentive plan, with the help of researchers and the supervision of management. The second group in the study is the “imposed” group, which received the exact same incentive plan as the participative group, although the imposed group had no say in its development; thus,
it was imposed upon them. The third group was the control group, and in actuality, there were two
different versions of control group. One control group talked to researchers about the pay for performance plans, absenteeism, and turnover, but no changes were made to its incentive plan. Another
control group received no information at all about the study and was not contacted by researchers or
top management about the study.
As you can see in Figure 4.5, the researchers looked at the percentage of scheduled hours worked,
both before the intervention (pay plans) and after. The graph presents the data for the participative
group. For example, if an employee were scheduled to work 40 hours in a week but was only present for 32 hours, on the y-axis this would yield a score of 80%. When averaged together, employees
in the participative group worked 88% of their scheduled hours before the incentive plan was introduced. After the incentive plan, they worked 94% of their scheduled hours (a statistically significant
increase). (continued)
115
lan66845_04_c04_p101-130.indd 115
4/20/12 2:47 PM
CHAPTER 4
Section 4.1 Quasi-Experimental Design Types
Classic Studies in Psychology: Work Decisions and Absenteeism
(Lawler & Hackman, 1969) (continued)
Figure 4.5: An example of quasi-experimental data
Percent of Scheduled Hours Actually Worked
100
95
90
85
80
75
70
12 11 10 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 10 11 1213141516
Before
After
Weeks
Participative group results from Lawler and Hackman study.
Source: Lawler and Hackman (1969)
Now consider the results of the “imposed” group, displayed in Figure 4.6. This group received the
same incentive plan, with the only difference being that its members did not design it or contribute
to its creation. The data for the imposed group are below. Before the incentive plan was introduced,
employees in the imposed group worked 83% of their scheduled hours; after the incentive plan, they
also worked 83% of their scheduled hours—no change. Both control groups saw no change either
from pre–incentive plan to post–incentive plan.
Figure 4.6: Another example of quasi-experimental data
Percent of Scheduled Hours Actually Worked
100
95
90
85
80
75
70
12 11 10 9 8 7 6 5 4 3 2 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Before
After
Weeks
“Imposed” group results from the Lawler and Hackman study.
Source: Lawler and Hackman (1969)
(continued)
116
lan66845_04_c04_p101-130.indd 116
4/20/12 2:47 PM
CHAPTER 4
Section 4.2 Observational Designs
Classic Studies in Psychology: Work Decisions and Absenteeism
(Lawler & Hackman, 1969) (continued)
Because this study was so well designed, it leads to some fascinating conclusions. Lawler and Hackman (1969) found that “the data show that employee attendance improved only in those groups that
participatively developed their own incentive plans. Neither the incentive plan alone nor participation
and discussion alone yielded any changes in attendance” (p. 470). Clever design, including the control groups, leads to this meaningful conclusion. If the incentive plan were all that was needed, then
the imposed group would have seen increases too. If just discussion were enough, one of the control
groups would have shown increased in the percentages of hours worked. The conclusion here was
that participation and implementation of the plan were necessary to see significant increases in work;
either option alone was ineffective. This study has been influential in informing the literature about
how top management teams operate (e.g., West & Anderson, 1996), as well as in educating us on
when pay-for-performance works, and when it may not (Beer & Cannon, 2004).
Critical Thinking Questions
1. It is one thing to talk about abstract ideas in the context of a textbook, but when people start
to discuss personal issues like salaries, you can imagine there might be some concern when
researchers want to enter a situation and implement programs that could affect an employee’s
paycheck. What role do researchers have in educating the public (and educating research participants) about the potential beneficial role (as well as potential drawbacks) to experimentation?
2. Completing coursework in a 5-week session requires diligence, motivation, and determination.
Think about the courses you have completed in this mold, and the different design features of
those courses. Were there conditions or scenarios in some classes that motivated you to work
harder than in other classes? Was that due to the content of the course, the design of the course,
or both? Using a quasi-experimental approach, how might a researcher study how a course is
structured and its relationship to student learning?
3. Can you look at a graph, study it, and decipher its meaning, or do you just gloss over graphs and
figures? Are you more convinced by a persuasive story when in is presented in words (text) or
pictures (graphs)? Why is that? What steps do you think you would need to take to become better
versed in your non-preferred modality?
4.2 Observational Designs
W
ithin the scope of this chapter, a thorough and comprehensive review of observational research is just not possible. We’ll focus here on two key approaches—
case studies and naturalistic observation, with a brief overview of the terminology most relevant to this topic (Brown, n.d.; Garson, 2008; Pope & Mays, 2006). Some
designs are summarized in Table 4.1.
117
lan66845_04_c04_p101-130.indd 117
4/20/12 2:47 PM
CHAPTER 4
Section 4.2 Observational Designs
Table 4.1: Terminology
Type
Brief Description
Field experiments
A field experiment involves a research study where the actual data collection
occurs in natural settings (in the field).
Case study
An extensive observation of an individual or a single group is the hallmark
of the case study approach. Case studies tend to look at a limited set of
behaviors rather than the totality of the person or group.
Naturalistic
observation
Using naturalistic observation, the researcher is involved in the direct
observation of behavior as it occurs in its natural setting. In principle, the
researcher does not interact within the environment being observed, but
only observes.
Participant
observation
In participant observation, the researcher inserts him- or herself into the
environment being studied, which can be especially useful when studying
group processes. Researchers using this technique must be careful to
remain objective and avoid observer effects (those who know they are being
observed may change their own behavior due to the observation).
Action research
Action research is a subset of participant observation in which the researcher
in the natural environment works to change some aspect of behavior or
the organization. These actions are designed to improve conditions for
the participants or the organization. Rather than test a hypothesis, action
research attempts to overtly change behavior.
Archival research
Archival researchers study the records that already exist that were originally
recorded in natural settings.
Surveys
Surveys are a versatile methodological approach because they can be
administered to individuals in natural settings as part of a fieldwork approach.
Program evaluation
Program evaluation involves the evaluation of systematic programs in
applied settings. That is, program effectiveness is determined by how
patients or clients are served in the field.
Ethnography
Ethnography involves the direct observation of people during daily life. This
term is sometimes used interchangeably with case study, and ethnography
refers to both a research process and the type of report that is written as a
product of that research.
Case Studies
The case study approach focuses on a particular case of interest, and this case may be a
person, a group, or perhaps an organization. Case studies can utilize qualitative and quantitative methods. In fact, a research strategy called triangulation encourages researchers
to study the variable of interest from multiple perspectives, and not over-rely on any one
research approach. Researchers using a case study approach can be forward-looking (prospective) or look back in time (retrospective); they can approach theories inductively or
deductively; and they can strive to describe, evaluate, or explain behavior (Garson, 2008b;
Walsche, Caress, Chew-Graham, & Todd, 2004). Because the approach is so diverse, it is
118
lan66845_04_c04_p101-130.indd 118
4/20/12 2:47 PM
CHAPTER 4
Section 4.2 Observational Designs
often difficult to define the case study, but the overriding concern may be the case of interest, rather than the methodological approach used (Stake, 1994):
Case study is not a methodological choice, but a choice of object to be studied. We choose to study the case. We could study it in many ways. The
physician studies the child because the child is ill. The child’s symptoms
are both qualitative and quantitative. The physician’s record is more quantitative than qualitative. The social worker studies the child because the
child is neglected. The symptoms of neglect are both qualitative and quantitative. The formal record the social worker keeps is more qualitative than
quantitative. In many professional and practical fields, cases are studied
and recorded. As a form of research, case study is defined by interest in
individual cases, not by the methods of inquiry used. (p. 236)
Case studies can be very influential in helping to
understand the historical background of the topic
under study, explore unexpected outcomes, delve
into the complexity of interrelationships among
people and entities, explore gaps between what
is intended and what happens, and in general
obtain a comprehensive look at the big picture.
Although there are limitations to the case study
approach, such as the inability to broadly generalize, case studies can be particularly useful in generating hypotheses and theories in newer fields
(Garson, 2008b). In other words, when we know
little about a topic, the case study can be extremely
useful in providing context about a new idea and
about how variables may affect behavior.
Naturalistic Observation
The term naturalistic observation typically implies
an observational situation where the researcher
does not interact in the environment, but merely
A social worker makes a case study of
observes it. However, observation-based research
a neglected child and will record both
is much more complicated. Naturalistic observaqualitative and quantitative data.
tion tends to fall into the category of qualitative
Liquidlibray/Thinkstock research, and the goal of qualitative research is to
understand human behavior holistically (rather
than analytically), and consider the social and cultural context in which we behave (Angrosino, 2007). The development of systematic observational protocols is a key component in
naturalistic observation research, and these studies tend to fall into one of three broad categories: (a) non-reactive (unobtrusive) research, where the researcher does not participate
in the events under observation; (b) reactive research, in which the research is immersed
and clearly present in the environment but strives for the role of outside observer; and
(c) participant research, where the researcher embeds him- or herself into the environment
and participants as an active member of the group being studied (Angrosino, 2007).
119
lan66845_04_c04_p101-130.indd 119
4/20/12 2:47 PM
CHAPTER 4
Section 4.2 Observational Designs
Observational research, including naturalistic observation,
strives for some of the same
goals as experimental research,
but it also faces unique challenges. Validity and reliability
are challenges to observational
research (Adler & Adler, 1994).
Because data recording and
interpretation are key in naturalistic observation, validity
may be threatened by an overreliance on any one particular
approach. One suggestion that
Adler and Adler (1994) offered
is to employ a multiple observer A marine biologist observes a shark in the ocean. Naturalistic
strategy such that there is not observation implies that the researcher does not interact with
overreliance on any one mem- the environment or participants but just watches and records
ber of the research team. Reli- observations.
ability is also a concern in natFlirt/SuperStock
uralistic observation, for if a
phenomenon is not stable and consistent, then it is difficult to interpret the impact of
the singular observation on overall behavior. Observations that are made over varying
times and places may yield higher confidence in the reliability of the findings, or as Adler
and Adler (1994) put it, “like many qualitative methods, naturalistic observation yields
insights that are more likely to be accurate for the group under study and unverified for
extension to a larger population. Observations conducted systematically and repeatedly
over varying conditions that yield the same findings are more credible than those gathered according to personal patterns” (p. 381).
So what would a naturalistic observation look like, or, in other words, what are the basic
steps that are followed? Angrosino (2007) described a typical sequence of events that is
followed in naturalistic observation research:
•
•
•
In the descriptive phase, the researcher is interested in reporting initial observations that are related to the general research questions under study, as well as providing descriptions of the environment being studied, including the people and
the place. At this point, observations should be as value-free as possible, without
interpretation—the goal would be statements of fact based on direct observation.
Once a broad base is established, the focusing phase begins as researchers strive to
sort out relevant observations from irrelevant observations, especially in how these
observations relate to the hypotheses and key questions under study. These observations would be more focused on well-defined activities (e.g., traditions, rituals,
events) rather than one-time random occurrences. The goal here is to identify patterns, especially how the observed patterns relate the research question of interest.
The selective phase might be analogous to a “highway merge” in a large city,
where six lanes of traffic are funneled into two lanes. There are still observations to
be recorded, but now the key behaviors have been focused on and are under careful scrutiny. Although the entire field of the environment is still under observation,
selected observations are used to help provide possible explanations for behaviors.
120
lan66845_04_c04_p101-130.indd 120
4/20/12 2:47 PM
CHAPTER 4
Section 4.3 Archival Research
•
Finally, by the time the saturation point is reached, no new findings are being discovered. The major patterns of behavior are well established and rarely change.
Further observation after reaching the saturation point has a relatively low
probability of discovering anything new. At this point the data analysis and interpretation phase is about to begin. Success during this phase of the observational
research will very much depend on the coding scheme used and the researcher’s
attention to detail in following the protocols established before beginning the
naturalistic observation.
Naturalistic observations can be great sources of new ideas. Take, for example, the study
by Chiang (2008) where children with autism were studied using naturalistic observation.
Children with autism often can be taught communication skills, but these children often
lack the ability to spontaneously utilize verbal and nonverbal skills. The goal of Chiang’s
research was to document and categorize the levels of communicative spontaneity in
autistic children. Thirty-two diagnosed autistic children, ranging in age from 3 to 16, were
videotaped in their natural settings, which included special schools for autistic children,
special education classrooms, and general education classrooms. One of the key findings
from this research was that autistic students exhibited higher levels of communicative
spontaneity in non-symbolic forms (e.g., keeping an item, pushing) than symbolic forms
(e.g., writing, speech). These types of results, while interesting on their own merit, can
provide fertile grounds for additional researchers to formulate ideas about how to better
understand autistic children and their communication patterns.
As with every methodological approach to studying human behavior, case studies and
naturalistic observations have drawbacks as well. Because the participants studied may
not be representative of the population, these types of studies lack external validity—
the ability to be generalized beyond the participants studied. Case studies and naturalistic observations are typically difficult to replicate unless extreme care has been taken to
explicitly record the procedures used. Since these approaches do not follow a true experimental protocol (including random assignment), causal inferences are not possible, even
those with compelling data. Finally, these types of studies rely on highly skilled researchers, because the potential for influencing the outcome of the study is great, whether it
would be experimenter bias in a case study or experimenter reactivity (Brown, n.d.).
4.3 Archival Research
A
rchival research is a broad term that can be used to describe a wide range of studies.
Essentially, archival research involves analysis of data from existing records that were
made in natural settings. For instance, by reviewing records from professional baseball and basketball championship games, Baumeister (1995) found that home teams are more
likely to choke (i.e., perform badly), perhaps due to the burden of high expectation made by
playing in front of hometown fans. Riniolo, Koledin, Drakulic, and Payne (2003) used archival records from 1912 United States Senate hearings (and from the British Board of Trade) to
compare eyewitness accounts of the Titanic sinking to the forensic data we now have, particularly examining the claim that the Titanic was breaking apart as it sank. Riniolo et al. (2003),
after carefully screening testimony that indicated clear observations, found that 15 of 20 eyewitnesses accurately reported this tragic event. Related to more recent events, Martz, Bodner, and Livneh (2009), in using archival data available from the National Vietnam Veterans
121
lan66845_04_c04_p101-130.indd 121
4/20/12 2:47 PM
CHAPTER 4
Section 4.3 Archival Research
Readjustment Study (a national
random sample of over 3,000
veterans drawn from 8.2 million
veterans who served in Vietnam),
found that for veterans with disabilities, teaching them problemsolving skills was beneficial only
for veterans with mild to moderate disabilities (the intervention
was ineffective for those veterans
with severe disabilities). These
three studies illustrate the versatility of archival research. None
of these researchers actively collected data; these researchers
examined baseball box scores,
congressional testimony, and preexisting survey data, respectively.
Archival research uses data from previous studies.
Hemera/Thinkstock
Although there are many different forms of archival research, we’ll end this chapter by
reviewing two major approaches to analyzing archival data—content analysis and meta
analysis. What is the relative contribution that archival research can make to our understanding of human behavior? Baumeister (1995) rightly pointed out that archival data
may not the best choice for testing and building theories in psychology; the laboratory
probably remains the best environment for achieving those goals. However, archival
research can be invaluable in providing real-world examples of phenomena that are studied in controlled laboratory settings. As Baumeister put it, “perhaps the best compromise
is that these [archival studies] should be regarded as extending, illustrating, and confirming laboratory studies rather than as primary, direct tests of theory” (p. 646).
Content Analysis
In some ways, content analysis bridges the worlds of qualitative research and quantitative research (Duriau, Reger, & Pfarrer, 2007). A typical approach to content analysis
would be to identify key textual information. Content analysis takes qualitative statements, writings, and other forms of language and quantifies that data. For example, a
conceptual analysis approach means that the frequency of representation of a particular
word or concept is quantified in content analysis (Busch et al., 2009). Thus, you could
examine presidential inaugural speeches and count the number of mentions of freedom, a
free society, and so on. After identifying the research question and selecting the samples to
be analyzed, the content is coded into predetermined categories, where frequency counts
are now possible. A second approach within content analysis is called relational analysis,
where other words are found in the text and then examined for their proximity to key
concepts being analyzed (Busch et al., 2009). Thus, when freedom is mentioned in inaugural addresses, what other concepts are mentioned within close proximity to mentions of
freedom, such as liberty, taxation, or civil rights?
Because of the versatility of content analysis, this methodological approach can be used to
study a wide variety of topics. For example, Sonpar and Golden-Biddle (2008) looked at
122
lan66845_04_c04_p101-130.indd 122
4/20/12 2:47 PM
CHAPTER 4
Section 4.3 Archival Research
adolescent theories of cognitive organization and found that content analysis was helpful
in elaborating on pre-existing theories. Chancey and Dumais (2009) used content analysis
in a more longitudinal approach. Their interest was in how couples who purposely choose
to be childless (“voluntarily childless”) have been depicted in marriage and family textbooks ranging from the 1950s through 1990s. In this archival research using content analysis, Chancey and Dumais discovered that different decades presented different themes.
Content analysis revealed that marriage and family textbooks in the 1950s depicted the
voluntarily childless as individuals who understood that child-rearing was a challenge
and to be feared, and those rejecting becoming parents were preemptively avoiding causing harm in children. In the 1960s, the voluntarily childless were seen as actively promoting marital satisfaction, whereas texts of the 1970s tended to say very little about the
voluntarily childless. Content analysis of textbooks revealed that in the 1980s there was
increased scholarly focus on the voluntarily childless, as well as discovering that there
was often a disparity in opinions within a couple of the voluntarily childless decision.
In the 1990s, content analysis revealed that the most effort was aimed at dispelling the
myths surrounding the voluntarily childless and sometimes characterized these couples’
decisions as brave in the face of societal expectations. Clearly Chancey and Dumais would
not be able to go back in time and interview voluntarily childless couples, but the content analysis of marriage and family textbooks over time provided insight into prevalent
thought processes at the time.
There are a number of advantages available to researchers who elect to use a content
analysis approach. This listing combines the advantages as stated by Busch et al. (2009)
and Duriau et al. (2007)—content analysis (a) allows for both qualitative and quantitative
analysis; (b) can use a longitudinal approach that allows for the examination of historical
contexts over time; (c) is an unobtrusive method of analyzing interactions; (d) can provide
insight into complex cognitive operations, such as language and thought; and (e) uses an
approach that can be replicated by others interested in the same phenomenon. Some of the
disadvantages to content analysis include the labor intensiveness of obtaining and coding
texts, that it can sometimes operate without a theoretical base, the reduction of complex
human thoughts into parsed words, the oversimplification of the complexity of language
by its reduction into frequency counts, and even the inability to capture the totality of context, such as the historical period or geographical context of what is spoken and written.
Even with these drawbacks, content analysis is a powerful tool for the analysis of language.
Meta-Analysis
Within a single year, multiple studies are published on the same topic, and when thinking
about a topic over a long range of time (like you would do if you were reviewing the literature), discrepancies appear in the interpretation of data. For instance, there are claims in
the literature that a moderate amount of red wine can be very healthy for you with regard
to cardiovascular function, but there are conflicting reports about the effect of red wine,
especially if you consider the risk of alcoholism. So if you had to make an overall decision
about whether or not to consume red wine, how would you combine the results of the
previous studies to make a decision? This is precisely the dilemma than then-senator Walter Mondale faced in 1970 (Walter Mondale later went on to serve as vice president of the
United States under President Jimmy Carter from 1977–1981). When Mondale addressed
the 1970 convention of the American Psychological Association, he wanted some insight
into the impact government programs were having on children’s development, and here’s
what he said (Attwood, 2009):
123
lan66845_04_c04_p101-130.indd 123
4/20/12 2:47 PM
CHAPTER 4
Section 4.3 Archival Research
I had hoped to find research to support or to conclusively oppose my belief
that quality-integrated education is the most promising approach. But I
have found very little conclusive evidence. For every study, statistical or
theoretical, that contains a proposed solution or recommendation, there is
always another, equally well documented, challenging the assumptions or
conclusions of the first. No one seems to agree with anyone else’s approach.
But more distressing, no one seems to know what works. As a result, I must
confess, I stand with my colleagues confused and often disheartened. (¶s
18–19)
The contribution of meta-analysis is that it provides an approach for combining the results of
multiple studies so that general effects can be
summarized (Bangert-Drowns & Rudner, 1991).
Although the idea of combining the results of studies can be traced back to Pearson and Fisher (Cooper & Lindsay, 1998), the 1970s saw an enhanced
effort to combine studies to answer complex questions. For example, in looking at the results of
studies of class size and academic achievement,
Glass and Smith (1979) combined the results of
725 studies that estimated that relationship. Smith
and Glass (1977) looked at the effectiveness of
psychotherapy by examining 833 estimates of the
effect, and Hunter, Schmidt, and Hunter (1979)
looked at how employment tests may or may not
differentiate between white and African-American employees. For each of these broad questions
(class size, psychotherapy, employment testing),
meta-analysis combined the results of hundreds
of studies to allow for overall conclusions to be
There are numerous studies that are
drawn. In fact, Glass (1976) is credited for the completed and published about the same
term meta-analysis (Cooper & Lindsay, 1998), and topic. Using meta-analysis provides an
Glass (1976) defined meta-analysis as “the statisti- approach that combines the results of these
cal analysis of a large collection of analysis results studies into a summary.
from individual studies for purposes of integratage fotostock/SuperStock
ing the findings” (p. 3). There are at least four
different types of meta-analyses that a researcher
could conduct (Bangert-Drowns & Rudner, 1991; Cooper & Lindsay, 1998), and you might
see more than one analytical approach used in a report of meta-analytic findings. Here are
brief descriptions of the approaches that are used:
•
•
Using the vote count method, the overall impact of the study is counted as a
“vote” in showing that the intervention has a positive (beneficial) effect, a negative (detrimental) effect, or no effect at all. The results are tallied to come to an
overall conclusion about the topic under study.
The classic method (combining probabilities across studies) might involve the summation of all the p values in used in significance testing, and then using a statistical procedure to determine if the overall result is likely to occur by chance or not.
124
lan66845_04_c04_p101-130.indd 124
4/20/12 2:47 PM
CHAPTER 4
Section 4.3 Archival Research
•
•
Rather than vote count, now each study contributes a degree of impact to the
overall conclusion. Since p values are relatively common in published studies, this
approach is straightforward, but sometimes overestimates the impact of the effect.
When using the effect size method in meta-analysis, the degree of impact is more
precise. With the p value approach above, studies with large Ns are more likely
to have small p values—effect size is not as influenced by the N as p values are.
There are procedures that can weigh the impact of an effect size when that effect
size comes from a larger-scale study.
Meta-analysis can also examine the relative contribution of variables within studies using effect size indices. Sometimes called homogeneity analysis or tests of
homogeneity, the researcher can examine the degree to which different aspects of
the research design impacted the effect sizes observed.
This is a brief overview of the types of approaches available to the researcher utilizing
meta-analysis. If you were to perform a meta-analysis, the steps you would follow might
look like this: (1) clearly define the hypothesis and the operations under study; (2) conduct
a thorough literature search; (3) categorize the studies; (4) transform to a common metric;
(5) analyze the data; (6) formulate discussion and draw conclusions; and (7) generate a
report (Botella & Gambara, 2006). Even from this description, you can see multiple metaanalytic approaches being used, such as vote counting and calculating effect sizes. One
of the benefits of the development and use of meta-analysis is that it has increased the
rigor with which research is discussed and published (Hedges, 1990). The meta-analytic
approach is not without drawbacks, however (Bangert-Drowns & Rudner, 1991; Nugent,
2009). Three of the most commonly mentioned challenges in meta-analysis are (a) missing
data on effect size, (b) missing data on study characteristics, and (c) publication bias. In
some cases, published research may not fully present the statistical results needed for a
researcher using meta-analysis to be able to calculate an effect size for each study, and in
other studies, such as a case study, there may be no statistical information at all. The complexity of the design might also make extracting relevant information about the effect size
complicated (Hedges, 1990). If too many data are missing from the studies on a particular
topic, this poses a threat to the value of the interpretation of the remaining studies.
Quality meta-analyses also need information about how the study was conducted, especially information about the procedures used in the study, how participants were assigned
to groups, precisely how the dependent variables were measured, and so on (Hedges, 1990).
If this type of information is not included in the studies being reviewed, then it is difficult to
determine the relative contribution of the study to the meta-analysis. Because journal space
is typically at a premium, sometimes key information that meta-analysis researchers need
might not be included in published accounts. Another area of concern for meta-analytic
research involves what is called publication bias. Typically, to publish an article in the psychological literature, you need to reject the null hypothesis—that is, find a significant difference between groups, relationship, or association between variables or predictors of a
criterion. Thus, studies that accept the null hypotheses are difficult to publish. A researcher
conducting a meta-analysis would prefer to have a complete picture of all the research conducted on the topic (regardless of outcome), but this is unlikely due to this publication bias.
If what is in the literature is not representative of the totality of the research, then the results
of the meta-analysis from an unrepresentative sample must be interpreted with caution.
Here is an example of a meta-analytic strategy that demonstrated the benefit of this
approach. Blinn-Pike, Worthy, and Jonkman (2007) wanted to arrive at an estimate of the
125
lan66845_04_c04_p101-130.indd 125
4/20/12 2:47 PM
CHAPTER 4
Chapter Summary
approximate percentage of college students who exhibit a serious gambling disorder.
Rather than conduct an entirely new study to examine this issue, they conducted a metaanalysis of 15 college student gambling studies through 2005. One of the beneficial features
of studying this topic is that there is a common metric used to measure gambling disorders, the South Oaks Gambling Screen (SOGS). Thus, each of the 15 studies had to use (at
least) the SOGS as an outcome (dependent) variable. Other screening criteria for inclusion
in the meta-analysis included (a) the percentage of disordered gambling in students had
to be reported; (b) the study had to come from a peer-reviewed publication or dissertation;
and (c) the study had to be conducted in the United States or
Canada. Blinn-Pike et al. (2007)
were able to estimate that the
percentage of disordered gamblers when considering a college
student population is 7.89%.
The results of 15 studies to
arrive at this estimate may provide a more accurate estimate
of the overall existence of gambling problems among college
students as compared to a study
conducted on one campus.
Blinn-Pike, Worthy, and Jonkman used a meta-analytical
strategy approach to analyze 15 studies on college gambling to
approximate the percentage of college students with a serious
gambling disorder.
Hemis.fr/SuperStock
In the next chapter, we’ll look at
a very different type of research
design—the
single
subject
design—and we’ll explore the
insights achievable from this
valuable research method.
Chapter Summary
R
esearch conducted in controlled settings, such as a laboratory, can often lead to
powerful conclusions about behavior, and at times may yield insights into causeand-effect relationships, the gold standard desired by scientists. However, people
do not live in laboratories, but rather the real world, so a number of additional research
approaches are necessary to better understand complex behaviors influenced by numerous
variables in social settings. Quasi-experimental designs offer many different approaches
to understanding these potential influences to behaviors, beliefs, and perceptions. Observational designs and archival research strategies can lead us to insightful revelations about
the multitudinous factors that influence each of us. A psychologist equipped to conduct
research in an applied setting wants to be as prepared as possible, and this means that he
or she should have access to multiple methodologies to ensure that the questions of interest are answered with the strongest approach available, given the constraints of collecting
data in the real world.
126
lan66845_04_c04_p101-130.indd 126
4/20/12 2:47 PM
CHAPTER 4
Chapter Summary
Research designs from this chapter
Type of Study
QuasiExperimental
QuasiExperimental
QuasiExperimental
Design Name
Nonequivalent
Control
Groups
Time Series
Cohort
Symbolic
Representation
X = treatment or intervention
O = observation, dependent
variable measurement
O
X
O
O
O
Brief Features
This design is superior to the pretestposttest only design comparing the
effect of the independent variable
to a control group, but because of
the lack of random assignment,
the groups may not be roughly
equivalent at pretest.
O O O X O O O
Repeated observations pre and
post the independent variable
manipulation allow for the tracking
of trends.
N/A
This design tracks a similar group
of people over time, such as an age
cohort of a generation or a period
cohort such as young adults during
the time of the Vietnam War.
QuasiExperimental
Regression
Discontinuity
N/A
Starting with a large number of
individuals, an intervention is applied
to some individuals in the group
based on a cutoff score, and the key
outcome is whether or not the trends
occurring before the intervention
occur after the intervention (that is,
continuous or discontinuous).
Observational
Case Study
N/A
This involves the intensive study of
one individual while examining a
limited set of specific behaviors.
N/A
Participants are studied in their
natural environment by the
researcher, but the researcher only
observes and does not interact with
participants.
N/A
This set of techniques allows for
the analysis of qualitative data such
that trends can be analyzed and
conclusions can be drawn from the
data.
N/A
This methodology allows for the
outcomes of multiple studies on
the same topic to be combined
meaningfully and statistically to form
overall, general conclusions.
Observational
Archival
Archival
Naturalistic
Observation
Content
Analysis
Meta-Analysis
127
lan66845_04_c04_p101-130.indd 127
4/20/12 2:47 PM
CHAPTER 4
Concept Check
Concept Check
1. The following symbols would be described in words as
X
O
O
A. two groups; only one gets the treatment, and both get the posttest.
B. one group gets two treatments.
C. randomly assign participants to two groups; both get treatments, and one gets
posttest.
D. two groups get the same treatment.
2. In an interrupted time series design, the treatment is done
A.
B.
C.
D.
before observations.
in the middle of observations.
after the observations.
after a single observation.
3. A group of people who share a critical experience over the same span of time
defines a
A.
B.
C.
D.
time series.
cohort.
control group.
treatment group.
4. According to Greene (2003), the sequence of program evaluation phases is
A.
B.
C.
D.
causation, implementation, generation.
causation, generation, implementation.
implementation, generation, causation.
generation, implementation, causation.
5. Fundamental lessons for program evaluation (McCartney & Weiss, 2007) include
all of the following EXCEPT
A.
B.
C.
D.
adopt a fair and reasonable scientific explanation.
use mixed-methods designs.
discourage critique of the data.
synthesize all research findings.
Answers
1. A. Two groups; only one gets the treatment, and both get the posttest. The answer can be found in the
Introduction.
2. B. In the middle of observations. The answer can be found in Section 4.1.
3. B. Cohort. The answer can be found in Section 4.1.
4. D. Generation, implementation, causation. The answer can be found in Section 4.1.
5. C. Discourage critique of the data. The answer can be found in Section 4.1.
128
lan66845_04_c04_p101-130.indd 128
4/20/12 2:47 PM
CHAPTER 4
Key Terms to Remember
Questions for Critical Thinking
1. Think about the variety of research approaches studied to date. How will you
know what research approach is more appropriate than another with a particular
research question or population of interest? How do psychologists come to have
confidence in making these types of decisions? What will you need to do to build
your confidence in your ability to select a methodological approach that matches
the research questions you are interested in?
2. As you reflect back on your undergraduate career, you may see now that a
particular sequence of courses might have been more helpful than others. Thinking about quasi-experimental designs, how might you design a study to show
that one sequence of prerequisites is superior to a different set of prerequisites?
How would you select one quasi-experimental design over another? How would
you determine the superiority of one particular course sequence compared to
another?
3. You may have conversations from time to time with students who are attending
college a different way from how you are attending college currently. You might
make the argument that an online education is superior, listing the advantages
you have experienced. Another person who attended a “brick-and-mortar” campus may not think as highly of online coursework, and may provide his or her
list of advantages for that type of learning experience. How might you design a
quasi-experimental study to systematically examine the similarities and differences that exist among online, classroom, and hybrid educational approaches?
Key Terms to Remember
archival research A research methodology
that involves analysis of data from existing
records that were made in natural settings.
content analysis A method of analysis
that takes qualitative statements, writings,
and other forms of language and quantifies those data.
case study A research methodology that
focuses on a particular case of interest.
This case may be a person, a group, or
perhaps an organization. Case studies
can utilize qualitative and quantitative
methods.
generation phase The phase of program
development where a needs assessment
may be conducted prior to the implementation of a program, to see if the need justifies
the implementation of a program, followed
by program development and refinement.
causation phase The phase of program
evaluation in which the program evaluator
attempts to demonstrate and measure the
impact of the program.
cause and effect An analysis that attempts
to examine the causes and results of
actions or behaviors.
cohort A group of people who have shared
experiences over the same span of time.
implementation phase The phase of
program development where the program
begins, provides services to clients and the
community, and hopefully has a positive
impact.
meta-analysis A method of analysis that
provides an approach for combining the
results of multiple studies so that general
effects can be summarized.
129
lan66845_04_c04_p101-130.indd 129
4/20/12 2:47 PM
CHAPTER 4
Web Resources
naturalistic observation An observational
situation where the researcher does not
interact in the environment, but merely
observes it.
quasi-experiment When a participant
is not randomly assigned to a group but
instead assigned to a group based on characteristics that he or she already possesses.
nonequivalent control groups design
A design in which an experimenter is
unable to randomly assign participants to
the different groups due to external factors.
regression discontinuity design A study
design in which assignment into the treatment group or the control group is based
on a predetermined cutoff.
panel studies Studies that are designed to
measure the same participants at different
points over time.
time series design A study design that
includes consistent and multiple observations of a variable that is repeated a substantial number of times.
program evaluation An assessment of
the impact of a program based on its prestated goals.
Web Resources
Explanation and examples relating to quasi-experimental design; explains the lack of
randomization in these designs and how that can affect research.
http://allpsych.com/researchmethods/quasiexperimentaldesign.html
Outline and definition of the regression-discontinuity design, which measures the effectiveness of a treatment or program that a psychological researcher may be investigating
in his or her research project.
http://www.socialresearchmethods.net/kb/quasird.php
130
lan66845_04_c04_p101-130.indd 130
4/20/12 2:47 PM
5
Science Faction/Superstock
Single Subject Designs
Chapter Learning Outcomes
After reading and studying this chapter, students should be able to:
• identify different types of reversal/withdrawal designs and understand the benefits and
drawbacks of each.
• comprehend the appropriate usage of more complex single-subject designs, such as the multiple baseline and changing criterion designs.
• summarize the challenges to analyzing data from single-subject designs as well as appreciate
the applications and limitations of these designs.
lan66845_05_c05_p131-156.indd 131
4/20/12 2:48 PM
CHAPTER 5
Introduction
Introduction
A
t the close of the last chapter, you read about case studies and archival research.
So what’s the difference between a case study involving one participant and a
single-subject design involving one participant? In a single-subject design
(SSD), there is more experimental control of the presentation and withdrawal of the
independent variable levels. That is, the single-subject design can do true experimental
manipulations, whereas case studies are typically observation-based or based on archival
records. Thus, the advantage of
the single-subject design is that
cause-and-effect
conclusions
can be approximated but with
limited generalizability since
the data are typically based on
one participant.
The history of the single-subject
design comes from the traditions of behaviorism, behavior
therapy, and behavior analysis
(Freeman, 2003). However, the
applications of single-subject
designs are diverse and not limited to those with a behavioristic
A machine records brain waves as a participant performs a task
orientation—this methodologion a computer. Single-subject designs allow more experimental
cal approach has been used in
control and manipulation of the independent variables than
social work, education, cognicase studies.
tive rehabilitation, sport psyAge fotostock/SuperStock chology, counseling, and occupational therapy, to name a few
examples (Freeman, 2003). SSDs were used to modify the behavior of Little League baseball coaches (Martin, Thompson, & Regehr, 2004) and to help those with autism develop
better independent work and play skills (Hume & Odom, 2007).
Voices from the Workplace
Your name: Jennifer B.
Your age: 35
Your gender: Female
Your primary job title: Director of Operations
Your current employer: Creative Community Options
How long have you been employed in your present position?
1 month
What year did you graduate with your bachelor’s degree in psychology?
1994 (continued)
132
lan66845_05_c05_p131-156.indd 132
4/20/12 2:48 PM
CHAPTER 5
Introduction
Voices from the Workplace (continued)
Describe your major job duties and responsibilities.
Operate a community-based residential program for adults with intellectual and developmental disabilities, as well as chronic mental illness. Also operate a supported employment program for the
same population groups. Both programs provide support to approximately 140–150 individuals. The
operations involve overseeing 10 to 12 coordinators who complete the program planning and provide
the direct supervision to the direct support professionals. In that oversight is managing service rates,
contracts, ensuring programs are operating within the state and federal regulations, providing support
to the coordinators with challenging staff issues as well as concerns with the individuals we provide
support to. There is also significant involvement with the budgeting process of the organization, working to develop and maintain relationships with stakeholders of the organization and assisting in team
meetings when there are significant challenges in providing service to an individual.
What elements of your undergraduate training in psychology do you use in your work?
In my current position, one of the programs provides services to individuals with chronic mental illness. My degree is helpful in providing support to the coordinator of those individuals. Many of the
intellectually disabled individuals also have varying degrees of mental illness. The oversight, both
directly and indirectly of 170+ employees requires regular use of the training I received in my undergraduate work as well.
What do you like most about your job?
I have been in the human service field since leaving undergraduate school. I have always worked
with individuals that have some form of a disability. I have a strong passion to see that individuals
with intellectual disabilities are given the same opportunities in their life as those without. I get to
support the individuals we serve, through the administration of our programs, in living their life as
they want to in their own homes, apartments and in jobs at businesses in their own community.
Through my experiences as a direct support professional, program manager/coordinator, and Medicaid case manager, I can now develop current coordinators and assist them in learning new skills
through which will provide the highest of quality of service to the individuals who receive support
from our agency.
What do you like least about your job?
In this field, there are many regulations both federal and through our state which impact the job
that we do daily. I feel the regulations are necessary due to past injustices our field allowed to occur.
However, many times the regulations go too far and impact the quality of service as we must focus
on items which don’t directly impact the people we serve. I now have a strong desire to begin to
work with legislators and representatives in an effort to impact the laws and regulations surrounding
this field.
Beyond your bachelor’s degree, what additional education and/or specialized training have you
received?
Various training regarding the support of individuals with Autism: TEACCH, PECS, Training in Positive
Behavior Support, MANDT, many conferences on supervision of employees
What is the compensation package for an entry-level position in your occupation?
$28,000–$34,000 salary for a coordinator or case manager. $9.00–$11.00/ hour for direct support staff.
What benefits (e.g., health insurance, pension, etc.) are typically available for someone in your
profession?
Health insurance with dental and vision in many places. 401K. Holidays off—unless you are working as a
direct support professional. Vacation, sick time or PTO. Emergency leave time. Flexible hours. (continued)
133
lan66845_05_c05_p131-156.indd 133
4/20/12 2:48 PM
CHAPTER 5
Introduction
Voices from the Workplace (continued)
What are the key skills necessary for you to succeed in your career?
Passion for the individuals served; Organization skills; Ability to manage large amounts of paperwork,
tight deadlines, and time to handle immediate person-served needs. Soft skills—working well with
people, developing resources and information, teaching as opposed to counseling.
Thinking back to your undergraduate career, what courses would you recommend that you believe are
key to success in your type of career?
Social Work courses would have been very helpful to me. I had to learn how to develop assessments
on the job and didn’t get that in a psychology degree. The assessments in the human service field are
looking at the main life domains—where you live, work, your finance situation, social life, spiritual life,
etc. Classes to learn how to develop a thorough social history and how to work with parents, guardians or family to get that delicate information. For my current place in my career, management classes
are key.
Thinking back to your undergraduate career, can you think of outside of class activities (e.g., research
assistantships, internships, Psi Chi, etc.) that were key to success in your type of career?
My degree in psychology did not require an internship as the social work degree did. I firmly believe
that there should be an internship requirement for a psychology degree. Getting out during undergraduate work and obtaining an entry-level position which doesn’t require a degree is a great way to
start to gain experience as well as to identify what field you want to work in. Psychology is a rather
general degree at this point, which is a great cornerstone to further graduate work but is starting to
be difficult to use in the human service field. If I were to graduate now with my psychology degree, I
would have a difficult time getting to this level in my field. Many positions are requiring a social work
license. I am fortunate that the state of Iowa doesn’t require it as often as many states do at this time.
As an undergraduate, do you wish you had done anything differently? If so, what?
I would have kept my study in psychology and sociology; however I would have at least minored, if not
had a triple major in Social Work. That would allow me to now obtain my Social Work license.
What advice would you give to someone who was thinking about entering the field you are in?
Obtain a job with an organization that provides services to individuals with a disability. Get a direct
support job and learn the field directly with the individuals we support. They are your greatest teachers and will help establish the passion for what we do. Many of the positions in my field require that
you have at least a year or two of direct support experience. It is crucial. Attempt to vary the disability
ranges. It is best to have experience with individuals that have a brain injury, individuals with an intellectual disability and individuals with a chronic mental illness. Substance abuse is also an area of this
field which is helpful to have experience with.
If you were choosing a career and occupation all over again, what (if anything) would you do
differently?
Absolutely nothing. I am very fortunate to be where I am at in my career at this time. Every position I
have had, has given me critical experience and information which has allowed me to develop to this
place in my professional life.
Copyright © 2009 by the American Psychological Association. Reproduced with permission. The official citation that should be used in referencing this material is R. Eric Landrum, Finding Jobs With a
Psychology Bachelor’s Degree: Expert Advice for Launching Your Career, American Psychological Association, 2009. The use of this information does not imply endorsement by the publisher. No further
reproduction or distribution is permitted without written permission from the American Psychological Association.
134
lan66845_05_c05_p131-156.indd 134
4/20/12 2:48 PM
CHAPTER 5
Section 5.1 Reversal/Withdrawal Designs
5.1 Reversal/Withdrawal Designs
M
ost SSDs begin with a phase where baseline data are collected concerning the
behavior of interest; this phase is labeled the A phase in a single-subject design.
After multiple observations to establish a stable baseline (more on this later),
some sort of intervention (i.e., independent variable manipulation) is introduced with th...
Purchase answer to see full
attachment