Journal of Substance Abuse Treatment 46 (2014) 498–505
Contents lists available at ScienceDirect
Journal of Substance Abuse Treatment
From counselor skill to decreased marijuana use: Does change talk matter?☆
Elizabeth Barnett, M.S.W. a,⁎, Theresa B. Moyers c, Steve Sussman, Ph.D., FAAHB, FAPA a, b,
Caitlin Smith, M.A. b, Louise A. Rohrbach, Ph.D. a, Ping Sun, Ph.D. a, Donna Spruijt-Metz, Ph.D. a
a
b
c
Department of Preventive Medicine, University of Southern California
Department of Psychology, University of Southern California
Department of Psychology, University of New Mexico
a r t i c l e
i n f o
Article history:
Received 6 March 2013
Received in revised form 31 October 2013
Accepted 12 November 2013
Keywords:
Motivational interviewing
Adolescent
Marijuana use
Mediation
Mechanisms of change
a b s t r a c t
Client language about change, or change talk, is hypothesized to mediate the relationship between counselor
fidelity in motivational interviewing (MI) and drug use outcomes. To investigate this causal chain, this study
used data from an MI booster delivered to alternative high school students immediately after a universal
classroom-based drug abuse prevention program. One hundred and seventy audio-recorded MI sessions
about substance use were coded using the motivational interviewing skill code 2.5. Structural equation
modeling showed that percentage of change talk on the part of the client mediated three of the four
relationships between MI quality indicators and marijuana outcomes, while percentage of reflections of
change talk showed a main effect of counselor skill on marijuana outcomes. Findings support change talk as an
active ingredient of MI and provide new empirical support for the micro-skills of MI.
© 2014 Elsevier Inc. All rights reserved.
1. Introduction
Motivational interviewing (MI), a client-centered counseling style
used for the exploration of ambivalence about behavior change
(Miller & Rollnick, 2002), has been identified as a promising
intervention for adolescent substance use treatment (Macgowan &
Engle, 2010) and appropriate for addressing a range of substances
across a variety of settings (Barnett, Sussman, Smith, Rohrbach, &
Spruijt-Metz, 2012; Jensen et al., 2011). MI also has a well-specified
technical model, whereby counselor behaviors or skills (X) are
expected to promote client language predictive of change or “change
talk” (M), and this language influences outcomes (Y; see Fig. 1;
adapted from Miller & Rose, 2009). A growing body of evidence exists
to support this hypothesized causal mechanism (Moyers, Martin,
Houck, Christopher, & Tonigan, 2009; Pirlott, Kisbu-Sakarya, DeFrancesco, Elliot, & MacKinnon, 2012).
One issue in measuring causal models in MI concerns how this
method is defined. The counselor skills within MI are commonly
measured using objective behavioral rating schema designed to assess
MI sessions (Houck, Moyers, Miller, Glynn, and Hallgren (2013). The
instruments measure the micro-skills of MI by categorizing counselor
statements as open or closed questions, complex or simple reflections.
☆ This paper was supported by a grant from the National Institute on Drug Abuse
(DA020138).
⁎ Corresponding author at: University of Southern California, Institute for Prevention
Research, Department of Preventive Medicine, Soto Street Building, 2001 N. Soto Street,
MC 9239, Los Angeles, CA 90089. Tel.: +1 562 208 6881.
E-mail address: embarnet@usc.edu (E. Barnett).
0740-5472/$ – see front matter © 2014 Elsevier Inc. All rights reserved.
http://dx.doi.org/10.1016/j.jsat.2013.11.004
They further create composite measures of counselor speech that
demonstrates adherence to the “way of being” prescribed in MI. MI
consistent behaviors (MICO) include instances of asking permission
before giving advice or making suggestions, offering support,
affirming, emphasizing personal choice and control, and sometimes,
depending upon the measurement instrument used, may include
open questions and reflections. MI inconsistent behaviors (MIIN)
include instances of confronting, warning, and giving advice without
permission and sometimes closed questions.
To date much of the evidence for a causal path or mediation has
been shown using MICO as the predictor (Moyers et al., 2009; Pirlott
et al., 2012; Vader, Walters, Prabhu, Houck, & Field, 2010). Because
MICO is a composite variable, none of the studies provide guidance as
to which of the MI micro-skills is the most effective at eliciting change
talk. Since counselors must decide which specific skill to employ as a
session unfolds, and these choices theoretically influence the
direction of the subsequent interactions between the client and
counselor, empirical evidence to support choosing one skill over the
other could increase both the efficiency and the efficacy of MI.
Current research has shown a relationship between some of these
specific skills and treatment outcomes (path c in Fig. 1). Gaume, Gmel,
Faouzi, and Daeppen (2009) modeled the unique MI counselor skills
separately to predict alcohol use at 12-month follow-up in a study of
alcohol-using adults in an emergency department. In so doing, they
found significant relationships between complex reflection, the ratio
of reflections to questions, and MIIN on outcomes when controlling
for client ability language. Similarly, McCambridge, Day, Thomas, and
Strang (2011) found a significant relationship between percentage
complex reflection and marijuana cessation at 3-months in a sample
of youth ages 14–19 attending further education colleges in London.
E. Barnett et al. / Journal of Substance Abuse Treatment 46 (2014) 498–505
499
a*b
(indirect effect)
Change Talk (M)
b
a
Counselor Skill (X)
Outcome (Y)
c’
(direct effect)
Counselor Skill (X)
c
(total/main effect)
Outcome (Y)
Fig. 1. Proposed mediation model illustrating the hypothesized causal mechanisms of MI being tested in this analysis.
Research has also been done to investigate the relationship
between counselor skills and client language about change (i.e.
change talk (CT) and counter change talk (CCT); path a in Fig. 1).
Sequential analyses have provided probabilistic support that MICO
behaviors are more likely to be followed by CT, while MIIN behaviors
are more likely to be followed by CCT (Gaume, Bertholet, Faouzi,
Gmel, & Daeppen, 2010; Gaume, Gmel, Faouzi, & Daeppen, 2008;
Moyers & Martin, 2006; Moyers et al., 2009). Regression analyses of
non-sequential count data have similarly shown associations between
MICO and the amount of CT (Catley et al., 2006) in MI sessions.
Further, experimental manipulations of counselor attempts to elicit
CT have resulted in higher levels of CT when counselors intend to
evoke it (Glynn & Moyers, 2010). Morgenstern et al. (2012), in a
three-condition RCT, found that the directive elements of MI are more
instrumental in producing CT than the non-directive elements.
CT has also been shown to predict client outcomes (path b in Fig. 1)
in several studies; although, as with MICO, it has been conceptualized
and defined slightly differently across research projects. Support has
been found for a single category of combined CT to predict alcohol use
outcomes (Campbell, Adamson, & Carter, 2010; Gaume, Bertholet,
Faouzi, Gmel, & Daeppen, 2013; Moyers et al., 2007) as well as
improvements in substance use rates in a sample of homeless youth
(Baer, Beadnell, Garrett, Hartzler, & Wells, 2008). Measures of the
strength of change talk, rather than its frequency, indicate that the
strength of client ability language predicted drinking rates and drug use
(Amrhein, Miller, Yahne, Palmer, & Fulcher, 2003; Gaume, Gmel, Faouzi,
et al., 2008).
Finally, mediation analyses are important for investigating the
mechanisms by which MI works as they aid in formulating a more
complete understanding of what is occurring during treatment.
Moyers et al. (2009) found significant main effects of MICO on
outcomes (path c) and significant indirect effects (path a*b) for MICO,
CT and drinks per week at 5-week follow-up after the personalized
feedback session. Vader et al. (2010), in a sample of college age
students, did not find evidence for significant indirect effects (path
a*b) though there were significant relationships between the MICO
and CT (path a) and CT and 3-month alcohol use (path b) in the
condition receiving personalized feedback. They did not report any
information about a main effect for MICO on alcohol use. Morgenstern
et al. (2012) conducted a 3-armed randomized controlled trial
comparing a standard care control, an MI condition that included
personalized feedback and other directive activities to elicit client
change talk such as importance/confidence rulers, and a spirit only
condition which relied on the non-directive elements of MI. They
found significant effects for condition on commitment language (path
a) and a trend toward significance for commitment language on
alcohol use at 7-day follow-up (path b), but no significant indirect
effects and no main effects. Finally, Pirlott et al. (2012), in a study
using personalized feedback, investigated the use of MI to encourage
fruit & vegetable consumption. This study showed significant effects
for MICO on total CT (path a) and CT on 12-month fruit & vegetable
consumption (path b), significant indirect effects (path a*b) and no
main effect (path c).
Taken together, these mediation results are inconclusive. While
the rigor and design of these studies are solid, any comparison of their
results should be made cautiously in light of the fact that they often
defined their predictors, mediators, and outcome variables differently,
used different versions of similar coding instruments, had widely
varying length of follow-ups, and used different statistical tests for
mediation. Although it is premature to draw strong conclusions about
the MI technical model at this point, the initial evidence supports
further investigation into the proposed mediation. Also, it is
important to note that alternative mediation models based on the
relational elements of the complete theoretical model are not
addressed in this study (Miller & Rose, 2009).
1.1. The current study
Using data from the MI condition of a 3-armed randomized
controlled trial of a universal classroom-based substance abuse
prevention program, we investigated whether the percentage of
change talk (PCT) present in an MI session mediates the relationships
between specific behaviors prescribed for MI fidelity and marijuana
outcomes. As a universal prevention program, outcomes included
prevention for non-users at baseline as well as reduction and cessation
for adolescents already experienced in drug use. While the main trial
addressed additional drug outcomes, marijuana use was the only
outcome that showed a trend toward significance (p = .07) suggesting that the MI condition performed better than the classroom-only
condition (Sussman, Sun, Rohrbach, & Spruijt-Metz, 2012). In this
study, we investigated five indicators of MI quality as predictors: 1)
the percentage of complex reflections (PCR), 2) the percentage of open
questions (POQ), 3) percentage of reflections of change talk (PRCT), 4)
percentage of MICO (PMIC) behaviors and 5) the reflections to
questions ratio (RQR). This study is the first to conduct mediation
analyses on the individual MI skills and the first to do so with
structural equation modeling (SEM). In a series of 5 SEM models, we
tested our hypotheses that PCT would mediate the relationship
between PCR, POQ, RQR, PRCT, PMIC, and marijuana use outcomes.
2. Methods
2.1. Procedure and Sample
The sample used in this study is derived from the 7th randomized
trial of Project Toward No Drug Abuse, a classroom-based substance
abuse prevention program. Twenty-four alternative high schools in
Southern California participated. In total, 2397 students were enrolled
500
E. Barnett et al. / Journal of Substance Abuse Treatment 46 (2014) 498–505
Students enrolled in
24 participating schools
N = 2397
Consented
N = 1704
Classroom Only Condition
N = 562
Classroom + Motivational Interviewing
Condition
N = 573, Total Sessions = 1040
Control Condition
N = 569
Session focused on Substance Use
N = 235
Recordings not available due to
participant refusal or technical problems
N = 12
Recordings
N = 223
Unique students represented in recordings
N = 170
Students with 1 Year Follow-Up Data
N = 122
Fig. 2. Consort diagram.
in the selected classes, and 1,704 (71.1%) were consented to
participate in the study. Of these, 573 students at 8 schools were
assigned to the 3-session MI booster condition and completed the pretest data collection. In order to be included in the study, students
under the age of 18 were required to return a signed parental consent
form and a signed subject assent. Parental consent was not required
for students over 18 years old. The University of Southern California's
Institutional Review Board approved all study procedures. More detail
about school selection can be found in Lisha et al. (2012).
In the MI booster condition, students were provided up to 3 MI
sessions; the first occurred at school within 1- to 3-days of the
classroom program, and the following two sessions were conducted
via telephone at 3- to 4-month intervals. Hand-held devices were
used for recording during the in-person sessions, while recorded
telephone lines were used for the 2nd and 3rd sessions. During the
first session, students discussed their impressions of the program and
their drug use. If drug use did not appear problematic they were
invited to choose a target behavior from an agenda setting tool that
included topics ranging from high school graduation and employment
to substance use.
Recordings were identified as discussing substance use from notes
kept by the MI interventionists. Coders then independently assessed
whether the sessions met the criteria as having a substance use target.
In order to be considered a substance use target, substance use had to
be addressed with the exploration exercise used during the MI
session. For example, if a participant reported that they had cut back
on their cigarette use, and the interventionist proceeded to explore
job seeking, this session would not be considered a substance use
target. The final sample of recordings excluded all MI sessions aiming
to affect a non-substance use related outcome as these were expected
to be irrelevant to changes in drug use. Meanwhile we included all
sessions related to any drug use as data from one study suggests that
the effects of an MI session may generalize to other substances
(McCambridge & Strang, 2004).
Of the 1040 MI sessions conducted, 235 discussed substance use.
Twelve of these substance use sessions did not have recordings,
resulting in 223 to be included in the coded sample. In order to
establish independence between observations only one substance
use related session per student was included in the final sample
(N = 170). Where multiple substance use sessions existed, the first
available session was chosen. Of the youth represented in the
substance use recordings, 122 completed the post-test assessment
(see Fig. 2: Consort diagram). The final sample includes data from 17
interventionists having from 1 to 49 sessions in the sample. All
interventionists participated in standardized training and regular
supervision conducted by a member of the Motivational Interviewing
Network of Trainers (MINT). Extensive details on the training and
supervision of the interventionists and the content of the booster are
published elsewhere (Barnett et al., 2012).
2.2. Coding and parsing
We coded the sample of substance MI sessions pertaining to
substance use using the MISC 2.5 (Houck et al., 2013). The MISC 2.5 is
a hybrid of the MISC 2.1 and the Sequential Code for Observing
Process Exchanges (MI-SCOPE; Martin, Moyers, Houck, Christopher, &
Miller, 2005) designed to optimize the features from each coding
scheme. Specifically the MISC 2.5 allows for the capture of specific
behaviors from the MISC 2.1, as well as valenced reflections and
temporal order from the SCOPE. Like all versions of the MISC, it codes
counselor and client language into mutually exclusive and exhaustive
categories. Coding was conducted using the Center on Alcoholism
E. Barnett et al. / Journal of Substance Abuse Treatment 46 (2014) 498–505
Substance Abuse and Addictions (CASAA) Application for Coding
Treatment Interactions (CACTI; Glynn, Hallgren, Houck, & Moyers,
2012). This software automates the parsing of recordings and stores
sequential coding of each utterance. Using this process for parsing
ensures that all coders code the same utterances, thereby increasing
reliability. Although CACTI software does not require or utilize
transcripts, we transcribed our entire sample of recordings for ease
of parsing and coding.
Coding was performed in two passes. In the first pass, coders
parsed the entire recording into utterances, or thought units. MISC
coding requires that any two consecutive counselor statements that
merit different codes (e.g., a reflection followed by a question), be
identified as separate utterances. Utterances of client change language
are parsed into separate utterances, even if the client emits
consecutive utterances from the same change talk category.
In the second pass, a different coder applied codes to each client
and counselor utterance. Each counselor utterance was assigned a
behavior skill code. Utterances were coded as open (OQ) or closed
questions (CQ) and simple (SR) or complex reflections (CR) with a
positive (+), negative (−), neutral (0), or both positive and negative
(±) valence. Valence refers to whether a reflection contains content
that directs, or steers, the conversation toward change, away from
change, or contains content that is unrelated to change. MICO
included specific codes for affirming, supporting, and asking permission before giving advice; while MIIN included codes for confronting,
warning, and giving advice without permission; and “other” included
codes for providing information about the session, filler, and
comments designed to facilitate conversation. Meanwhile, each client
utterance was categorized as either change talk (CT), counterchange
talk (CCT), or unrelated to change (FN). CT includes statements of
commitment (“I will cut back on smoking”), taking steps (“I’ve
already slowed down.”), desire (“I want to quit.”), ability (“I think I
can do it.”), reason (“I have to stop for my health.”), need (“I need to
cut back so I can keep a job.”) and “other” statements that do not fall
into the previous categories. CCT includes statements counter to
commitment (“There’s no way I will stop.”), taking steps (“I had a
drink last night.”), desire (“I really don’t want to.”), ability (“There is
no way I’d be able to give it up.”), reason (“It’s not affecting my
health.”), need (“I really don’t think I need to change.”) and “other”
statements that do not fall into the previous categories.
2.3. Training and Supervision of Coders
We provided 5 undergraduate and graduate students 40 hours of
initial training in the MISC 2.5 and the CACTI software. Weekly coding
meetings were held throughout the project to improve and maintain
reliability. During the training period, all coding disagreements were
resolved by a supervisor. Coders practiced on a series of nonsubstance use recordings until their inter-rater reliability was at
criterion of 0.60 using established intraclass correlation (ICC)
501
guidelines (Cicchetti, 1994). We randomly selected 20% of our
coded sample using a random number generator for double coding.
These 47 recordings were double coded in order to calculate final ICCs.
Cicchetti's criterion identifies ICCs below .40 as poor, .40–.59 as fair,
.60–.74 as good, and above .75 as excellent. For our data, final ICCs for
counselor codes were .94 for open questions, .80 for closed questions,
.94 for reflections overall, .48 for simple reflections, .45 for complex
reflections, .84 for reflections of change talk, .82 for reflections of
counter change talk, .68 for MI-consistent behaviors and .29 for MIinconsistent behaviors. Client codes were .92 for change talk, .86 for
counter change talk, and .88 for neutral responses. These results
indicate that coders had some difficulty differentiating simple
reflections from complex reflections, and difficulty reliably identifying
MIIN behaviors, which occurred infrequently. Only seven (.04%)
recordings in the final sample contained any MIIN-behaviors in
our dataset.
2.4. Measures
2.4.1. Predictors
For these analyses, five summary variables were constructed. We
used four standard measures of quality from the MISC 2.5 and
constructed one additional measure using the valenced reflection
data. Summary variables for PCR, POQ, PRCT, RQR, and PMIC (see
Table 1 for variable formulas) were calculated using the coded
counselor data.
2.4.2. Mediator
For this analysis, in order to account for the highly variable length
of sessions in this sample, we used percentage change talk (PCT) as
the mediator.
2.5. Outcome
An ordinal measure with equal spacing between levels of
marijuana use and a true zero was collected by asking respondents
how many times they used marijuana during the past 30 days.
Subjects were provided with twelve response options ranging from
0 to 100 (Sussman et al., 2012). The log of this variable was used to
account for non-normality. At baseline, data were collected as a paper
and pencil measure administered at the subjects school by project
staff, unrelated to the MI intervention. One-year follow-up data were
gathered either in person at the school if the subject was still enrolled
or via the telephone by the same staff as baseline collection.
2.6. Analytical Approach
Structural equation modeling (SEM) was conducted to test for
mediation using Mplus (v.6) (Muthén & Muthén, 1998). SEM allows
for more precise estimates of direct and indirect effects than
Table 1
Measurement details for all cariables included in the final models.
Predictor
POQ: Percent Open Questions
PCR: Percent Complex Reflection
RQR: Reflection to Question Ratio
PRCT: Percent Reflection of Change Talk
PMIC: Percent MI Consistent Behaviors
Mediator
PCT: Percent Change Talk
Outcome
# of times of marijuana use in the past 30 days at 1 year follow-upb
a
b
OQ/(OQ + CQ)
CR + CR− + CR0 + CR±/(CR+ + CR− + CR0 + CR ± + SR+ + SR− + SR0 + SR±)
(CR+ + CR− + CR0 + CR ± + SR+ + SR− + SR0 + SR±)/(OQ + CQ)
(CR+ + SR+)/(CR+ + CR− + CR0 + CR ± + SR+ + SR− + SR0 + SR±)
MICO/(MICO + MIINa)
CT/(CT + CCT + FN)
0, 1–10, 11–20, 21–30, 31–40, 41–50, 51–60, 61–70, 71–80, 81–90, and 91–100+
MIIN = MI inconsistent behaviors; this variable was not included in the final models due to lack of variance.
Past 30 day marijuana use at baseline was included in all models as a covariate.
502
E. Barnett et al. / Journal of Substance Abuse Treatment 46 (2014) 498–505
Table 2
Univariate and bivariate statistics for quality indicators.
POQ
PCR
PRCT
RQR
PCT
Percent Open Questions (POQ)c
1.00
Percent Complex Reflection (PCR)c
0.22
1.00
Percent Reflection of Change Talk
0.09 −0.15
1.00
(PRCT)c
0.36
0.29
0.07
1.00
Reflection: Question Ratio (RQR)c
Percent Change Talk (PCT)c
0.19
0.22
0.76
0.18
1.00
Number of Observations
170
168
170
170
170
Mean
0.56
0.57
0.42
1.29
0.33
Std Dev
0.18
0.26
0.21
0.61
0.17
Bold indicates significance p b .05.
traditional regression approaches (Bentler & Chou, 1987). Mplus
provides estimates for the relationship between indicator and
mediator (path a), the relationship between mediator and
outcome controlling for indicator (path b), the main effect, or
relationship between indicator and outcome (path c), the direct
effect, or relationship between indicator and outcome when
controlling for the mediator (path c’) and indirect effects (path
a*b) using the delta method (Bishop, Fienberg, & Holland, 1975).
Mplus uses maximum likelihood estimation to retain data from all
cases, including those with missing data at follow-up.
We tested 5 models, one for each of the following MI behavioral
skill measures (POQ, RQR, PCR, PRCT, and PMIC). All models included
PCT as the proposed mediator, logged marijuana use as the outcome,
and controlled for logged baseline marijuana use. All mediation
results are presented as both standardized and unstandardized
estimates. Attrition analyses were conducted in two ways. First,
demographics and baseline alcohol, cigarette and marijuana use were
used to predict those without 1-year follow-up data. Second, the SEM
models were run on the sample with complete data to determine if
results differed from the larger sample.
3. Results
The sample investigated in this study included 170 youth (70%
male, 71% Latino, with a mean age of 16.7 years), with reported past
30 day drug use of 68% for alcohol use, 59% for cigarette use, and 36%
for other drugs. Forty percent (40%) reported not using marijuana in
the past 30 days, while 36% reported being a daily or near daily users
of marijuana. Attrition analyses showed no significant predictors of
dropout, and results from SEM models with only youth with complete
data did not differ from models run with the entire sample.
The sessions had on average 56% POQ, 57% PCR, 42% PRCT, 33% PCT
and an RQR of 1.29. Due to the infrequency of MIIN behaviors in the
dataset we could not include PMIC in the final analyses. Table 2 shows
that although MI skill variables are significantly correlated, all
correlations were below .36 with the exception of PRCT and PCT
(r = 0.76).
Main effects (path c): Only one model showed a main effect
between the MI indicator and drug use outcomes. PRCT directly
influenced marijuana use (β = − 0.19, p b .05); all others had
coefficients smaller than − 0.06 and were non-significant. All models
controlled for baseline drug use.
Indirect effects (path a*b): Significant indirect effects of MI skill on
marijuana use were found for POQ (β = − 0.05, p b .05), PCR
(β = − 0.06, p b .05), and a trend toward significance was found
for RQR (β = − 0.04, p = .07). Results for an indirect effect of PRCT
were not significant (β = − 0.18). Results for all indicators are
presented in Table 3. Fig. 3 presents one indicator, PRCT, as a path
model for illustrative purposes.
Additional post-hoc analyses were conducted on the limited
sample of youth (n = 74) who had a target behavior of marijuana.
Results showed a similar trend for a main effect of PRCT on outcomes
(β = − 0.88, p b .10), and significant relationships between MI skills
and change talk (path a) and change talk and outcome (path b) for the
other MI skills. These findings are consistent with the overall findings
of mediation, however due to decreased sample size, significant
indirect effects were not seen.
4. Discussion
The goal of this study was to compare the relative strength of the
micro-skills of MI in a test of the hypothesized mediation model of MI.
Overall, significant indirect effects were more common than significant main effects of counselor skill on outcomes. We found evidence
of percent change talk as a mediator (i.e. significant indirect effects) of
the relationship between marijuana outcomes and POQ (p b .05), PCT
(p b .05), and RQR (p b .10). The strength of these relationships was
quite similar leaving no strong conclusion about which micro-skill is a
better predictor of outcome. Only PRCT behaved differently, showing
a main effect on marijuana outcomes (β = − .19, p b .05), and no
significant indirect effect via percent change talk.
4.1. Lack of Main Effects
We propose two explanations for this lack of main effects for POQ,
PCR, and RQR on drug use. First, we propose that seeing no main
effect, but a significant indirect effect, suggests that there is no reason
to believe that a high percentage of a particular indicator, e.g. open
questions, alone, would predict change (path c); our findings suggest
that it is only when these open questions result in change talk (path a)
that one would presume change to follow (path b). For example, we
would not expect open questions such as “What are the reasons you
drink?” to result in expression of change talk. In this case, the
association between open questions and improved outcomes may be
Table 3
Mediation results for MI quality indicators predicting change in marijuana use at 1 year follow-up.
n
Percent Open Questions
Percent Complex Reflection
Percent Reflection of Change Talk
Reflection to Question Ratio
160
158
160
160
X–NM
M–NY
Indirect Effect
Direct Effect
Total/Main Effect
a
b
a*b
c'
c
0.22⁎⁎(0.21⁎⁎)
0.25⁎⁎⁎(0.14⁎⁎⁎)
0.86⁎⁎⁎(0.69⁎⁎⁎)
0.18⁎(0.05⁎)
−0.22⁎⁎(−0.82⁎⁎)
−0.24⁎⁎(−0.87)
−0.20 (−0.77)
−0.22⁎⁎(−0.83⁎⁎)
−0.05⁎(−0.17⁎)
−0.06⁎(−0.14⁎)
−0.18 (−0.53)
−0.04+ (−0.04+)
0.01
0.03
−0.02
0.03
(0.03)
(0.08)
(−0.05)
(0.03)
−0.04 (−0.14)
−0.03 (−0.06)
−0.19⁎(−0.57⁎)
−0.02 (−0.02)
Standardized (unstandardized) results; X = predictor variable; M = mediator; Y = outcome variable. a = percent change talk on quality indicator; b = percent change talk on
1 year marijuana outcome controlling for quality indicator; c' = effect of quality indicator on 1 year marijuana outcome controlling for percent change talk; c = main effect of
quality indicator on 1 year marijuana outcome. All models control for baseline marijuana use.
+
p b .10.
⁎ p b .05.
⁎⁎ p b .01.
⁎⁎⁎ p b .001.
E. Barnett et al. / Journal of Substance Abuse Treatment 46 (2014) 498–505
Indirect Effect Model
a
0.86*** (.69)
503
a*b = -0.18 (-.53)
Percentage
Change Talk (M)
Percent Reflection
Change Talk (X)
c’
-0.02 (-.05)
b
-0.2 (-.77)
Marijuana Use at OneYear Follow-up (Y)
Main Effect Model
Percent Reflection
Change Talk (X)
c
-0.19* (-0.57)
Marijuana Use at OneYear Follow-up (Y)
Fig. 3. Results of path model for percent reflection of change talk on marijuana use at 1-year follow-up controlling for baseline drug use. Significant standardized (and
unstandardized) estimates, and p values + p b .10 *p b .05, **p b .01, ***p b .001.
related to the valence or direction of the question. In other words,
knowing the valence of a question may be much more informative
than just knowing that an open question occurred. Similarly, having a
high percentage of complex reflections or more reflections than
questions, while ignoring the valence of the component skills, may tell
very little about subsequent change talk and ultimately behavior
change. In contrast, knowing the percentage or ratio of positively
valenced vs. negatively valenced skills may tell us much more.
Second, while conventional understanding of mediation holds that
if there is no main effect between the predictor and the outcome, then
there cannot be mediation, many have argued that due to the timing
of predictors, mediators and outcomes, inadequate power, or differing
direction of effects, mediation may exist even in cases where a main
effect does not appear (Hoyle & Kenny, 1999; Kenny, Kashy, & Bolger,
1998; MacKinnon, Fairchild, & Fritz, 2007; Shrout & Bolger, 2002;
Zhao, Lynch, & Chen, 2010). In this case it is likely that main effects
were not detected due to the length of time between the measurement of the predictor (counselor in-session skill) and the outcome
variable (marijuana use at 1 year follow-up).
While differences in measurement make it difficult to compare
results, main effects between counselor skills and behavior change
have been found in other studies using some of these same measures.
The relationship between PCR and outcomes was found again in a
study by McCambridge et al. (2011). In a sample of adolescents
attending alternative educational institutions in London, they found
significant relationships between PCR and marijuana use outcomes at
3-month follow-up. Because this study did not measure client
language or valence of counselor responses, they could not control
for or investigate these variables as mediators. Their findings may
have been the result of highly skilled reflective listening that
emphasized CT over CCT. If their interventionists were trained to
reinforce change talk, a higher percentage of CR may have been a
proxy for higher PRCT.
In addition, Gaume et al. (2009) found a main effect of the
counselor's PCR and PMIC after controlling for one category of change
talk, client ability language. This finding suggests that ability language
may operate differently than other categories of change talk. It may
represent client confidence or self-efficacy about change, more than
client motivation or the importance of behavior change. This notion
appears to be supported by findings from Martin, Christopher, Houck
& Moyers (2011) whereby factor analysis revealed that ability
language had a unique relationship to outcomes than did the other
categories of change talk. Additional investigations into the association between confidence/self-efficacy and change might be important
for establishing ability language as a unique phenomenon demanding
differential treatment.
4.2. Percent Reflection of Change Talk
Our findings reinforce the importance of the directional
component of MI micro-skills and have implications for the use of
this counseling method. The MISC 2.5 codes the valence or direction
of the counselor's response to a client's statement about change by
indicating whether a reflection is toward, away, or neutral about
change. We further summarized these codes to create a new
indicator of the percentage of reflections of change talk (PRCT) that
captures the number of positive or toward change reflections over
all reflections. In our data, PRCT, the percentage of the session
during which the counselor specifically reflected change talk, was
the only skill to demonstrate a main effect on outcomes. PRCT
differs from the other indicators because it is a discrete counselor
behavior conceptually tied to the explicit counseling goal, or
behavior change target. To a novice listener, counselor reflections
may appear to constitute a neutral mirroring of the content the
client has offered. However, objective ratings indicate that this is
not the case. Counselors often add meaning, feeling or direction, to
their reflections, show a preference in choosing which aspects of the
client's speech to reflect, and reframe “negative” client statements
to “positive” ones. For example, clients often present change talk
and counter change talk together (“I want to …, but….”) and
counselors must choose how to respond. If these counselor choices
result in differential amounts of client change talk, then they are
closely temporally related to the causal mechanism and consequently outcomes. Sequential analyses of this data are underway
and will be presented in a future report. Our findings suggest that
the specific directional skill of PRCT is an important indicator of
competence in MI practice. As such we believe it should be
considered one of the core MI skills and be given increased
emphasis in research, training and supervision of practitioners.
4.3. Percent Change Talk
Before discussing limitations of our findings it is important to
further address the issue of comparability between our project and
504
E. Barnett et al. / Journal of Substance Abuse Treatment 46 (2014) 498–505
other published studies addressing change talk as a mediator in MI. As
previously noted, this is first use of percent change talk to measure
this theoretically important mediator, and we chose it explicitly to
account for the variability in session length for our sample. In this way,
percent change talk is a measure with greater generalizability to
actual clinical settings where sessions are less and less bound to the
traditional 1 hour psychotherapy format. Various ways to account for
variation in the length of session have been used by previous
researchers. Baer et al. (2008) used frequency of change talk per
5 minute interval, Amrhein et al. (2003) broke sessions up into deciles
to perform comparisons, and studies with a more uniform session
length often use a straight measure of frequency. Since length of
session has a direct influence on the frequency of any behavior
occurring during session it is imperative to use some method to
standardize these numbers across sessions. We chose to divide the
number of instances of change talk by all other client utterances,
which is conceptually straightforward and has the advantage of ease
of calculation and replicability.
4.4. Limitations
Findings from this study should be considered in light of several
limitations to generalizability due to the unique intervention and
population of study participants. First, this intervention was unusual
in that it was delivered in two settings, at school and via telephone.
While the school sessions occurred easily, the telephone sessions
posed challenges to reaching students for follow-up, maintaining
consistent call lengths, and establishing rapport and keeping
participants' attention. Second, we note that variation in client levels
of change talk was influenced by including a pros and cons exercise in
the intervention. This choice explicitly increased the amount of
counter change talk present, thus reducing variance in our mediator.
Third, the intervention was also different in that participants had the
ability to set the agenda, which may have introduced some selfselection bias. We know that more students self-reported using drugs
on the paper and pencil measure than talked about drugs during the
MI sessions. It is conceivable that those who did speak to us were
more motivated to change their behavior, thus biasing them toward
change. However, this may have also been related to the counselors'
ability to engage subjects about drug use. Validity of our findings may
also be influenced by social-desirability bias, as marijuana use was
only measured through self-report, and not biochemically verified.
The decision to limit the analyses to marijuana outcomes does
negatively impact the generalizability of this study's results, however
it was most logical to investigate mediational hypotheses in the
context of the only substance on which MI had an impact in the parent
randomized controlled trial.
Our community sample of at-risk youth (70% male and 71% Latino)
is unique in the MI literature. We contend that this population may
have provided greater variability in drug use and problems associated
with drug use than often seen in clinical samples, resulting in a floor
effect, where there was less room for improvement. Any floor effect
may also have been exacerbated by the choice to include nonmarijuana users in the sample and the difficulty of tracking at-risk
youth. However, despite these floor effects, we were still able to find
robust relationships in our hypothesized model.
In addition there are limitations related to counselor skill that
should be explored. While our analyses did not control for nesting
within counselors, it is conceivable that counselor characteristics
beyond MI skill may be associated with client change talk and
outcomes. A practice effect resulting from varying number of sessions
per counselor might have skewed data. If good counselors had more
sessions this would also have restricted variance in the predictor. For
instance, as a result of rigorous training and supervision of interventionists we had so few instances of MIIN that we were unable to
include the percentage of MICO in our analysis. Agency staff in actual
intervention settings might be more likely to exhibit MIIN behaviors.
Finally, it is important to note that despite extensive supervision and
training, our coders reached only “fair” ICCs for complex and simple
reflections, similar to other published studies with these variables
(Gaume, Gmel, & Daeppen, 2008; Moyers et al., 2009). This modest
level of reliability allows less confidence in the findings for PCR and
may not accurately reflect counselor skill in this area.
Furthermore, there may be alternative explanations for our
findings or alternative untested mediational pathways that deserve
consideration. These analyses did not control for the number of
sessions that subjects received. Our decision to include only one
session per subject did not take into account the cumulative
motivational effect of multiple sessions or account for variation in
MI skill across sessions. In addition though we did not measure client
readiness to change, it also may account for variability in a client's
expression of change talk (Hallgren & Moyers, 2011; Moyers et al.,
2009). Finally, our mediation model may be inaccurately specified.
The indicator and mediator used in these analyses represent
correlational data, as they were collected at the same point in time.
While we know that path a and b preceded outcomes, we must
consider the possibility that client language may influence counselor
language as much as counselors influence client language. Even if this
is true, understanding the relative contribution of the MI fidelity
indicators is critically important for the training and development
of practitioners.
4.5. Conclusions
Despite limitations, this study contributes to the current search for
causal mechanisms in MI. It expands evidence for mediation to a study
of MI on adolescent marijuana use without personalized feedback;
whereas other mediation studies have all relied on hybrids of MI and
objective information-giving, this intervention relied on the “relational” and “technical” aspects only (Miller & Rose, 2009). Additionally, it is the only study to look at micro-skills in MI separately and
provide information about the relative merit of different clinical
choices as MI sessions progress. Future research should attempt to
replicate these findings in an effectiveness trial where greater
variance in counselor skill would enhance our understanding of the
relationship between indicators of MI quality, client change talk and
outcomes. In conclusion, our findings support change talk as an active
ingredient of MI and provide new empirical support for individual MI
skills and their contribution to outcomes. Findings also support a call
for increased training and measurement of the valence of counselor
skills both in response to and eliciting change talk. From this, data are
clear that the counselor's ability or tendency to reflect change talk
appears to be an important predictor of client success.
References
Amrhein, P. C., Miller, W. R., Yahne, C. E., Palmer, M., & Fulcher, L. (2003). Client
commitment language during motivational interviewing predicts drug use
outcomes. Journal of Consulting and Clinical Psychology, 71, 862–878.
Baer, J. S., Beadnell, B., Garrett, S. B., Hartzler, B., & Wells, E. A. (2008). Adolescent change
language within a brief motivational intervention and substance use outcomes.
Psychology of Addictive Behaviors, 22, 570–575, http://dx.doi.org/10.1037/
a0013022.
Barnett, E., Spruijt-Metz, D., Unger, J. B., Sun, P., Rohrbach, L. A., & Sussman, S. (2012).
Boosting a teen substance use prevention program with motivational interviewing.
Substance Use & Misuse, 47, 418–428, http://dx.doi.org/10.3109/10826084.
2011.641057.
Barnett, E., Sussman, S., Smith, C., Rohrbach, L. A., & Spruijt-Metz, D. (2012).
Motivational interviewing for adolescent substance use: A review of the literature.
Addictive Behaviors, 37, 1325–1334, http://dx.doi.org/10.1016/j.addbeh.2012.
07.001.
Bentler, P. M., & Chou, C. P. (1987). Practical issues in structural modeling. Sociological
Methods & Research, 16, 78–117.
Bishop, Y. M. M., Fienberg, S. E., & Holland, P. W. (1975). Discrete multivariate analysis:
Theory and practice. Cambridge, MA: MIT Press.
E. Barnett et al. / Journal of Substance Abuse Treatment 46 (2014) 498–505
Campbell, S. D., Adamson, S. J., & Carter, J. D. (2010). Client language during
motivational enhancement therapy and alcohol use outcome. Behavioural and
Cognitive Psychotherapy, 38, 399–415.
Catley, D., Harris, K. J., Mayo, M. S., Hall, S., Okuyemi, K. S., Boardman, T., et al. (2006).
Adherence to principles of motivational interviewing and client within-session
behavior. Behavioural and Cognitive Psychotherapy, 34, 43–56.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed
and standardized assessment instruments in psychology. Psychological Assessment,
6, 284.
Gaume, J., Bertholet, N., Faouzi, M., Gmel, G., & Daeppen, J. B. (2010). Counselor
motivational interviewing skills and young adult change talk articulation during
brief motivational interventions. Journal of Substance Abuse Treatment, 39, 272–281.
Gaume, J., Bertholet, N., Faouzi, M., Gmel, G., & Daeppen, J. B. (2013). Does change talk
during brief motivational interventions with young men predict change in alcohol
use? Journal of Substance Abuse Treatment, 44, 177–185.
Gaume, J., Gmel, G., & Daeppen, J. B. (2008). Brief alcohol interventions: Do counsellors’
and patients’ communication characteristics predict change? Alcohol and Alcoholism, 43, 62–69.
Gaume, J., Gmel, G., Faouzi, M., & Daeppen, J. -B. (2008). Counsellor behaviours and
patient language during brief motivational interventions: A sequential analysis of
speech. Addiction, 103, 1793–1800, http://dx.doi.org/10.1111/j.1360-0443.2008.
02337.x.
Gaume, J., Gmel, G., Faouzi, M., & Daeppen, J. B. (2009). Counselor skill influences
outcomes of brief motivational interventions. Journal of Substance Abuse Treatment,
37, 151–159.
Glynn, L. H., Hallgren, K. A., Houck, J. M., & Moyers, T. B. (2012). CACTI: Free, opensource software for the sequential coding of behavioral interactions. PLoS ONE, 7,
e39740, http://dx.doi.org/10.1371/journal.pone.0039740.
Glynn, L. H., & Moyers, T. B. (2010). Chasing change talk: The clinician’s role in
evoking client language about change. Journal of Substance Abuse Treatment, 39,
65–70, http://dx.doi.org/10.1016/j.jsat.2010.03.012.
Hallgren, K. A., & Moyers, T. B. (2011). Does readiness to change predict in‐session
motivational language? Correspondence between two conceptualizations of client
motivation. Addiction, 106, 1261–1269.
Houck, Moyers, Miller, Glynn, & Hallgren (2013). Motivational Interviewing Skill Code
2.5. Retrieved from from the Center on Alcoholism, Substance Abuse and
Addictions. http://casaa.unm.edu/download/misc25.pdf.
Hoyle, R. H., & Kenny, D. A. (1999). Sample size, reliability, and tests of statistical
mediation. Statistical strategies for small sample research, 195–222.
Jensen, C. D., Cushing, C. C., Aylward, B. S., Craig, J. T., Sorell, D. M., & Steele, R. G. (2011).
Effectiveness of motivational interviewing interventions for adolescent substance
use behavior change: A meta-analytic review. Journal of Consulting and Clinical
Psychology, 79, 433–440.
Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social psychology. The
Handbook of Social Psychology, 1, 233–265.
Lisha, N. E., Sun, P., Rohrbach, L. A., Spruijt-Metz, D., Unger, J. B., & Sussman, S. (2012).
An evaluation of immediate outcomes and fidelity of a drug abuse prevention
program in continuation high schools: Project Towards No Drug Abuse (TND).
Journal of Drug Education, 42, 33–57.
Macgowan, M. J., & Engle, B. (2010). Evidence for optimism: Behavior therapies and
motivational interviewing in adolescent substance abuse treatment. Child and
Adolescent Psychiatric Clinics of North America, 19, 527–545.
505
MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review
of Psychology, 58, 593–614, http://dx.doi.org/10.1146/annurev.psych.58.110405.
085542.
Martin, T., Christopher, P. J., Houck, J. M., & Moyers, T. B. (2011). The structure of client
language and drinking outcomes in project match. Psychology of Addictive
Behaviors, 25, 439–445, http://dx.doi.org/libproxy.usc.edu/10.1037/a0023129.
Martin, T., Moyers, T. B., Houck, J., Christopher, P., & Miller, W. R. (2005). Motivational
Interviewing Sequential Code for Observing Process Exchanges (MI-SCOPE) coder’s
manual. Unpublished, 16. Retrieved from http://casaa.unm.edu/download/
scope.pdf.
McCambridge, J., Day, M., Thomas, B. A., & Strang, J. (2011). Fidelity to motivational
interviewing and subsequent cannabis cessation among adolescents. Addictive
Behaviors, 36, 749–754, http://dx.doi.org/10.1016/j.addbeh.2011.03.002.
McCambridge, J., & Strang, J. (2004). The efficacy of single‐session motivational
interviewing in reducing drug consumption and perceptions of drug‐related risk
and harm among young people: Results from a multi‐site cluster randomized trial.
Addiction, 99, 39–52.
Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people for change.
New York: Guilford (Retrieved from http://www.guilford.com/pr/miller2.htm).
Miller, W. R., & Rose, G. S. (2009). Toward a theory of motivational interviewing.
American Psychologist, 64, 527–537.
Morgenstern, J., Kuerbis, A., Amrhein, P., Hail, L., Lynch, K., & McKay, J. R. (2012).
Motivational interviewing: A pilot test of active ingredients and mechanisms of
change. Psychology of Addictive Behaviors, http://dx.doi.org/10.1037/a0029674.
Moyers, T. B., & Martin, T. (2006). Therapist influence on client language during
motivational interviewing sessions. Journal of Substance Abuse Treatment, 30,
245–251.
Moyers, T. B., Martin, T., Christopher, P. J., Houck, J. M., Tonigan, J. S., & Amrhein, P. C.
(2007). Client language as a mediator of motivational interviewing efficacy: Where
is the evidence? Alcoholism, Clinical and Experimental Research, 31, 40s–47s.
Moyers, T. B., Martin, T., Houck, J. M., Christopher, P. J., & Tonigan, J. S. (2009).
From in-session behaviors to drinking outcomes: A causal chain for motivational interviewing. Journal of Consulting and Clinical Psychology, 77, 1113–1124,
http://dx.doi.org/10.1037/a0017189.
Muthén, L. K., & Muthén, B. O. (1998-2011). Mplus User's Guide. Sixth Edition. Los
Angeles, CA: Muthén & Muthén.
Pirlott, A., Kisbu-Sakarya, Y., DeFrancesco, C., Elliot, D., & MacKinnon, D. P. (2012).
Mechanisms of motivational interviewing in health promotion: A Bayesian
mediation analysis. International Journal of Behavioral Nutrition and Physical
Activity, 9, 69, http://dx.doi.org/10.1186/1479-5868-9-69.
Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental
studies: New procedures and recommendations. Psychological Methods, 7, 422–455.
Sussman, S., Sun, P., Rohrbach, L. A., & Spruijt-Metz, D. (2012). One-year outcomes
of a drug abuse prevention program for older teens and emerging adults:
Evaluating a motivational interviewing booster component. Health Psychology,
31, 476–485.
Vader, A. M., Walters, S. T., Prabhu, G. C., Houck, J. M., & Field, C. A. (2010). The language
of motivational interviewing and feedback: Counselor language, client language,
and client drinking outcomes. Psychology of Addictive Behaviors, 24, 190–197,
http://dx.doi.org/10.1037/a0018749.
Zhao, X., Lynch, J. G., & Chen, Q. (2010). Reconsidering Baron and Kenny: Myths and
truths about mediation analysis. Journal of Consumer Research, 37, 197–206.
2/11/2019
1. Summary of a research study: (two pages double spaced plus article)
The purpose of this assignment is to introduce you to the wide range of research that supports MI
as an evidence-based practice.
Locate a journal article that is a research study of MI in an area of your interest. It must be about
an actual study where there was a research question, sample, intervention, etc. Make a hard copy
of the article and highlight it as you read it. You may choose an article from the assigned ones in
the course syllabus, the Bibliography list, or one on a topic that you are interested in, as pertains
to MI.
You are to answer the following questions about the study (do not use as headings):
A. Write the correct APA reference for your paper at the top.
B. What was the purpose of the study?
C. Why was MI used as an intervention?
D. Was MI modified at all for the intervention? If so, why and how?
E. What was a hypothesis or research question?
F. Who was the intervention tested on? (Sample, sample size)
G. What measures were used?
H. What was the design of the study? Pre-test/Post-test? Post-test only?
I. Was fidelity to MI measured? If yes, then how was it measured?
J. What were the outcomes of the study? What did the authors find out? Did the use of MI
make a difference?
K. What are your reactions to the study? How might you use what you learned? What are the
implications for social workers?
All papers must be double spaced, Times New Roman 12 point font, 1 inch margins, using APA
format. Your paper should be about two pages long. Please attach your article that has been read
and highlighted to your paper. Your paper will be graded based on how well you answered each
area, spelling and grammar, and use of APA formatting. Your TurnItIn score should be under
10%. Be sure to submit your paper to TurnItIn by the due date and time. You may use TurnItIn
multiple times ahead of time, to get your similarity score and revise your paper accordingly. You
must leave 24 hours between submissions.
Please be sure USE THE SDSU LIBRARY to access your article. Do not use Google. You
should NEVER have to pay for an article. If the library doesn’t have it, they will get it from
Interlibrary loan for FREE and email it to you. If you aren’t familiar with this system, ask
the librarian. You are already paying for this service in your fees.
This is due on 2/14/19 at the beginning of class. You are to submit the paper to both TurnItIn and
a hard copy with your highlighted article to me.
2/11/2019
SW 381 Research Article Summary Paper Grading Rubric
Criteria
Correct article citation at beginning, use
of APA
Purpose of Study/Use of MI
summary/Hypothesis
Methods Summary (sample, measures,
data collection, fidelity)
Results Summary
Implications for social work practice
Spelling, grammar, low similarity score
Other: APA
Score:
Not Observed
Fair
Good
Excellent*
Purchase answer to see full
attachment