Prev Sci (2017) 18:174–182
DOI 10.1007/s11121-016-0717-5
Identification of At-Risk Youth by Suicide Screening in a Pediatric
Emergency Department
Elizabeth D. Ballard 1 & Mary Cwik 2,3 & Kathryn Van Eck 4,5 & Mitchell Goldstein 5 &
Clarissa Alfes 2 & Mary Ellen Wilson 6 & Jane M. Virden 6 & Lisa M. Horowitz 7 &
Holly C. Wilcox 2,4
Published online: 27 September 2016
# Society for Prevention Research (outside the USA) 2016
Abstract The pediatric emergency department (ED) is a critical location for the identification of children and adolescents
at risk for suicide. Screening instruments that can be easily
incorporated into clinical practice in EDs to identify and intervene with patients at increased suicide risk is a promising
suicide prevention strategy and patient safety objective. This
study is a retrospective review of the implementation of a brief
suicide screen for pediatric psychiatric ED patients as standard
of care. The Ask Suicide Screening Questions (ASQ) was
implemented in an urban pediatric ED for patients with psychiatric presenting complaints. Nursing compliance rates,
identification of at-risk patients, and sensitivity for repeated
ED visits were evaluated using medical records from 970
* Elizabeth D. Ballard
Elizabeth.Ballard@nih.gov
1
Experimental Therapeutics and Pathophysiology Branch, Intramural
Research Program, National Institute of Mental Health, National
Institutes of Health, Building 10, CRC Room 7-3345, MSC 1282,
Bethesda, MD 20892, USA
2
Department of Psychiatry and Behavioral Sciences, Johns Hopkins
University School of Medicine, Baltimore, MD, USA
3
Center for American Indian Health, Johns Hopkins Bloomberg
School of Public Health, Baltimore, MD, USA
4
Department of Mental Health, Johns Hopkins Bloomberg School of
Public Health, Baltimore, MD, USA
5
Department of Pediatrics, Johns Hopkins University School of
Medicine, Baltimore, MD, USA
6
Pediatric Emergency Department, The Johns Hopkins Hospital,
Baltimore, MD, USA
7
Office of the Clinical Director, Intramural Research Program,
National Institute of Mental Health, National Institutes of Health,
Bethesda, MD, USA
patients. The ASQ was implemented with a compliance rate
of 79 %. Fifty-three percent of the patients who screened positive (237/448) did not present to the ED with suicide-related
complaints. These identified patients were more likely to be
male, African American, and have externalizing behavior diagnoses. The ASQ demonstrated a sensitivity of 93 % and
specificity of 43 % to predict return ED visits with suiciderelated presenting complaints within 6 months of the index
visit. Brief suicide screening instruments can be incorporated
into standard of care in pediatric ED settings. Such screens can
identify patients who do not directly report suicide-related
presenting complaints at triage and who may be at particular
risk for future suicidal behavior. Results have the potential to
inform suicide prevention strategies in pediatric EDs.
Keywords Suicide . Emergency department . Screening .
Pediatrics
Introduction
Suicide is now the 2nd leading cause of death in both 10–14year olds and 15–19-year olds, with rates remaining relatively
stable since 1990 (Centers for Disease Control and Prevention
(CDC) (2016b)). According to the most recent data from the
Youth Risk Behavior Survey, 8.6 % of American high school
students report that they have attempted suicide in the last year
and 3 % made a suicide attempt that required medical treatment (CDC 2016a). Across all ages, adolescents aged 15 to
19 years visit the ED for self-harm behaviors more frequently
than any other age group (Ting et al. 2012). Given the relationship between past history of suicidal thoughts and behaviors with death by suicide, it is possible that intervening with
adolescents who think about and attempt suicide may result in
a reduced suicide rate, reduced healthcare burden, and,
Prev Sci (2017) 18:174–182
because of their developmental stage, a significant number of
life-years saved.
The pediatric ED is an important setting for identifying children and adolescents at risk for suicide in order to intervene
ideally before youth attempt or die by suicide (Horowitz et al.
2009). Not only do many children and adolescents present to
the ED for treatment of their suicidal thoughts or after a suicide
attempt, ED patients without suicide-related complaints often
report suicide risk factors, such as depression, anxiety, aggression, and drug or alcohol abuse (Mahajan et al. 2009; Sheridan
et al. 2015). Furthermore, subgroups of children and adolescents use the ED as their primary source of medical care and
may not utilize primary care or mental health services; therefore, the ED clinician may be the sole healthcare provider able
to detect suicidal thoughts (Wilson and Klein 2000). Even
when these patients report thoughts of self-harm, they may
not receive needed assessments and resources; reviews of
Medicaid records suggest that only 39 % of youth who present
to the ED for self-harm receive a mental health evaluation before discharge and only 43 % are linked to mental health outpatient resources (Bridge et al. 2012). Standard of care in the
ED may also be heterogeneous, with different levels of access
to mental health providers across EDs (Grupp-Phelan et al.
2009; Cappelli et al. 2012). Thus, the pediatric ED population
forms a particularly underserved population of youth who experience high risk for suicidal behavior.
Screening in the ED is therefore a unique opportunity for
suicide prevention. Screening creates a context with at-risk
youth and their families to discuss suicide risk, identify risk
levels, and engage them in prevention efforts. Through screening, ED clinicians can initiate early prevention strategies, such
as linkages to treatment, in order to prevent later suicide attempts and potentially save lives. Many ED settings use informal assessments practices; standardized assessment allows clinicians to link positive responses to specific clinical interventions and permits researchers to track the prevalence of acute
suicide risk in their ED population. All of these reasons have
led the National Action Alliance for Suicide Prevention
(NAASP), the group charged with implementing the National
Suicide Prevention Strategy, as well as the Joint Commission,
to recommend screening for suicide risk in EDs as critical patient safety and suicide prevention initiatives (National Action
Alliance for Suicide Prevention Research Prioritization Task
Force 2014; The Joint Commission 2010; The Joint
Commission 2016). In fact, the NAASP has identified suicide
screening efforts that can be easily incorporated into healthcare
settings as an Burgent priority^ for prevention efforts.
Suicide risk screening in pediatric EDs requires validated
instruments and effective screening procedures which fit the
needs of the healthcare setting (Boudreaux and Horowitz
2014). Few, if any, analyses have addressed the implementation of suicide screening instruments in real-world settings.
This study is a systematic examination of suicide risk
175
screening as routine care for patients presenting to the ED
for psychiatric reasons using a brief screen (Ask Suicide
Screening Questions or ASQ) in an urban ED (Horowitz
et al. 2012). ED screening for suicidal behavior may be particularly relevant for urban environments, in which many children and adolescents are exposed to violence, trauma, and
stress, which may lead youth to seek emergency treatment
(Breslau 2009). In particular, screening interventions that
can identify African American youth are sorely needed.
Most research on suicidal behavior has focused on
European-American youth, yet the suicide rates in African
American children have recently increased (Bridge et al.
2015). These youth are particularly hard to reach with services, are less likely to seek mental health treatment when
needed, and are more likely to isolate from supportive social
groups when distressed (Goldston et al. 2008;
Langhinrichsen-Rohling et al. 2009; Molock et al. 2006).
There is the potential that through such screening, youth with
suicidal thoughts can be identified before they go on to make
attempts.
This study had several goals: (1) to examine nursing compliance with administration and the degree to which patient
characteristics impact nursing compliance; (2) to describe the
relationship between screening results and primary complaint,
demographics, and disposition; (3) to identify the added value
of the ASQ in identifying children and adolescents for whom
suicide risk may have otherwise gone undetected; and (4) to
evaluate the relationship between ASQ screening results and
repeat visits to the ED for suicide-related reasons.
Methods
This was a retrospective cohort study of a consecutive case
series of patients in the Johns Hopkins Hospital Pediatric ED
from March 2013 through August 2014 (76 weeks/
18 months), where the ASQ was implemented as a selective
prevention strategy. ED nursing staff used the ASQ as a standard of care during intake procedures for patients entering the
ED for psychiatric reasons. The Johns Hopkins Pediatric ED
is part of an urban academic pediatric medical center with
approximately 30,000 patient visits per year. Eligible patients
for this analysis were ages 8 through 18 years and presented
with a psychiatric presenting complaint. No patients were excluded on the basis of sex, minority status, or insurance type.
This medical record review was approved by the Johns
Hopkins School of Medicine Institutional Review Board.
Screening Assessment
The Ask Suicide Screening Questions (ASQ) is a four-item
non-proprietary suicide screening instrument that can be administered to patients in the ED for psychiatric or non-
176
psychiatric reasons, aged 10 to 21 years, by nurses regardless
of psychiatric training (Horowitz et al. 2012). All questions
are asked to the patient, and a Byes^ response to any of the four
items is considered a positive screen. The four items are the
following: BIn the past few weeks, have you wished you were
dead?^, BIn the past few weeks, have you felt that you or your
family would be better off if you were dead?^, BIn the past
week, have you been having thoughts about killing
yourself?^, and BHave you ever tried to kill yourself?^ The
ASQ was developed from a study of 524 patients across three
pediatric EDs using the Suicide Ideation Questionnaire (SIQ)
as the criterion standard (Reynolds 1987). In the initial development study, for psychiatric patients, the ASQ was found to
have a sensitivity of 97.6 %, a specificity of 65.6 %, and a
negative predictive value of 96.9 % compared with the SIQ.
Evaluations are underway to determine if other scoring
methods, including adding the number of Byes^ responses,
would be a beneficial screen, but no empirical data has been
published as of yet.
Implementation Procedures
This screening effort involved an interdisciplinary team with
trans-disciplinary collaboration, including emergency medicine physicians, emergency department nurses, epidemiologists, and psychologists involved in the creation of the screening instrument. After discussion among the team, the ASQ
was presented to a hospital-wide multidisciplinary group
charged with monitoring changes to the electronic medical
record (EMR). Once the group agreed to adopt the ASQ as
standard of care, the screen was built into the EMR. Several
factors facilitated ASQ implementation and sustainability in
this setting including the Joint Commission’s National Patient
Safety Goal requiring behavioral healthcare organizations,
psychiatric hospitals, and general hospitals treating individuals for emotional or behavioral disorders to identify individuals at risk for suicide. The characteristics of the screen also
contributed to the decision to implement as the ASQ is very
brief, easily scored, and in the public domain. The ASQ was
added to the EMR in the pediatric ED for patients presenting
with psychiatric concerns in March of 2013.
Before implementation, the nurses were trained via a series
of brief in-services on the floor of the ED; charge nurses were
given additional training in order to facilitate monitoring of
the screening efforts. The nursing department for this ED is
around 60 individuals; over 35 are specifically trained on triage assessment. The trainer was a clinical psychology postdoctoral fellow who was part of the team that developed and
conducted validation studies of the ASQ instrument. After a
few months of implementation, an additional presentation was
made to the charge nurses in June 2013 to share initial compliance rates and problem-solve any administration concerns.
While the trans-disciplinary team was involved throughout to
Prev Sci (2017) 18:174–182
trouble-shoot any concerns, the continued use of the ASQ was
primarily due to the nursing and physician champions, who
ensured that screening continued to be done throughout the
year.
The ASQ was administered at triage with the triage nurse,
in front of the parent/guardian who brought the patient into the
ED. Information on positive screens on the ASQ was relayed
to ED physicians, nurses, and social workers. Of note, the
ASQ was developed on patients aged 10 to 21 years. Given
feedback from nursing staff that suicidal patients often present
to the ED who are younger than 10 years of age, it was agreed
that the ASQ would be administered to patients ages 8 to
18 years. Age 8 years had been the lower age limit used for
an instrument that was the precursor to the ASQ (Horowitz
et al. 2001).
Study Procedures
The retrospective chart review was conducted by querying the
ED’s electronic health record database to identify patients who
presented to the ED with psychiatric complaints over the
76 weeks of review. The patient’s arrival date, gender, age,
race, insurance status, presenting complaint, ASQ responses,
disposition, and discharge diagnoses were extracted from the
medical record. Only the index visit was used during the study
period; repeat visits were excluded from initial analyses. As
there were 16 discrete presenting complaint and 16 discharge
diagnosis categories, these classifications were collapsed into
clinical categories through the consensus of two licensed clinical psychologists and one psychology postdoctoral fellow.
Presenting complaints were mutually exclusive (a patient
had only one presenting complaint), but patients could receive
more than one discharge diagnosis. Patients under age 8 and
over 18 years were excluded from the analysis, in addition to
patients with a recorded diagnosis of mental retardation, developmental delay, and autism/Asperger’s. Although these individuals may have been at risk for suicide, no validated suicide screening instruments currently exist for this population
(Ludi et al. 2012; Segers and Rawana 2014).
In an additional analysis, the relationship between ASQ
response on index visit and repeat visits in the subsequent
6 months after index ED visit was evaluated. In order to be
included in this subanalysis, patients had to have at least
6 months of follow-up data available in the study period;
therefore, only patients with index visits before February
2014 were included.
Statistical Analysis
Univariate analyses were used to describe (1) nursing compliance rates, (2) characteristics of patients with dichotomized
screening results (positive or negative) on the ASQ, and (3)
characteristics of patients who screened positive on the ASQ
Prev Sci (2017) 18:174–182
by presenting complaint to identify youth whose suicide risk
may have otherwise been undetected during the visit.
Sensitivity and specificity analyses were used to evaluate the
relationship between ASQ response on index visit and repeated visits for psychiatric and suicide-related reasons in the
6 months after index visit. SPSS version 21.0 was used for
all analyses, and statistical significance was considered at
p < .05.
Results
During the study period, there were 1484 consecutive patient
visits for psychiatric reasons in the 8–18-year age range. Of
these patient visits, 400 were repeated visits (27 %), resulting
in 1084 unique patients. One hundred fourteen patients were
excluded from analysis due to diagnoses of mental retardation,
developmental delay, and autism/Asperger’s.
In total, 970 patients were included in the analysis. The
average age was 13.4 years (SD = 2.6); the sample was 53 %
female and 66 % African American. Of this sample, 288
(30 %) were hospitalized or transferred to another facility after
their visit.
Nursing Compliance with Screening
Over three fourths of patients were screened (768/970),
resulting in a 79 % nursing compliance rate. Characteristics
of patients who were and were not screened during their visit
are presented in Table 1. There was no significant age difference between patients who were and were not screened.
Nurses were more likely to screen females, patients with
suicide-related chief complaints, and patients who presented
with emergency petitions for a mental health evaluation.
Nurses were less likely to screen patients with internalizing
symptoms, alcohol or drug overdoses, and bizarre behavior or
hallucinations.
Characteristics of Patients Who Screened Positive
on the ASQ
Of the psychiatric patients screened, over half screened positive for suicide risk (448/768, 58 %). Demographic and clinical characteristics of patients who screened positive as compared to negative on the ASQ are presented in Table 2.
Patients who screened positive were more likely to be older,
female, have a suicide-related presenting complaint, and be
hospitalized or transferred on their visit compared to those
who screened negative. Patients who screened positive were
less likely to present to the ED with an externalizing behavior
or emergency petition/mental health evaluation presenting
complaint.
177
Characteristics of Patients Who Screened Positive
on the ASQ by Presenting Complaint
Of those 448 psychiatric patients who screened positive on the
ASQ, 237 (53 %) presented without suicide-related chief complaints. In order to focus on patients who may have been
identified by the screening procedures, Table 3 displays patients who screened positive on the ASQ by presenting complaint. Patients who did not have a presenting complaint related to suicide, but screened positive on the ASQ (and may have
gone undetected had they not been screened), were older,
more likely to be male, African American, have bipolar disorder, ADHD, CD, ODD, or aggression noted in the medical
record. These patients were less likely to endorse the first three
items of the ASQ, which are primarily related to suicidal
thoughts, and were more likely to endorse the final item,
which relates to past suicidal behavior. In addition, they were
less likely to be hospitalized or transferred postdischarge.
Subsample Analysis of Sensitivity/Specificity
Of the subsample of patients with 6 months post-ASQ screening follow-up data available (n = 618), 131 made at least one
repeated visit to the ED for psychiatric reasons (21 %). Thirtytwo of these patients (32/618 or 5 % of sample with follow-up
data) made a repeat visit to the ED with suicide risk as the
presenting complaint. Of those 32 patients with repeat visits
for suicide-related reasons, 28 had screened positive on the
ASQ on index visit, 2 had screened negative on the ASQ,
and 2 were not screened. Due to the two individuals who were
not screened with the ASQ on index visit, the overall sensitivity of the entire patient population was 88 %, 95 % CI 71–
96 %. As a further subanalysis, we focused on patients that did
not have a presenting complaint of suicide risk on index visit
(n = 329). Of these patients, seven made a repeat visit to the
ED with suicide as the presenting complaint (7/329 or 2 %),
five of which had screened positive on the ASQ on index visit,
one had screened negative, and one was not screened.
Sensitivity and specificity values are presented in Table 4 for
all patients with follow-up data. Sensitivity values for suiciderelated outcomes ranged from 83 to 93 %, while specificity
values were lower from 43 to 58 %. Including the one patient
who was not screened, sensitivity values fell to 71 %, 95 % CI
31–95 %.
As males were less likely to be screened than females, a
subanalysis of sensitivity/specificity for repeated ED visit for
suicide-related complaints was conducted. Screening positive
on the ASQ was associated with 100 % sensitivity in
predicting repeat ED visits for suicide-related complaints
and 58 %, 95 % CI 55–99 %, specificity in males. For females,
there was 90 %, 95 % CI 69–98 %, sensitivity and 32 %, 95 %
CI 30–33 %, specificity for the same outcome.
178
Prev Sci (2017) 18:174–182
Table 1 Pediatric ED patients
with psychiatric complaints:
comparing those who were
screened with the ASQ to those
who were not screened
Not screened with
ASQ
Screened with
ASQ
(n = 202)
(n = 768)
Mean
13.4
SD
2.9
Mean
13.4
SD
2.5
t
−0.3
n
%
n
%
ORa
p
Males
African American race
112
137
55
68
341
507
44
66
0.64
0.92
.005
.63
Medicaid insurance
Presenting complaint
138
68
536
70
1.07
.69
Suicide ideation or attempt
Externalizing: aggression, ADHD, violence
24
44
12
22
230
147
30
19
3.17
0.85
Purchase answer to see full
attachment