Hindawi Publishing Corporation
e Scientific World Journal
Volume 2015, Article ID 727694, 9 pages
http://dx.doi.org/10.1155/2015/727694
Review Article
Information Technology in Critical Care: Review of Monitoring
and Data Acquisition Systems for Patient Care and Research
Michael A. De Georgia,1 Farhad Kaffashi,2 Frank J. Jacono,1,3 and Kenneth A. Loparo2
1
University Hospitals Case Medical Center, 11100 Euclid Avenue, Cleveland, OH 44106-5040, USA
Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH 44106-7078, USA
3
Louis Stokes Cleveland VA Medical Center, 10701 East Boulevard, Cleveland, OH 44106, USA
2
Correspondence should be addressed to Michael A. De Georgia; michael.degeorgia@uhhospitals.org
Received 17 October 2014; Accepted 2 January 2015
Academic Editor: Anastasia Kotanidou
Copyright © 2015 Michael A. De Georgia et al. This is an open access article distributed under the Creative Commons Attribution
License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly
cited.
There is a broad consensus that 21st century health care will require intensive use of information technology to acquire and analyze
data and then manage and disseminate information extracted from the data. No area is more data intensive than the intensive care
unit. While there have been major improvements in intensive care monitoring, the medical industry, for the most part, has not
incorporated many of the advances in computer science, biomedical engineering, signal processing, and mathematics that many
other industries have embraced. Acquiring, synchronizing, integrating, and analyzing patient data remain frustratingly difficult
because of incompatibilities among monitoring equipment, proprietary limitations from industry, and the absence of standard data
formatting. In this paper, we will review the history of computers in the intensive care unit along with commonly used monitoring
and data acquisition systems, both those commercially available and those being developed for research purposes.
1. Introduction
There is a broad consensus that health care in the 21st
century will require the intensive use of information technology and clinical informatics to acquire and manage data,
transform the data to actionable information, and then
disseminate this information so that it can be effectively used
to improve patient care. Nowhere is this more evident and
more important to patient outcomes than in the intensive care
unit (ICU). Critical care involves highly complex decision
making. It is by nature data-intense. Despite the growth of
critical care, however, the basic approach of data collection
and management has remained largely unchanged over the
past 40 years. Large volumes of data are collected from
disparate sources and reviewed usually retrospectively; and
even that is difficult. Providers must navigate through a
jungle of monitors, screens, software applications, and often
paper charts that provide supplemental patient data inherent
in today’s cacophony of information management systems.
Data from patient monitors and medical devices, although
available visually at the bedside, is challenging to acquire
and store in digital format. There is limited medical device
interoperability and integration with the electronic medical
record (EMR) remains incomplete at best and cumbersome.
In addition (and partly as a result of these limitations),
standard analytical approaches provide little insight into a
patient’s actual pathophysiologic state. Understanding the
dynamics of critical illness requires precisely time-stamped
physiologic data (sampled frequently enough to accurately
recreate the detail of physiologic waveforms) integrated with
clinical context and processed with a wide array of linear
and nonlinear analytical tools. This is well beyond the
capability of typical commercial monitoring systems. Such
an understanding derived from advanced data analytics can
aid physicians in making timely and informed decisions
and improving patient outcomes. Ultimately, an integrated
critical care informatics architecture will be required that
includes acquisition, synchronization, integration, and storage of all relevant patient data into a single, searchable
database (numeric and waveforms) and data processing to
2
extract clinically relevant features from raw data and translate
them into actionable information [1]. Advances in technology
are beginning to bring all of this together.
In this paper, we will review the history of computers
in the ICU along with commonly used monitoring and data
acquisition systems, both those commercially available and
those being developed for research purposes.
2. Computers in the ICU
Clinical information management systems are now common
in most hospitals. These systems have evolved along several
parallel lines beginning, not surprisingly, in 1946 with the
introduction of the Electronic Numerical Integrator and
Computer (ENIAC), the first general-purpose computer (see
Table 1). The size of a room and weighing in at 27 tons,
ENIAC was developed to calculate missile trajectories for
the U.S. Army [2]. Five years later, IBM introduced the first
commercially available computer, the Engineering Research
Associates (ERA) 1103. Because of their exorbitant cost, use
of early computers was limited to large corporations to
help manage accounting. In the 1960s, academic institutions
followed suit and began developing computer systems to
streamline their growing business operations. A decade later,
hospitals began to develop EMR systems including the Problem Oriented Medical Record (POMR) at the University of
Vermont [3], Health Evaluation through Logical Processing
(HELP) at the University of Utah [4], and The Medical Record
(TMR) at Duke University [5] and the Computer Stored
Ambulatory Record (COSTAR) at Harvard [6]. COSTAR was
programmed in MUPMS (Massachusetts General Hospital
Utility Multi-Programming System), a computer language
better formatted for medical data than COBOL and FORTAN, which were routinely used at the time (MUMPS was
eventually adopted for use by the Department of Veterans
Affairs). While Indiana’s Regenstrief Medical Record System
(RMRS) was one of the first systems for both in-patient
and outpatient settings [7], these early EMRs were rarely
connected to the real-time data-intense environment of the
ICU. This was a world unto itself.
Shubin and Weil are credited with introducing the computer to the ICU in 1966 for the purpose of automatically
collecting vital signs from the bedside monitor [8]. By connecting an IBM 1710 computer through an analog-to-digital
converter to bedside devices, they were able to collect arterial
and venous pressure, heart rate, temperature, and urinary
output. This had actually been done before in the operating
room, though not easily. Using a mechanical contraption,
McKesson recorded tidal volumes, fraction of inspired oxygen, and blood pressure in 1934 [9]. The development of
the computer (and particularly the microprocessor) made
this onerous task much easier. Basic analytical tools, such
as trend analysis, were soon added to the automated data
collection systems [10]. Other early applications of computers
in medicine included one of the first clinical decision support
systems to aid in the diagnosis of hematologic disorders by
Lipkin and colleagues [11], systems for respiratory monitoring
by Stacy and Peters [12], and automation of blood transfusion
after cardiac surgery by Sheppard and colleagues [13]. The
The Scientific World Journal
computer-based Clinical Assessment, Research, and Education System (CARE) was a clinical decision support system
developed to aid in the treatment of critically ill surgical
patients. The system continuously monitored physiologic and
metabolic functions of critically ill patients and managed data
about fluid and electrolytes as well as cardiac and respiratory
functions [14].
This experience with computers in academic institutions
inspired Hewlett-Packard to offer a commercial version of
these systems. Adoption of their Patient Data Management
System (PDMS), however, was slow because the primitive
user interfaces and complex menus were not suited to the fast
pace of the ICU [15, 16]. In the 1980s, automatic collection of
heart rate and blood pressure became more advanced with
data being presented in graphical displays that mimicked the
familiar bedside flow sheet [17]. The architecture also evolved
from the locally contained model to the client/server model
in which a workstation in the ICU (the client) interacted
with a central computer housing patient data (the server)
via a local area network (LAN). Navigational tools became
more user friendly though analytical capabilities remained
limited [18]. Links to the fledgling hospital EMR systems were
also being made beginning with the computer system that
handled admissions, discharges, and transfers (ADT) so that
patient demographic data could be readily accessed. Physician and nursing notes were soon being entered electronically
into a problem-oriented medical record [19]. In parallel to
the ICU, computers were also being introduced in the 1980s
into the operating room. Picking up where McKesson left
off, in 1986 Gravenstein introduced computerized anesthesia
records [20], which allowed for more reliable collection,
storage, and presentation of data during the perioperative
period as well as provide basic record keeping functions (thus
in their infancy such systems were called “anesthesia record
keepers”). Still, as in the ICU, data from medical devices were
rarely integrated with the other physiological data.
In the 1990s, ICU systems improved significantly with
increased clinical functionality and Internet access. Webbased software used Web browsers to display the user
interface and simple queries of cumulative patient data
were supported. Vendors migrated the technology that had
been developed for the OR and the ability to record and
present continuous patient data as well as provide links to
physician notes, nursing documentation, and laboratory and
imaging data from the evolving EMR systems, thus creating
large enterprise systems, now broadly referred to as Clinical
Information Systems [21].
3. Clinical Information Systems
Several Clinical Information Systems are commercially available today for the ICU and competition among vendors
is intensifying. Frost & Sullivan have estimated that the
annual US market for emergency, perioperative, and intensive
care software solutions is currently approximately $842.2
million and are expected to reach $1.3 billion in 2015 [22].
Not one company has a dominant share of the market and
several have evolved over the last decade, through various
acquisitions of smaller participants, to offer broad end-to-end
The Scientific World Journal
3
Table 1: Timeline of computers in the ICU.
Computer
1946
1951
ENIAC introduced
IBM ERA 1103 introduced
1966
HP 2115 introduced
Electronic medical record
Computer systems for the ICU
1961
Clinical decision support for diagnosis of hematologic disorders
(Lipkin)
1964
Marquette Electronics founded
1965
Computerized respiratory monitoring (Stacy and Peters)
1966
Computerized collection of vital signs (Shubin)
1969 IDX System founded for revenue cycle management (utilizing
MUMPS)
1973
Computer assisted monitoring with trend analysis (Lauwers)
1975
HP introduces Patient Data Management System (PDMS)
1976
Clinical decision support for management of critically ill
surgical patients (Siegel)
1977
The Medical Record (TMR), Duke University
Regenstrief Medical Record System (RMRS), Indiana University
1978
Problem Oriented Medical Record (POMR), University of
Vermont
Computer Stored Ambulatory Record (COSTAR), Harvard
University (MUMPS)
1979 Epic Systems founded (MUMPS)
1980
1983
Computerized management system for ICU (Manzano)
Health Evaluation through Logical Processing (HELP),
University of Utah
1997 Veterans Health Information Systems and Technology
Architecture (VistA) founded
Computerized system for automating blood transfusion
(Sheppard)
HP introduces Carevue system
Capsule Technologie introduces DataCaptor
1999
Excel Medical Electronics introduces Bedmaster XA
1986
2000 MedicaLogic founded
Clinical Information Systems
2003
GE’s Centricity Critical Care system introduced
2007
Philips IntelliVue Clinical Information Portfolio (ICIP) Critical Care introduced
platforms [23]. For example, GE’s Centricity Critical Care
system from GE HealthCare (Chalfont St. Giles, UK), introduced in 2003, is the culmination of the acquisitions of,
among others, Marquette Medical Systems, a leading manufacturer of patient monitors, Instrumentarium, a manufacturer of mechanical ventilators and anesthesia equipment,
and iPath, the basis of the Operating Room Management
Information System. For the EMR side, GE also acquired
MedicaLogic, a leading provider of outpatient digital health
records, and IDX Systems, primarily a practice management
and billing system. IDX was written using MUMPS, which
currently also forms the basis for EpicCare (Epic Systems
Corporation, Verona, WI) and Veterans Health Information
Systems and Technology Architecture (VistA). GE’s Centricity Critical Care automatically collects data from monitors
and ventilators displays it in spreadsheets reminiscent of the
typical paper ICU chart. Data are collected from medical
devices through device interfaces that connect with GE’s
Unity Interface Device (ID) network.
Philips Healthcare (Andover, MA) also has a long history in the ICU with the introduction of the Patient Data
Management System in the early 1970s under the HewlettPackard brand. In the 1990s this became CareVue [24] and
the most recent iteration is the IntelliVue Clinical Information Portfolio (ICIP) Critical Care introduced in 2007.
Like GE’s Centricity, Philips’ ICIP Critical Care also evolved
4
through a series of acquisitions [25]. These included Agilent
Technologies Healthcare Solutions Group, a leader in patient
monitoring and critical care information management, Witt
Biomedical Corporation, a leader in hemodynamic monitoring in catheterization laboratories, and Emergin, a developer
of alarm management software. In 2008, Philips acquired
TOMCAT Systems Ltd., a company that offers software for
the collection of cardiac data and that same year, acquired
VISICU Inc., a provider of tele-ICU technology. On the
EMR side, Philips also partnered in 2004 with Epic in order
to provide end-to-end integration with electronic medical
records. As with GE’s Centricity, Philips’ ICIP Critical Care
supports automatic (or manual) documentation of physiologic data with time resolutions up to every 5 minutes. Philips
Information Support Mart interfaces with ICIP and provides
a relational database that archives clinical information such as
lab results, text notes, medications, and patient demographics
that can be queried with special scripts (see MIMIC II below).
4. Limitations of Commercially Available
Clinical Information Systems
While modern Clinical Information Systems do provide endto-end platforms for the ICU, there are several limitations that
remain. First, they remain limited in terms of functionality
and the acquisition of high-resolution physiologic data. For
example, the actual physiological waveform signals are not
acquired or stored by either the GE’s Centricity system or
the Philips ICIP system. This is an important limitation of
most commercially available enterprise systems today and is
the result of a tradeoff between the memory requirements
of capturing high-resolution physiological data (including the
underlying waveform morphology) versus capturing only
data snapshots that may be sufficient for certain clinical
decision-making. Currently, no standards have been defined
as to where that balance lies. Philips has developed its own
proprietary solution for automated acquisition of waveform
data for research called the Research Data Exporter (RDE).
This solution does not acquire the data at the native sampling
rate of the signal and limits the number of waveforms that
can be exported. Better real time acquisition of physiological
waveform signals is needed along with education of clinicians
regarding its value in understanding of complex physiology.
Second, there is currently neither processing nor analysis
of data. While a few monitors can display raw trends,
even basic analyses (mean, median, standard deviations) are
difficult to perform at all let alone in real time and higherlevel analyses are impossible. New physiological models are
now emerging suggesting that nonlinear changes in dynamics
over time may have more predictive value. Understanding
this complex physiology can lead to more timely intervention
and better outcomes. Techniques for the analysis of nonlinear
systems have emerged from the mathematical and engineering sciences but have not been applied to physiological data
in the ICU (in part because the acquisition and integration
challenges have not been met). The promise of critical care
informatics lies in the potential to use these advanced analytical techniques on high-resolution multimodal physiological
data in order to have a better understanding of the complex
The Scientific World Journal
relationships between physiological parameters, improve the
ability to predict future events, and thereby provide targets
for individualized treatment in real time. In the future,
we will use a system that does not simply report streams
of raw data to physicians but also synthesizes it to form
hypotheses that best explain the observed data, a system that
translates multidimensional data into actionable information
and provides situational awareness to the clinician [26].
Third, visual displays in the ICU have advanced little since
bedside electronic monitors were introduced more than four
decades ago despite the increasing volume of data collected.
For example, clinicians may be confronted with more than
200 variables [27] when caring for critically ill patients yet
most people cannot judge the degree of relatedness between
more than two [28]. This greatly contributes to preventable
medical errors [28]. Graphical displays must be carefully
and thoughtfully designed by applying a human systems
integration approach. It is important to understand not only
how information should be optimally presented to promote
a better understanding of the patient’s pathophysiologic
state and support decision-making but also to facilitate
collaboration and work-flow among the team [29]. Finally,
despite the recent growth of tele-ICUs, networks of audiovisual communication and computer systems that link ICUs
to intensivists, most of these same technical limitations
remain. That is, whether these systems are used to monitor
a patient located 300 miles away or 3 feet away, the underlying principles, equipment interoperability, data acquisition,
synchronization, and data analysis, are equally applicable.
Investment in this basic information technology architecture
is needed for the next generation of tele-ICU care.
5. Medical Device Interoperability and
Data Integration
Central to the growth of critical care has been the proliferation of monitoring technology and stand-alone medical
devices. For example, a typical critically ill patient may
undergo frequent or continuous monitoring of dozens of physiological parameters. An enormous amount of data is generated reflecting dynamic and complex physiology, dynamics
that can only be understood by data integration and clinical
context. Most of these parameters, however, are generated
from stand-alone devices that do not easily integrate with one
another. Some connect directly into the bedside monitor but
many others do not (or do so incompletely meaning that not
all the data is captured electronically). A lack of functional
medical device interoperability is one of the most significant
limitations in health care today. For example, more than 90%
of hospitals recently surveyed by HIMSS use six or more types
of medical devices and only about a third integrate them with
one another or with their EMRs [30].
In contrast to the “plug and play” world of consumer
electronics, most acute care medical devices are not designed
to interoperate. Most devices have data output ports (analog,
serial, USB, and Ethernet) for data acquisition but there is no
universally adopted standard that facilitates multimodal data
acquisition and synchronization in a clinical setting; each
device often has a unique communication protocol for data
The Scientific World Journal
transfer and often the time base for each device is independently set rather than determined from a standard source.
The development and adoption of medical device standards
to improve interoperability is ongoing. Although ISO/IEEE
11073 and ASTM F-2761 (Integrated Clinical Environment,
ICE) are two applicable standards, the former has not been
widely adopted and the latter is still relatively new (2009).
Many groups are tackling the problem of interoperability on
their own by developing the hardware and software interfaces
that enable device connectivity. Connecting with analog data
ports requires appropriate hardware interfaces, analog-todigital (A/D) converters, and filters to eliminate aliasing due
to a mismatch between sampling rate and the frequency content of the signal being acquired. It also requires that the data
be properly scaled to the voltage range of the A/D converter
(microvolts to millivolts) to maximize the resolution. Digital
data is available from some devices through connection
to serial (RS-232 or USB) or Ethernet (802.3) ports, or
using wireless (e.g., 802.11b/g or Bluetooth) communications.
Although these approaches provide the opportunity to individually interface with a variety of devices in the ICU, a system
that provides comprehensive, cross-manufacturer medical
device integration for the care of a single critically ill patient
at the bedside is not available.
Several third party systems have recently emerged specifically to help facilitate this comprehensive data acquisition
and integration. For example, Bedmaster XA (Excel Medical
Electronics, Jupiter, and Florida) is a product that can be
used to collect medical device data with access through the
hospital local area network. First introduced in 1999 to assist
clinicians by automatically acquiring a patient’s vital signs
from a GE/Marquette patient monitor, the current system
works with both GE and Philips patient monitors acquiring
parameter data (such as vital signs) from every five seconds to
every hour. DataCaptor (Capsule Technologie, Paris, France)
is another similar product that can be used to collect medical
device data.
Time synchronization of the data is a critical feature for
multimodal data acquisition from different devices and monitors. Without a “master clock” ensuring that all the values
and waveforms acquired at the same time “line up” exactly in
synch, interpreting the information and understanding the
interrelationships are difficult, if not impossible. There are
two issues. First, when data is being acquired from different
devices, each with its own internal clock, the time stamps
of data acquired simultaneously can all be different. Time
synchronization is therefore necessary when simultaneous
analog and digital data streams are acquired in order to align
the data. Second, even when acquiring data from a single
patient monitor, time drifting from natural degradation,
daylight savings time, or incorrect adjustments made by the
clinical staff need to be corrected. The Unity Time feature in
the Bedmaster XA system manages time synchronization by
insuring the accuracy of the time clocks on all GEMS devices
connected to the Unity Network. Unity Time functions in
conjunction with a NTP (Network Time Protocol) server,
as specified by the medical facility. Time clocks on all
GEMS patient monitors connected to the unity network are
automatically reset to the NTP server at a time interval
5
selected by the hospital. It is primarily these obstacles that
have limited wide spread adoption of multimodal monitoring
technology in the ICU.
6. Data Acquisition and Integration
Systems for Research
Commercial off-the-shelf products do not support highresolution physiologic data acquisition, archiving, or annotation with bedside observations for clinical applications. Such
systems have been developed in academic settings though
mainly for clinical research. Because they are not open source,
most of these systems are not readily available. This has
resulted in considerable duplication of effort in software
development for acquiring and archiving physiological data.
There have been a variety of efforts ranging from developing and testing of new mathematical and analytical tools,
to hardware/software solutions for patient data acquisition,
archiving, and visualization. A complete listing is beyond the
scope of this review but several stand out (see Table 2). For
example, Tsui and colleagues developed a system for acquiring, modeling, and predicting ICP in the ICU using wavelet
analysis for feature extraction and recurrent neural networks
to compute dynamic nonlinear models [31]. Smielewski
and colleagues from Cambridge University have developed
the Intensive Care Monitoring (ICM+) system; configurable
software based on MATLAB (The Mathworks, Natlick, MA)
that allows real-time acquisition, archiving, and analysis of
multimodal data that can then be displayed in several ways
including simple trends, cross histograms, correlations, and
spectral analysis charts. The software is intended for research
so it stores the raw signals acquired from bedside monitors for
subsequent reprocessing, thus providing the means of building a data repository for testing novel analytical methods [32].
Others have focused on multimodal data collection
linked with clinical annotation. In London, Gade and his
colleagues reported the development of the Improved Monitoring for Brain Dysfunction in Intensive Care and Surgery
(IBIS) data library that contained continuous EEG signals,
multimodal evoked potential recordings, and ECG [33]. The
system captured trend data from patient monitors, laboratory
data, and some clinical annotations. In 2001, Kropyvnytskyy
and colleagues [34] reported a similar system in Boston
called SWE for sampling, data display (WAVE), and ECG
processing. SWE was initially developed for the MIT-BIH
arrhythmia database (and now used for publicly available
databases on the National Institutes of Health-sponsored
PhysioNet website).
Sorani and colleagues from San Francisco General Hospital [35] also developed a system that captured over 20 physiological variables (plus date, time, and annotated clinical
information) from Viridia bedside monitors (Philips), Licox
tissue oxygen monitors (Integra NeuroSciences, Plainsboro,
NJ), and Draeger ventilators (Luebeck, Germany). Data was
collected automatically at 1-minute intervals and was output
into text files. Monitoring data was integrated by special custom developed middleware (Aristein Bioinformatics). Goldstein and colleagues from Oregon Health Sciences University
developed a physiologic data acquisition system that could
6
The Scientific World Journal
Table 2: Data acquisition and integration systems for research.
1996
1998
2000
2001
2003
2007
2007
2008
2011
Moody and colleagues [38]
Tsui and colleagues [31]
Gade and colleagues [33]
Kropyvnytskyy and colleagues [34]
Goldstein and colleagues [36]
Sorani and colleagues [35]
Meyer and colleagues [41]
Smielewski and colleagues [32]
Feng and colleagues [42]
Multiparameter Intelligent Monitoring for Intensive Care (MIMIC)
Acquisition, modeling, and predicting ICP
Improved Monitoring for Brain Dysfunction in Intensive Care and Surgery (IBIS)
Sampling, data display (WAVE), and ECG processing (SWE) system
Data acquisition system to capture both parametric data and underlying waveforms
Aristein Bioinformatics system
“OR of the Future”
Intensive Care Monitoring (ICM+) system
intelligent System for Neuro-Critical-Care (iSyNCC)
capture and archive parametric data (such as blood pressure
and heart rate) along with the underlying waveforms to assess
dynamic changes in the patient’s physiologic state [36]. The
system consisted of a laptop computer, a standard PCMCIA
serial card (Socket Communications, Newark, CA), RS232
serial interface cables, and custom software. The system
acquired analog data from devices incorporating antialiasing
filters along with analog-to-digital conversion. Parametric
data was sampled at a rate of 0.98 Hz and continuous wave
data either at 500 Hz (ECG) or 125 Hz (pressures, arterial
saturation, and respiration). Software managed communications with the monitoring devices; the collected signal data
were sent to a patient data server and workstation where
the files were archived. Standard analytical software packages, such as MATLAB, facilitated advanced mathematical
analyses including time and frequency domain methods and
linear and nonlinear signal metrics. Annotation of important
clinical events, such as changes in a patient’s condition or
timing of drug administration, was limited. In 2006, the same
group reported the next generation of their system that added
event markers and clinical annotation [37].
Moody and Mark from Massachusetts General Hospital
initially reported on their initial efforts in developing the
MIMIC (Multiparameter Intelligent Monitoring for Intensive
Care) database [38]. An updated version, MIMIC II, was
reported in 2002 [39]. Each record consisted of four continuously monitored waveforms (two leads of ECG, Arterial
Blood Pressure, and Pulmonary Artery Pressure) sampled at
125 Hz, along with other basic parameters (heart rate, oxygen
saturation, and cardiac output) collected every minute. The
waveforms and parameters were originally acquired from
Philips bedside patient monitors using their RDE software
tool. Using a customized archiving agent, the waveform
and parameter data were later stored onto storage drives
and converted from Philips’ proprietary RDE data format
into an open-source format (WFDB), thereby making it
accessible to others for research. Various tools were used to
analyze data. The 1-minute parameter data were processed
using wavelet analysis to identify potentially relevant clinical
events. Matching waveform records to clinical data was based
on unique identifiers such as medical record numbers, dates
of admission, and patient names. A text search engine was
created to allow users to query the database for key words and
patterns of interest. In 2011, this MIMIC II database was made
public and is available for research [40].
While MIMIC II represents a major achievement, because
physiological data and clinical annotations are collected
separately, the two datasets are poorly synchronized. Also,
physiological data and clinical annotations have different
time “granularity,” making it difficult to retrospectively determine the timing of a clinical event down to seconds. Reconstructing the clinical context with these limitations results in
correlations being largely speculative. In parallel to MIMIC
II, Meyer and Colleagues at MGH, as part of the “OR of
the Future” project, introduced a system for the operating
room to perform integration and display of data from a
variety of disparate sources, including hospital information
systems, patient monitors, surgical equipment, and a location
tracking system [41]. At the core of this system is custom
integration software (LiveData OR RTI Server, Live-Data,
Inc., Cambridge, MA, USA) that is used to capture in realtime all device data (except from infusion pumps), including
detailed physiological waveform data and all data elements,
without data loss. Custom interfaces are needed for devices
with proprietary data formats. Data are maintained in a
relational database with an archive of all captured OR data,
including trends and full resolution waveforms, information
from the location tracking system, and patient and scheduling
information (from multiple hospital information systems).
This automated database allows time-based playback and
analysis of the events of the surgery. Selected data are
displayed in real-time on an integrated display created using
scalable vector graphics.
In 2011, Feng and colleagues developed the intelligent
System for Neuro-Critical-Care (iSyNCC) to facilitate multimodal data acquisition and transmission across a local
network for storage in a database and computational analysis.
The system includes an “artifact removal” module to ensure
high quality data and the ability to provide short term data
forecasting (for example of ICP elevations). There is also a
“Recovery Outcome Prediction” module to estimate patient
long-term outcomes based on integration of the clinical
records and physiological data [42].
7. Developing the Integrated
Medical Environment
At University Hospitals Case Medical Center and Case Western Reserve University, we have focused on overcoming many
of the obstacles described and putting everything together:
The Scientific World Journal
7
Bedside
monitor
Device
Device
Device
A/D acquisition card MIB RS232
Integrated, time-synchronized
physiologic data and advanced analytics
CWRU
acquisition
software
Storage in
searchable database
Waveform
database
CWRU
analytic
software
Integration
with EMR
Lab data
Medications
Cultures
Numeric
database
Imaging
Clinician-centric
query interface
and display
Figure 1: Schematic illustration of the integrated medical environment (tIME).
high-resolution physiologic data acquisition, integration,
processing, archiving, annotation with bedside observations
for clinical applications, and visualization. The Integrated
Medical Environment™ (tIME™) (Figure 1), is a new open
source architecture that we believe can provide the backbone
for the ICU of the future. Specifically tIME™ provides (1)
real-time data acquisition, integration, time-synchronization,
and data annotation of multimodal physiological waveform
data (both analog and digital) from a variety of medical
devices and bedside monitors using custom developed parsing algorithms. Both the waveform data and the extracted
parametric numeric data are displayed using real-time algorithms developed by our group and simultaneously stored
in a local database for easy access, retrieval, and queries.
The local database can connect and import into the hospital
EMR using a secure HL7 data transfer protocol; (2) a new
critical care open middleware informatics architecture that
facilitates complex systems analysis methods and data mining
capabilities for hypothesis generation and testing; and (3)
a clinician-centric visual display and interface, to present
an integrated overview of the patient state (past, present,
and predicted futures) so that providers can make sensible
decisions at the bedside based on all the data. Only when
all of these components work in concert will we be able to
fully harness the power of information technology to improve
patient outcomes in the ICU.
computer science, biomedical engineering, signal processing,
and mathematics that many other industries have readily
embraced. Acquiring, synchronizing, integrating, and
analyzing patient data remains frustratingly difficult because
of insufficient computational power and a lack of specialized
software, incompatibility between monitoring equipment,
and limited data storage. All of these technical problems
are now surmountable. Today, we are on the verge of the
data-intensive science era in which hypotheses will be
generated automatically among the enormous amount of
data available by using computational science with inductive
reasoning. In this new era, information technology enabling
the development of an integrated critical care informatics
architecture that supports clinical decision-making at the
bedside will be essential.
8. Conclusions
[1] T. G. Buchman, “Novel representation of physiologic states during critical illness and recovery,” Critical Care, vol. 14, no. 2, p.
127, 2010.
[2] K. Tegtmeyer, L. Ibsen, and B. Goldstein, “Computer-assisted
learning in critical care: from ENIAC to HAL,” Critical Care
Medicine, vol. 29, no. 8, supplement, pp. N177–N182, 2001.
While there have been major improvements in intensive
care monitoring, including the development of enterprise
Clinical Information Systems, the medical industry, for the
most part, has not incorporated many of the advances in
Conflict of Interests
The authors declare that there is no conflict of interests
regarding the publication of this paper.
Acknowledgment
This work was supported, in part, by the Veterans Administration Research Service (I01BX000873).
References
8
[3] S. L. McLeroy and R. V. Klover, “Implementing ProblemOriented Medical Records (POMR) in an out-patient clinic,”
Journal of the American Dietetic Association, vol. 72, no. 5, pp.
522–524, 1978.
[4] T. A. Pryor, R. M. Gardner, P. D. Clayton, and H. R. Warner,
“The HELP system,” Journal of Medical Systems, vol. 7, no. 2, pp.
87–102, 1983.
[5] W. W. Stead, R. G. Brame, W. E. Hammond, F. R. Jelovsek, E.
H. Estes, and R. T. Parker, “A computerized obstetric medical
record,” Obstetrics and Gynecology, vol. 49, no. 4, pp. 502–509,
1977.
[6] J. M. Harris Jr., “COSTAR—a working computer system for
ambulatory care,” Military Medicine, vol. 143, no. 12, pp. 880–
882, 1978.
[7] C. J. McDonald, R. Murray, D. Jeris, B. Bhargava, J. Seeger, and
L. Blevins, “A computer based record and clinical monitoring
system for ambulatory care,” The American Journal of Public
Health, vol. 67, no. 3, pp. 240–245, 1977.
[8] H. Shubin and M. H. Weil, “Efficient monitoring with a digital
computer of cardiovascular function in seriously ill patients.,”
Annals of Internal Medicine, vol. 65, no. 3, pp. 453–460, 1966.
[9] E. I. McKesson, “The technique of recording the effects of
gas-oxygen mixtures, pressures, rebreathing and carbon-dioxid,
with a summary of the effects,” Anesthesia & Analgesia, vol. 13,
no. 1, pp. 1–7, 1934.
[10] P. Lauwers, P. Ferdinande, J. van de Walle, J. Willems, and
W. Theunissen, “Computer assisted monitoring in intensive
medicine,” Acta Anaesthesiologica Belgica, vol. 29, no. 3, pp. 287–
304, 1978.
[11] M. Lipkin, R. L. Engle Jr., B. J. Davis et al., “Digital computer
as aid to differential diagnosis. Use in hematologic diseases,”
Archives of Internal Medicine, vol. 108, pp. 56–72, 1961.
[12] R. Stacy and R. Peters, “Computations of respiratory mechanical
parameters,” in Computations in Biomedical Research, Academic
Press, New York, NY, USA, 1965.
[13] L. C. Sheppard, N. T. Kouchoukos, M. A. Kurtts, and J. W.
Kirklin, “Automated treatment of critically ill patients following
operation,” Annals of Surgery, vol. 168, no. 4, pp. 596–604, 1968.
[14] J. H. Siegel, J. Fichthorn, J. Monteferrante et al., “Computer
based consultation in ‘care’ of the critically ill patient,” Surgery,
vol. 80, no. 3, pp. 350–364, 1976.
[15] A. Seiver, “Critical care computing: past, present, and future,”
Critical Care Clinics, vol. 16, no. 4, pp. 601–621, 2000.
[16] K. Milholland, “Patient data management systems (PDMS).
Computer technology for critical care nurses,” Computers in
nursing, vol. 6, no. 6, pp. 237–243, 1988.
[17] F. Booth, “Patient monitoring and data processing in the ICU,”
Critical Care Medicine, vol. 11, no. 1, pp. 57–58, 1983.
[18] A. S. Sado, “Electronic medical record in the intensive care unit,”
Critical Care Clinics, vol. 15, no. 3, pp. 499–522, 1999.
[19] J. L. Manzano, J. Villalobos, A. Church, and J. J. Manzano,
“Computerized information system for ICU patient management,” Critical Care Medicine, vol. 8, no. 12, pp. 745–747, 1980.
[20] J. S. Gravenstein, “The automated anesthesia record,” International Journal of Clinical Monitoring and Computing, vol. 3, no.
2, pp. 131–134, 1986.
[21] J. M. Fitzmaurice, K. Adams, and J. M. Eisenberg, “Three
decades of research on computer applications in health care:
medical informatics support at the agency for healthcare
research and quality,” Journal of the American Medical Informatics Association, vol. 9, no. 2, pp. 144–160, 2002.
The Scientific World Journal
[22] Frost & Sullivan, U.S. High Acuity Care Information Systems
Market, 2011.
[23] GE Healthcare, “Legacy products,” http://www3.gehealthcare
.com/en/Products/Categories/Healthcare IT/Legacy Products.
[24] M. Michael Shabot, “The HP care vue clinical information
system,” International Journal of Clinical Monitoring and Computing, vol. 14, no. 3, pp. 177–184, 1997.
[25] Philips, History & Timeline, October 2013, http://www.healthcare.philips.com/main/about/company/historyandtimeline.wpd.
[26] T. G. Buchman, “The digital patient: predicting physiologic
dynamics with mathematical models,” Critical Care Medicine,
vol. 37, no. 3, pp. 1167–1168, 2009.
[27] G. Morris and R. Garder, “Computer applications,” in Principles
of Critical Care, J. Hall, G. Schmidt, and L. Wood, Eds., pp. 500–
514, McGraw-Hill, New York, NY, USA, 1992.
[28] M. Imhoff, R. Fried, and U. Gather, “Detecting relationships
between physiological variables using graphical modeling,” in
Proceedings of the AMIA Annual Symposium, pp. 340–344, 2002.
[29] J. Abraham, T. G. Kannampallil, and V. L. Patel, “Bridging gaps
in handoffs: a continuity of care based approach,” Journal of
Biomedical Informatics, vol. 45, no. 2, pp. 240–254, 2012.
[30] Medical Devices Landscape: Current and Future Adoption, Integration withEMRs, and Connectivity, 2010.
[31] F. Tsui, L. Ching-Chung, and M. Sun, “Acquiring, modeling,
and predicting intracranial pressure in the intensive care unit,”
BioMedical Engineering, vol. 8, pp. 96–108, 1998.
[32] P. Smielewski, A. Lavinio, I. Timofeev et al., “ICM+, a flexible
platform for investigations of cerebrospinal dynamics in clinical
practice,” Acta Neurochirurgica, Supplementum, vol. 102, pp.
145–151, 2008.
[33] J. Gade, I. Korhonen, M. Van Gils, P. Weller, and L. Pesu, “Technical description of the IBIS Data Library,” Computer Methods
and Programs in Biomedicine, vol. 63, no. 3, pp. 175–186, 2000.
[34] I. Kropyvnytskyy, F. Saunders, P. Schierek, and M. Pols, “A computer system for continuous long-term recording, processing,
and analysis of physiological data of brain injured patients in
ICU settings,” Brain Injury, vol. 15, no. 7, pp. 577–583, 2001.
[35] M. D. Sorani, J. C. Hemphill, D. Morabito, G. Rosenthal, and
G. T. Manley, “New approaches to physiological informatics in
neurocritical care,” Neurocritical Care, vol. 7, no. 1, pp. 45–52,
2007.
[36] B. Goldstein, J. McNames, B. A. McDonald et al., “Physiologic
data acquisition system and database for the study of disease
dynamics in the intensive care unit,” Critical Care Medicine, vol.
31, no. 2, pp. 433–441, 2003.
[37] K. Vinecore, M. Aboy, J. McNames et al., “Design and implementation of a portable physiologic data acquisition system,”
Pediatric Critical Care Medicine, vol. 8, no. 6, pp. 563–569, 2007.
[38] G. Moody and R. Mark, “A database to support development
and evaluation of intelligent intensive care monitoring,” in Proceedings of the Computers in Cardiology, pp. 657–660, Indianapolis, Ind, USA, September 1996.
[39] M. Saeed, C. Lieu, G. Raber, and R. G. Mark, “MIMIC II: a
massive temporal ICU patient database to support research in
intelligent patient monitoring,” Computers in Cardiology, vol.
29, pp. 641–644, 2002.
[40] M. Saeed, M. Villarroel, A. T. Reisner et al., “Multiparameter
intelligent monitoring in intensive care II: a public-access
intensive care unit database,” Critical Care Medicine, vol. 39, no.
5, pp. 952–960, 2011.
The Scientific World Journal
[41] M. A. Meyer, W. C. Levine, M. T. Egan et al., “A computerized
perioperative data integration and display system,” International
Journal of Computer Assisted Radiology and Surgery, vol. 2, no.
3-4, pp. 191–202, 2007.
[42] M. Feng, Z. Zhang, F. Zhang et al., “iSyNCCc: an intelligent
system for patient monitoring & clinical decision support in
neuro-critical-care,” in Proceedings of the Annual International
Conference of the IEEE Engineering in Medicine and Biology
Society. IEEE Engineering in Medicine and Biology Society.
Annual Conference, pp. 6426–6429, 2011.
9
MEDIATORS
of
INFLAMMATION
The Scientific
World Journal
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Gastroenterology
Research and Practice
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Journal of
Hindawi Publishing Corporation
http://www.hindawi.com
Diabetes Research
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
International Journal of
Journal of
Endocrinology
Immunology Research
Hindawi Publishing Corporation
http://www.hindawi.com
Disease Markers
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Volume 2014
Submit your manuscripts at
http://www.hindawi.com
BioMed
Research International
PPAR Research
Hindawi Publishing Corporation
http://www.hindawi.com
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Volume 2014
Journal of
Obesity
Journal of
Ophthalmology
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Evidence-Based
Complementary and
Alternative Medicine
Stem Cells
International
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Journal of
Oncology
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Parkinson’s
Disease
Computational and
Mathematical Methods
in Medicine
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
AIDS
Behavioural
Neurology
Hindawi Publishing Corporation
http://www.hindawi.com
Research and Treatment
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Oxidative Medicine and
Cellular Longevity
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Review Article
Iran J Public Health, Vol. 45, No.9, Sep 2016, pp.1146-1148
Technological Ecosystems in Health Informatics: A Brief
Review Article
*Zhongmei WU 1, Xiuxiu ZHANG 2, Ying CHEN 1, Yan ZHANG 1
1. Xu Zhou Institute of Medical Sciences, Xu Zhou Central Hospital, 199# South Jiefang Road, Xuzhou, Jiangsu, China
2. Xuzhou City Children's Hospital, Xuzhou, Jiangsu, China
*Corresponding Author: Email: 1187793255@qq.com
(Received 15 Feb 2016; accepted 17 Jul 2016)
Abstract
Background: The existing models of information technology in health sciences have full scope of betterment and extension. The high demand pressures, public expectations, advanced platforms all collectively contribute towards hospital environment, which has to be kept in kind while designing of advanced technological ecosystem for information technology.
Moreover, for the smooth conduct and operation of information system advanced management avenues are also essential in
hospitals. It is the top priority of every hospital to deal with the essential needs of care for patients within the available resources of human and financial outputs. In these situations of high demand, the technological ecosystems in health informatics come in to play and prove its importance and role. The present review article would enlighten all these aspects of
these ecosystems in hospital management and health care informatics.
Methods: We searched the electronic database of MEDLINE, EMBASE, and PubMed for clinical controlled trials, preclinical studies reporting utilizaiono of ecosysyem advances in health information technology.
Results: The primary outcome of eligible studies included confirmation of importance and role of advances ecosystems in
health informatics. It was observed that technological ecosystems are the backbone of health informatics.
Conclusion: Advancements in technological ecosystems are essential for proper functioning of health information system
in clinical setting.
Keywords: Ecosystems, Health informatics, Hospital administration
Introduction
Ecosystems are the information technology advancements being used worldwide for the efficient exchange of health information among
health personal and patients (1, 2). A lot of work
is being focused worldwide for improvement of
security issues associated with the above technology (3). Benkler refers to economic and technological ecosystem as a dynamic structure, which
entails of an interconnected population of organizations (4). On the other hand, Hadzic and
Chang (5) group is inclined towards digital ecosystem design methodology for the health domain. Moreover, the same group also suggested
that the analogy between information systems
and biological systems could be extended into the
systems design space.
1146
The present review article is focused on the current views of ecosystem in the health sector.
Methods
We searched the electronic database of MEDLINE, EMBASE, and PubMed for clinical controlled trials, pre clinical studies, and research articles reporting utilization of ecosystem advances
in health information technology.
Results
Importance of information system (IS) and information technology (IT) Planning in Healthcare
The better planning is essential need at present in
health sector for examining the potential utility of
Available at:
http://ijph.tums.ac.ir
Wu et al.: Technological Ecosystems in Health Informatics …
ecosystems concepts in support of understanding
the Hospital Management Information System
(HMIS). The main problem predicted in firms
involved in production of health information systems has been ruled out to be the prediction of
the effects of future technological developments
on the value of present technologies (5). Furthermore, a recent study presented another view
describing information systems as assets of information advancements in the health sector (6).
On the other hand, investigators from financial
service industry highlighted the importance of
planning with regard to inspiration to practitioners in health sector
Furthermore, a case study from the financial services industry specifically studied the issue of information systems planning. (7). Further, the impact of IS and IT planning might be, on planning
and decisions in the hospital management environment. Organizations should dedicate towards
establishment of a strategic plan in relation to key
information systems acquisitions (8). Another
explanatory research in the IT systems planning
space in healthcare revealed that there is a range
of different kinds of IT strategies in healthcare
that require diverse decisions, investments and
prioritized actions as well as differing implementation approaches (9).
IS and IT success and failure in healthcare
A key underpinning of this research is a desire to
see more effective implementation and usage of
information systems in the healthcare environment, and more particularly in the HMIS environment. There is certainly healthcare literature
pointing to success and failure in relation to hospital information systems, and the reasons for it,
but there is theory and some principles describing
the success of IS as well as IT projects. For instance, importance of management support and
the role of task interdependence as a moderating
factor collectively contributed towards the success of information systems in hospital environment (10, 11). Moreover, end user training as well
as moderating factors contributes a lot towards
success of IT and IS ecosystems (11). Another
study revealed 8 separate models of user accepAvailable at:
http://ijph.tums.ac.ir
tance of technology (12). Moreover, UTUAT the Unified Theory of Acceptance and Use of
Technology is another contribution of above
study in the field of IS circles.
In terms of the potential for information technology to assist in health care, the possible gains
are great. For instance, Gonzalez-Moler et al.
studied the implementation of novel approach of
telemedicine in the patients affected by diabetes
(13).
Most of the potential benefits of information systems, in healthcare are collectively grouped in a
study (14). The authors undertook a cost benefit
analysis in relation to the implementation of an
electronic medical record (EMR) system. Successful implementation of planning of IT and IS
in health sectors resulted in ease of creation of
medical records, decreased full time equivalent
(FTE) employees and prevented adverse drug
events too. Another, recent study reported potential benefits of IT planning success to the
hospital managers of health IT systems (9). This
implementation and duration of use of health
information technologies resulted in better follow-up of mediation guidelines at US hospitals.
Major gains can be achieved through the implementation of multifunctional, interoperable HIT
systems. There are evidences supporting the successful IT implementation in healthcare (10-15).
The collectively suggested many further insights
can be obtained about IT planning and implementation in health care, it is possible that an examination of technology ecosystems could have a
beneficial impact in this regard as a new lens
through which to examine these issues.
Conclusion
This is obvious from above discussion that technological eco systems are the key players in the
successes of health informatics. Further, improvements in these ecosystems will definitely
upshot in better health information management
system in near future. The authors declare that
there is no conflict of interest.
1147
Iran J Public Health, Vol. 45, No.9, Sep 2016, pp.1146-1148
Ethical considerations
Ethical issues (Including plagiarism, informed
consent, misconduct, data fabrication and/or falsification, double publication and/or submission,
redundancy, etc.) have been completely observed
by the authors.
Acknowledgements
The authors declare that there is no conflict of
interest.
References
1. Tweya H, Feldacker C, Gadabu OJ et al. (2016).
Developing a point-of-care electronic medical
record system for TB/HIV co-infected patients: experiences from Lighthouse Trust, Lilongwe, Malawi. BMC Res Notes, 9:146.
2. Jergensen C, Sarma A, Wagstrom P (2011). The
Onion Patch: Migration in Open Source Ecosystems.
ESEC/FSE’11, Szeged, Hungary.
3. Rlsson B, Jacobsson A (2005). On Contamination
in Information Ecosystems: A Security Model Applied on Small and Medium Sized Enterprises. 38th
Hawaii International Conference on System
Sciences, Hawaii.
4. Benkler Y (2001). The Battle over the Institutional Ecosystem in the Digital Environment.
Communications of the ACM, 44: 84-90.
5. Hadzic M, Chang E (2010). Application of Digital
Ecosystem Design Methodology Within the Health
Domain. IEEE Transactions on Systems, Man
and Cybernetics - Part A: Systems and Humans 40.
6. Besson P and Rowe F (2012). Strategizing information systems-enabled organizational
transformation: A transdisciplinary review
and new directions. J Strategic Inform Sys,
21:103-24.
1148
7. Teubner RA (2007). Strategic information systems planning: A case study from the financial services industry. J Strategic Inform Sys,
16:105-25.
8. Hosseini J (2005). Strategic Technology Planning
for the e-Commerce Enabled Enterprise. International Conference on Information
Technology: Coding and Computing
(ITCC’05).
9. Iveroth E, Fryk P, Rapp B (2013). Information
technology strategy and alignment issues in
health care organizations. Health Care Manage
Rev, 38, 188-200.
10. Enns H, Huff S, Higgins C (2003). CIO Lateral
Influencing Behaviours: Gaining Peers'
Committment to Strategic Information Systems MIS Quarterly, 27:155-76.
11. Sharma R and Yetton P (2007). The Contingent
Effects of Training, Technical Complexity,
and Task Interdependence on Successful Information Systems Implementation. MIS
Quarterly, 31:219-38.
12. Venkatesh V, Morris M, Davis G, Davis F
(2003). User Acceptance of Information
Technology: Toward a Unified View. MIS
Quarterly, 27:425-478.
13. González-Molero I, Domínguez-López M,
Guerrero M, Carreira M, Caballero F, RubioMartín E, Linares F, Cardona I, TeresaAnarte M, De Adana M, Soriguer F (2012).
Use of telemedicine in subjects with type 1 diabetes equipped with an insulin pump and
real-time continuous glucose monitoring. J Telemed Telecare, 18:328-32.
14. Li K, Naganawa S, Wang K, Li P, Kato K, Li X,
Zhang J, Yamauchi K (2012). Study of the
cost-benefit analysis of electronic medical
record systems in general hospital in China. J
Med Syst, 36:3283-91.
15. Appari A, Carian E, Johnson M Anthony D
(2012). Medication administration quality and
health information technology: a national
study of US hospitals. J Am Med Inform Assoc,
19: 360-7.
Available at:
http://ijph.tums.ac.ir
Reproduced with permission of the copyright owner. Further reproduction prohibited without
permission.
Adm Policy Ment Health (2016) 43:471–477
DOI 10.1007/s10488-015-0702-5
COMMENTARY
At the Intersection of Health Information Technology
and Decision Support: Measurement Feedback Systems…and
Beyond
Bruce F. Chorpita1 • Eric L. Daleiden2 • Adam D. Bernstein2
Published online: 24 November 2015
Springer Science+Business Media New York 2015
Abstract We select and comment on concepts and
examples from the target articles in this special issue on
measurement feedback systems, placing them in the context of some of our own insights and ideas about measurement feedback systems, and where those systems lie at
the intersection of technology and decision making. We
contend that, connected to the many implementation
challenges relevant to many new technologies, there are
fundamental design challenges that await a more elaborate
specification of the clinical information and decision
models that underlie these systems. Candidate features of
such models are discussed, which include referencing
multiple evidence bases, facilitating observed and expected
value comparisons, fostering collaboration, and allowing
translation across multiple ontological systems. We call for
a new metaphor for these technologies that goes beyond
measurement feedback and encourages a deeper consideration of the increasingly complex clinical decision models
needed to manage the uncertainty of delivering clinical
care.
Keywords Measurement feedback Dashboard
Evidence base Treatment Clinical decision support
& Bruce F. Chorpita
chorpita@ucla.edu
Eric L. Daleiden
e.daleiden@practicewise.com
Adam D. Bernstein
a.bernstein@practicewise.com
1
UCLA Department of Psychology, University of California,
Los Angeles, Franz Hall 3227, Los Angeles, CA 90095, USA
2
PracticeWise, LLC, 340 Lee Avenue, Satellite Beach,
FL 32937, USA
This issue pulls together a valuable assortment of ideas and
observations from research teams working with a variety of
measurement feedback systems (MFS) to guide clinical
care, while confronting and studying both the human and
technological implementation challenges. Their collective
insights paint a complicated picture that documents both
promises and challenges associated with MFS. In our own
efforts to improve service delivery in children’s mental
health systems over the past 15 years, we have both used
and developed such systems, and have thus encountered
both the good and the bad firsthand. In the context of these
papers, we would like to offer some general insights for
consideration as the field moves forward.
Many of these insights concern the basic relation
between technology and decision making. Technology
generally does what we tell it to do, and thus not surprisingly, early developments in health information technology
(HIT) managed the most essential and best-understood
processes in health care systems, including utilization,
billing, and documentation. These were the questions the
field had to ask in order to function: who was seen when for
what reason, and how it was paid for? But more recently,
there has been a focus on the more complex questions we
want to ask to guide clinical care: What treatments
approaches are we using? Are they helping? What should
we do when they are not? In hindsight, many of these
questions were naively ambitious on our part, which may
explain why they were often met with such equivocal
answers as ‘‘22 sessions of individual therapy’’ or ‘‘30 days
of residential,’’ along with the associated billing codes.
Thanks to a growing vanguard of thinkers, answering
the clinically interesting questions is getting closer to a
common reality. But precisely because technology does
what we tell it to do, this function of HIT will not fully
mature until we have better articulated the complex
123
472
information models that underlie it. We can think of these
latter questions as belonging to the domain of clinical
decision support, as shown in Fig. 1. Note that there are
many strategies to support decision making that don’t
involve HIT (e.g., supervisor recommendations, use of a
treatment manual), and there are many functions of HIT
that don’t involve supporting clinical decision making
(e.g., documentation, service authorization, scheduling).
At this intersection lies a tremendous set of possibilities
and opportunities, within which MFS serves a specific
function. As the figure implies, although we believe there is
much to explore within the context of MFS development—
which entails a human-technology interaction defined by a
user receiving measurement information in the form of a
report or an alert (e.g., Bickman 2008)—there is even more
work to be done within the broader context defined by the
intersection of technology and clinical decision making
(Fig. 1, shaded region). In this regard, we prefer a metaphor of telecommunication (or an interactive workspace),
which suggests collaboration, communication, reasoning,
interaction, and even design. Such systems of course can
feed back information, but they should also be able to feed
forward information to guide action (e.g., setting expectations for what should happen next, exploring ‘‘what if’s’’).
As Bickman et al. (2014) suggest in their ‘‘final coda,’’
although the technology may be important, it should
operate in the service of decision support. In our metaphor,
we know that a telephone conference call must not drop
participants and must be free of background noise, but its
participants will also benefit from having an agenda,
speaking a common language, knowing who else is on the
call, having their thoughts organized, and sharing similar
goals for their meeting. Thus, better articulation of the
work processes to be supported on the ‘‘decision side’’ of
the figure is prerequisite to any technology that could
ultimately better serve those processes.
Fig. 1 Measurement feedback systems in the context of health
information technology and clinical decision support. MFS measurement feedback systems
123
Adm Policy Ment Health (2016) 43:471–477
There is another matter, not represented in Fig. 1,
involving the human implementation challenges that arise
once appropriate technologies are developed. For example,
Gleacher et al. (2015) highlighted the key facilitating role of
organizational leadership in achieving widespread use of
the contextualized feedback system. Without such support,
providers may mute the phone calls, or agencies may disconnect the phone service altogether, metaphorically
speaking. This is but one of many examples that involve
technology implementation issues rather than design issues
(of course the two are inevitably connected in the real
world; e.g., Higa-McMillan et al. 2011; Lyon et al. 2015;
and in treatment contexts, design alone has been associated
with implementation success; e.g., Palinkas et al. 2013,
Southam-Gerow et al. 2014). Nevertheless, in the measurement feedback context, this issue’s authors provide
considerable discussion of implementation, so we focus on
a brief list of ideas relevant to clinical decision support and
its broad intersection with technology, followed by a simple
illustration. These ideas include considerations for sources
of the evidence to be displayed, the value of exposing discrepancies between what has happened and what should or
could happen, the types and configuration of evidence to be
displayed (e.g., beyond progress rating alerts or plots) to
allow users to impose a logic model on interpretation, the
importance of automated translation of relevant ontologies
(e.g., DSM diagnoses vs. elevated scales on a standardized
symptom measure), and the creation of structures to facilitate communication and collaboration.
Multiple Evidence Bases
Daleiden and Chorpita (2005) outlined a model to coordinate and inform service delivery, which among other things
described four core evidence bases relevant to decisionmaking: case-specific historical information, local aggregate evidence, general services research, causal mechanism
research. These are outlined in Table 1, with reference to
corresponding traditions or schools of thought as well as
example questions addressed by each. In the current context of MFSs, there is a predominant emphasis on only one
of these four sources of evidence—the case-specific history
(but see Steinfeld et al. 2015, for examples of using local
aggregate evidence in the form of departmental and system
level reports). We feel much can be gained from coordinating a fuller set of relevant information from all four of
the evidence bases, which gives us the ability to detect,
consider, and act on knowledge that might otherwise
remain out of view, lost in our decision-making ‘‘blind
spots.’’
This task is not possible without a significant amount of
‘‘background complexity,’’ given that each evidence base
Adm Policy Ment Health (2016) 43:471–477
473
Table 1 Four evidence bases relevant to clinical decision making (‘‘Evidence-Based Services System Model’’)
Evidence base
Relevant tradition
Example questions
Case-specific historical
information
Individualized case
conceptualization
What is the primary clinical concern? Is this youth improving?
Local aggregate evidence
Practice-based evidence
What is the most common focus of care? How is the system performing on
average for that specific group?
General Services Research
(evidence-based practice)
Evidence-based practice
What are the best supported treatment approaches for a defined group? How
well have they worked in research trials?
Causal Mechanism Research
(clinical theory)
Individualized case
conceptualization
What would theory suggest is a promising next step? What is a logical
expectation for the outcome?
can have multiple indicators, which can even disagree (e.g.,
two independent randomized trials with discrepant findings; improvement on one measure of depression with
deterioration on another). The challenge of determining
which knowledge is ‘‘best’’ is likely an impossible pursuit,
but fortunately, perhaps not a necessary one. Rather, a
sufficient knowledge management function may be for
systems to prioritize ‘‘better’’ knowledge from a number of
sources through a series of rules or knowledge filters (e.g.,
psychometric validation for case-specific measures;
strength of evidence models for literature review; e.g.,
Chorpita et al. 2011). This is possible because MFSs
inherently create a self-correcting context, in which the real
validation of the knowledge used is accomplished by
observing whether the desired outcomes were achieved.
This notion reflects our fundamental belief that a legitimate
function of these systems can be to provide multiple
promising contextualized ideas to a decision maker, rather
than merely to provide a ‘‘right answer’’ or single prescribed action. In other words, ultimately an agent must
prioritize, act, and test results based on the information
made available, and the technology should support that
process rather than replace it.
from the one indicated by strong research support, then
there is a discrepancy between the observed value and the
expected value. Such discrepancies can motivate and guide
action, perhaps in this case to consideration of a different
treatment, and we expect that a core function of MFS is to
assist with making these discrepancies known.
Several papers in this issue provide examples of using
expected values. For example, Steinfeld et al. (2015)
describe reporting related to expected measure completion
at each encounter. Similarly, Bickman et al. (2014) illustrate alerts related to the expectation that feedback reports
be viewed by practitioners. Similarly, the model-specific
implementations (e.g., Bruns et al. 2015; Nadeem et al.
2015) communicate an expectation that particular service
activities occur, simply by incorporating descriptions of
those activities into the workflow and visual displays.
Although there is evidence that observed-value-only feedback offers advantages over no feedback (e.g., Lambert
et al. 2005), it is worth considering the decision support
value afforded by contextualizing these with expected
values.
Multiple Domains
Expected and Observed Values
In clinical care, expected values (e.g. Chorpita et al.
Research Network on Youth Mental Health 2008) refer to
information that informs our best guesses about what
should happen. For instance, if one needed to select a
treatment that might work for a given youth, one might
consider research trials involving similar youth to inform
that decision (as would be consistent with the evidencebased treatment paradigm); the treatment with the best
research support is thus the expected value for the treatment to be delivered. Expected values can be contrasted
with observed values, which represent information about
what has happened or is happening now. Staying with this
example, if the youth is receiving a treatment that differs
In the same manner that MFS often prioritize the casespecific evidence base and observed values, to date many
systems have also placed a heavy emphasis on the progress
domain (e.g., visualizing symptom change over time).
However, to facilitate technology’s relevance to clinical
decision making, we think MFS platforms should help
examine any type of information (not just progress) that fits
within the larger decision model used to guide care. For
instance, if one believes practice is related to progress, then
organizing information about practices delivered over time
and configuring that information to be synchronized with
progress measurements might be of considerable value.
More generally, any events (e.g., change of medication,
change of placement, stakeholder participation, session noshows, or end of school year) deemed relevant to progress
123
474
interpretation or strategy selection may be useful for
decision support display. This special issue provides two
illustrations of strategies for extending beyond progress
measurement. For example, Nadeem et al. (2015) show
how to track practices delivered directly on the feedback
reports. Using a different strategy, Bruns et al. describe
integration of service activities into the workflow of the
system itself.
Multiple Languages
In keeping with our telecommunication metaphor, in
addition to a shared channel, senders and receivers must
share a language and concept system to transfer knowledge. Accordingly, we see a need for translating across
the diverse ontologies that are found in mental health
research and services (e.g., Diagnostic and Statistical
Manual, Research Domain Criteria, Standardized Instrument Scores, Evidence-Based Practices, Practice Elements, etc.; see Chorpita and Daleiden 2014). Constraints
that require a single common language (e.g., a fixed set of
measures; a single clinical model) are less likely to generalize to diverse contexts and to facilitate communication
in the language of the local jurisdiction or system. The
papers in this special issue clearly illustrate the underlying dilemma. Several of the systems describe a capacity
to support multiple outcome measures (e.g., Bickman
et al. 2014; Bruns et al. 2015; Nadeem et al., 2015),
whereas Steinfeld et al. (2015) highlight some benefits of
committing to a single measurement model even though
their electronic medical record could potentially support
many. On the practice metric side, Bruns et al. (2015)
illustrate the construction and use of a model-specific
system whereas Nadeem et al. (2015) illustrate a modelspecific configuration of a generalized platform for progress and event (practice) representation. In our work
with systems, we have found tremendous value in the
ability to support diversity (of models, measures, display
preferences, etc.) within a single platform, but think that
diversity is best supported in the context of a strong
default configuration that is designed to bias users initially
toward ‘‘best practices,’’ while allowing extension and
adaptation as user expertise develops. For this to happen,
‘‘translator’’ functions (e.g., is an elevated score on the
Children’s Depression Inventory sufficiently similar to a
DSM-III-R diagnosis of Major Depression to draw an
inference for this youth?) as well as diverse ontological
libraries (e.g., configurable lists of practice elements,
evidence-based protocols, or other metrics to represent
practice delivery) are a necessary infrastructure operation
for MFS.
123
Adm Policy Ment Health (2016) 43:471–477
Collaboration
Another fundamental premise is that MFS should both
foster and structure collaboration. Rather than serving as a
substitute for human decision makers, we believe a key
role of these systems is to organize and inform those
involved in care. Collaboration can be an implicit feature,
for example, by requiring treatment team members to
select targets and measures as well as selecting benchmarks
(i.e., choosing expected values from various evidence
bases); or it can be a more explicit feature, for example, by
contiguously displaying scores from multiple informants or
practices delivered by different members of the treatment
team, allowing a full view of team activities and perspectives. Dynamic configuration, such as being able to toggle
elements on and off, extends this capability, allowing different views for different users (e.g., sharing progress and
practice history with a family member). Bruns et al. (2015)
describe features built into the workflow of the TMSWrapLogic system that prompt the type of collaboration
that is central to the wraparound service model, and Lyon
et al. (2015) found in their contextual assessment that
communication, both internal and external, were key
functions of service providers.
An Illustration
Nathan is a 17 year old Asian American male receiving
treatment for depression. Figure 2 shows parent and selfreported depression scores on a depression scale over time
in days (plots a and b). In terms of the concepts above, this
panel represents the case-specific evidence base, using
observed values, in a single domain (progress), in a single
language (T-scores on a standardized measure).
In Fig. 3, we enrich the display in a number of ways.
First, the progress panel now displays two additional plots
(c and d), corresponding to expected values. Both can
therefore be thought of as scores representing goal states.
When selecting expected values, it helps to consider all
four evidence bases outlined in Table 1, keeping in mind
that one or more expected value could be derived from
each evidence base. For example, a case-specific expected
value might be a discharge score from a previous successful treatment for Nathan. In this example, plot c is
derived from the local aggregate evidence base and represents the average post-treatment score for youth in the
system who had received treatment for depression. Plot d is
taken from the general services research evidence base, and
represents pre-post scores from a randomized clinical trial
for depression that included youth of the same age and
ethnicity as Nathan. Of note is that simply adding expected
Adm Policy Ment Health (2016) 43:471–477
475
Progress and Practice Monitoring Tool
Case ID: NZ
Age (in years): 17.4
Treatment Target: Depression
Gender: Male
Ethnicity: Asian American
85
Progress Measures
80
Left Scale
a RCADS-Depression
75
b RCADS-P-Depression
70
65
60
Right Scale
55
50
180
160
140
120
100
80
60
40
20
0
45
Fig. 2 A progress panel with observed values for depression scores over time. RCADS revised child anxiety and depression scale, RCADS-P
revised child anxiety and depression scale-parent version
Progress and Practice Monitoring Tool
Case ID: NZ
Age (in years): 17.4
Treatment Target: Depression
Gender: Male
Ethnicity: Asian American
85
1
Progress Measures
80
Left Scale
a RCADS-Depression
1
75
b RCADS-P-Depression
1
70
c Benchmark
65
d Trend
1
60
Right Scale
0
55
0
50
45
Literature
Practices
Performed
180
160
140
120
100
80
60
40
20
0
0
Expert
******* FOCUS *******
Relationship/Rapport Building
Child Psychoed: Depression
Psychoed: Depression (CG)
Activity Selection
Cognitive: Depression
Problem Solving
Relaxation
Social Skills
Communication Skills (CG)
Support Networking (CG)
Goal Setting (CG)
Maintenance (CG)
******* INTERFERENCE *******
Child Psychoed: Anxiety
Cognitive: Anxiety
Exposure
Self-Rewards
Fig. 3 A progress and practice panel showing observed and expected
values over time. CG indicates a caregiver directed practice, RCADS
revised child anxiety and depression scale, RCADS-P revised child
anxiety
and
depression
scale-parent
version,
Psychoed
psychoeducation. The shaded region of the practice panel indicates
a hypothetical illustration of practices supported by relevant research
trials (i.e., expected values for practice)
123
476
values for treatment progress can have a dramatic effect on
interpretation of progress by creating context. That is, it is
quite possible to infer satisfactory progress when examining Fig. 1, but less possible when inspecting the top panel
of Fig. 2, given the discrepancies between observed and
expected values.
This discrepancy may effectively indicate when to act,
but without additional information, it may say less about
how to act. Given a basic logic model that practices affect
outcomes, inclusion of a practice panel can help in this
regard. The bottom of Fig. 2 uses white circles plotted on
the same time axis as the progress ratings to indicate that
Nathan has had 15 separate treatment sessions involving 9
different clinical procedures. This panel is one place to
explore to determine why he has lagged the expected rate
of progress. Once again, expected values can help, and
plotting practices coded from evidence-based treatments
(which is in essence, a translation exercise, as noted above)
indicates that the 12 practices in the ‘‘focus’’ region of the
practice panel are part of an evidence-based protocol for
depression in adolescents. Comparing these expected
practices with those observed, we find that three sessions
involved ‘‘off focus’’ practices, targeting anxiety (possible
errors of commission), and only one of five caregiver-directed practices has been delivered (communication skills),
occurring 150 days into treatment (possible errors of
omission). One other notable observation is the increasing
latency between sessions starting at about day 60.
A treatment team may thus hypothesize that caregiver
engagement may be an issue, and begin deeper inquiry,
which could include adding additional measures to the
progress panel (e.g., a caregiver assessment of barriers to
treatment or treatment expectancy) or otherwise enhance
caregiver services. The system remains interactive, collaborative, and exploratory. Once the system may have
indicated how to act, it should ideally support the treatment
team in taking the next steps, including referencing the
provider analogue of the case-specific history: does this
provider have experience and expertise with promoting
caregiver engagement? If not, are there written materials or
learning resources that could be launched from the display
to facilitate action?
Of course, this is but one example, restricted to two
domain panels, two of the evidence bases, often with only a
single expected value from each. The possibilities, however, can be as varied as conversations, consistent with our
telecommunication metaphor. When everything aligns
neatly, these conversations can be swift and clear, indicating next steps, but even when information does not align
(e.g., the research literature indicates one set of expected
values for practice, whereas an expert supervisor recommends a different set of practices), the communication and
collaboration is biased toward investigating and resolving
123
Adm Policy Ment Health (2016) 43:471–477
the discrepancy prospectively using evidence. More complex examples and illustrations are available elsewhere in
the literature, specifically regarding multiple provider
teams (e.g., Bruns et al. 2014); observed and expected
values and model integrity (Chorpita et al. 2008; Regan
et al. 2013), coordination of multiple evidence bases (e.g.
Daleiden et al. 2005), and complex collaboration structures
(Chorpita and Daleiden 2014).
Conclusion
This group of authors is to be commended for their efforts to
implement and evaluate MFS in a variety of real-world
contexts. The promises of such systems are clear, and the
design challenges, although considerable, can and will be
resolved. For technology to serve our will, however, our
field must continue to wrestle with models for how we wish
to select and organize health information. Thus, we disagree
with the notion that this burden lies with HIT developers.
There is, for the moment, a significant underspecification of
the general information and decision models needed to
dictate the functional requirements of promising new technologies. Technology will do what we tell it to, and thus, the
burden, for now, lies primarily with mental health experts.
That said, if we do our jobs right, the complexity of
these fully articulated models may soon become a constraint. Our theories of psychopathology have moved to
multifactorial risk and protective factor models, and our
intervention research has identified a multitude of interventions that, although effective when measured at the
group level, involve uncertainty at the case level. To help
manage that uncertainty, the current technologies struggle
tremendously and with only modest success at displaying a
‘‘human readable’’ form of a ‘‘progress only’’ model, much
less a basic practice-yields-progress logic model. Although
we encourage practice-progress type models for current
applications to help those using the technology of today,
we await the truly disruptive technology needed to support
the highly elaborated models necessary to help humans
contextually detect and respond to abstract phenomena
(e.g., behavior, interactions) in the service of their personal
pursuits and in their duty to help others.
References
Bickman, L. (2008). A measurement feedback system (MFS) is
necessary to improve mental health outcomes. Journal of the
American Academy of Child and Adolescent Psychiatry, 47,
1114–1119.
Bickman, L., Douglas, S. R., De Andrade, A. R. V., Tomlinson, M.,
Gleacher, A., Olin, S., et al. (2014). Implementing a
Adm Policy Ment Health (2016) 43:471–477
measurement feedback system: A tale of two sites. Administration and Policy in Mental Health and Mental Health Services
Research, 1–16. doi:10.1007/s10488-015-0647-8.
Bruns, E. J., Hyde, K. L., Sather, A., Hook, A. N., & Lyon, A. R.
(2015). Applying user input to the design and testing of an
electronic behavioral health information system for wraparound
care coordination. Administration and Policy in Mental Health
and Mental Health Services Research, 1–19. doi:10.1007/
s10488-015-0658-5.
Bruns, E. J., Walker, J. S., Bernstein, A. D., Daleiden, E. L.,
Pullmann, M. D., & Chorpita, B. F. (2014). Family voice with
informed choice: Coordinating wraparound with research-based
treatment for children and adolescents. Journal of Clinical Child
and Adolescent Psychology, 43, 256–269.
Chorpita, B. F., Bernstein, A., Daleiden, E. L., & Research Network
on Youth Mental Health. (2008). Driving with roadmaps and
dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy
in Mental Health and Mental Health Services Research, 35(1–2),
114–123.
Chorpita, B. F., & Daleiden, E. L. (2014). Structuring the collaboration of science and service in pursuit of a shared vision.
Journal of Clinical Child and Adolescent Psychology, 43,
323–338.
Chorpita, B. F., Daleiden, E. L., Ebesutani, C., Young, J., Becker, K.
D., Nakamura, B. J., et al. (2011). Evidence-based treatments for
children and adolescents: An updated review of indicators of
efficacy and effectiveness. Clinical Psychology: Science and
Practice, 18, 153–171.
Daleiden, E., & Chorpita, B. F. (2005). From data to wisdom: Quality
improvement strategies supporting large-scale implementation of
evidence-based services. Child and Adolescent Psychiatric
Clinics of North America, 14, 329–349.
Gleacher, A. A., Olin, S. S., Nadeem, E., Pollock, M., Ringle, V.,
Bickman, L., et al. (2015). Implementing a measurement
feedback system in community mental health clinics: A case
study of multilevel barriers and facilitators. Administration and
Policy in Mental Health and Mental Health Services Research,
1–15. doi:10.1007/s10488-015-0642-0.
477
Higa-McMillan, C. K., Powell, C., Daleiden, E., & Mueller, C. W.
(2011). Pursuing an evidence-based culture through contextualized
feedback: Aligning youth outcomes and practices. Professional
Psychology: Research and Practice,. doi:10.1037/a0022139.
Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E.
J. (2005). Providing feedback to psychotherapists on their
patients’ progress: Clinical results and practice suggestions.
Journal of Clinical Psychology, 61, 165–174. doi:10.1002/jclp.
20113.
Lyon, A. R., Wasse, J. K., Ludwig, K., Zachry, M., Bruns, E. J.,
Unützer, J., et al. (2015). The Contextualized Technology
Adaptation Process (CTAP): Optimizing health information
technology to improve mental health systems. Administration
and Policy in Mental Health and Mental Health Services
Research, 1–16. doi:10.1007/s10488-015-0637-x.
Nadeem, E., Cappella, E., Holland, S., Coccaro, C., & Crisonino, G.
(2015). Development and piloting of a classroom-focused
measurement feedback system. Administration and Policy in
Mental Health and Mental Health Services Research, 1–15.
doi:10.1007/s10488-015-0651-z.
Palinkas, L. A., Weisz, J. R., Chorpita, B. F., Levine, B., Garland, A.,
Hoagwood, K. E., & Landsverk, J. (2013). Continued use of
evidence-based treatments after a randomized controlled effectiveness trial: A qualitative study. Psychiatric Services, 64,
1110–1118.
Regan, J., Daleiden, E. L., & Chorpita, B. F. (2013). Integrity in
mental health systems: An expanded framework for managing
uncertainty in clinical care. Clinical Psychology: Science and
Practice, 20, 78–98.
Southam-Gerow, M. A., Daleiden, E. L., Chorpita, B. F., Bae, C.,
Mitchell, C., Faye, M., & Alba, M. (2014). MAPping Los
Angeles County: Taking an evidence-informed model of mental
health care to scale. Journal of Clinical Child and Adolescent
Psychology, 43, 190–200.
Steinfeld, B., Fraynt, R., & Simon, G. (2015). Progress monitoring in
an integrated health care system: Tracking behavioral health
vital signs. Administration and Policy in Mental Health and
Mental Health Services Research, 1–10. doi:10.1007/s10488015-0648-7.
123
Reproduced with permission of the copyright owner. Further reproduction prohibited without
permission.
Aller et al. BMC Health Services Research (2015) 15:323
DOI 10.1186/s12913-015-0968-z
RESEARCH ARTICLE
Open Access
Development and testing of indicators to
measure coordination of clinical information
and management across levels of care
Marta-Beatriz Aller1*, Ingrid Vargas1, Jordi Coderch2, Sebastià Calero3, Francesc Cots4, Mercè Abizanda5, Joan Farré6,
Josep Ramon Llopart7, Lluís Colomés8 and María Luisa Vázquez1
Abstract
Background: Coordination across levels of care is becoming increasingly important due to rapid advances in
technology, high specialisation and changes in the organization of healthcare services; to date, however, the
development of indicators to evaluate coordination has been limited. The aim of this study is to develop and test a
set of indicators to comprehensively evaluate clinical coordination across levels of care.
Methods: A systematic review of literature was conducted to identify indicators of clinical coordination across levels of
care. These indicators were analysed to identify attributes of coordination and classified accordingly. They were then
discussed within an expert team and adapted or newly developed, and their relevance, scientific soundness and
feasibility were examined. The indicators were tested in three healthcare areas of the Catalan health system.
Results: 52 indicators were identified addressing 11 attributes of clinical coordination across levels of care. The final set
consisted of 21 output indicators. Clinical information transfer is evaluated based on information flow (4) and the
adequacy of shared information (3). Clinical management coordination indicators evaluate care coherence through
diagnostic testing (2) and medication (1), provision of care at the most appropriate level (2), completion of diagnostic
process (1), follow-up after hospital discharge (4) and accessibility across levels of care (4). The application of indicators
showed differences in the degree of clinical coordination depending on the attribute and area.
Conclusion: A set of rigorous and scientifically sound measures of clinical coordination across levels of care were
developed based on a literature review and discussion with experts. This set of indicators comprehensively address the
different attributes of clinical coordination in main transitions across levels of care. It could be employed to identify
areas in which health services can be improved, as well as to measure the effect of efforts to improve clinical
coordination in healthcare organizations.
Keywords: Quality indicators, Coordination across levels of care, Clinical management coordination, Clinical
information coordination, Health services research
Background
Healthcare systems are in a constant process of adaptation
due to rapid advances in technology, new treatments, high
specialisation and changes in the organization of health services [1]. As a consequence, patients are seen by an everexpanding array of different providers in a variety of
* Correspondence: maller@consorci.org
1
Health Policy and Health Services Research Group, Health Policy Research
Unit, Consortium for Health Care and Social Services of Catalonia, Avenida
Tibidabo, 21, 08022 Barcelona, Spain
Full list of author information is available at the end of the article
locations, making coordination difficult [1, 2]. This is particularly challenging in the care of patients with chronic and
multiple conditions, who tend to use healthcare services
more frequently and use a greater array of services than
other patients [3, 4]. Clinical coordination across levels of
care should prevent wasteful duplication of diagnostic testing, perilous polypharmacy and conflicting care plans [5, 6];
thus the effects of clinical coordination extend beyond cost
reduction through improving quality of care [7–9].
This study is set within a conceptual framework for
analysing the performance of integrated healthcare
© 2015 Aller et al. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a
link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain
Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this
article, unless otherwise stated.
Aller et al. BMC Health Services Research (2015) 15:323
networks, which is based on an extensive literature review [6, 10] and could be applied in any healthcare area
that arranges to provide a coordinated continuum of services to a defined population. In this framework, clinical
coordination, together with continuity of care and access
to health services, is considered an intermediate objective of integrated healthcare networks and is regarded as
a means by which to reach the ultimate objectives of
quality of care, efficiency and equity of access [6, 10, 11].
To analyse the achievement of these objectives, both external and internal processes and contextual factors are
taken into account, as well as the different perspectives
(services, professionals and users) and approaches.
In this conceptual framework, clinical coordination is defined as the harmonious connection of the different health
services needed to provide care to a patient throughout the
care continuum in order to achieve a common objective
without conflicts [10, 12]. Continuity of care refers to how
individual patients experience coordination of services, and
it is defined as the degree to which patients experience care
over time as coherent and linked [1]. Clinical coordination
across levels of care consists of the coordination of both
clinical information and clinical management [6, 10]. Clinical information coordination is the transfer and use of patients’ clinical information in order to harmonize activities
between providers, and consists of two dimensions: transfer
of clinical information and the use of this information [13].
Clinical management coordination is the provision of care
in a sequential and complementary way according to a
healthcare plan shared by the different services and healthcare levels involved, and consists of three dimensions: care
coherence (i.e., the existence of similar approaches and
treatment objectives among professionals from different
levels of care), follow-up across care levels (i.e., the adequate monitoring of the patient when there are transitions
from one care setting to another) and accessibility across
levels (provision of care without interruption across levels
of care throughout the clinical episode of the patient) [13].
The results of clinical coordination can be assessed by
analysing processes aimed at coordination or their outputs (immediate results of activities related to clinical
coordination) or outcomes (final expected middle-long
term results of clinical coordination, such as hospital readmissions or avoidable hospital admissions), and using different perspectives (services, professionals, users (continuity)).
The focus of this study relies on measures to assess the outputs of clinical coordination across levels of care (primary
and secondary) by using service-based indicators.
Despite the interest this subject has generated, there
are still important gaps in terms of measures to assess
clinical coordination across levels of care and the development of new indicators continues to be considered a
priority in health policy and health services research [14,
15]. Many of the attempts to address this to date have
Page 2 of 16
focused on developing indicators to measure healthcare
outcomes which are attributed to improvements in clinical coordination [16]. However, the development of output indicators has been limited, and without this type of
indicators it is not possible to conclude that outcomes in
health care can be attributed to improvements in clinical
coordination across levels of care [15].
Existing sets of indicators are usually designed to analyse a single dimension (e.g. transfer of information) or
attribute (e.g. due completion of referral forms and discharge reports) of clinical coordination [17–20]. Those
which address more than one dimension of clinical coordination are not exhaustive in their approach to clinical coordination and are often insufficiently operative or
are not directed at the assessment of clinical coordination across levels of care [21–25]. Furthermore, the
conceptual framework used to develop these measures is
not generally explained in detail, so it is not obvious
exactly which aspects of clinical coordination are being
analysed or how measures relate to clinical coordination.
As a result of these issues, there is an overrepresentation
of some dimensions of clinical coordination addressed by
indicators, whilst other dimensions have scarcely been investigated [26]. Studies have concentrated in particular on
the transfer of clinical information [22–24, 27–30], especially in terms of completeness of information in discharge
reports [22, 30–34] and to a lesser degree in emergency
reports [30] and referral forms [20, 35], and on the followup of patients and accessibility across care levels [22, 24, 29,
30, 36]. Only a few studies have used indicators to measure
clinical coherence between care levels [30, 37].
The aim of this study is to develop and test a set of output indicators to comprehensively evaluate clinical coordination across care levels of care, i.e. addressing both types
of clinical coordination, information and management,
and their dimensions and attributes.
Methods
The study consisted of two phases: in the first phase, a
set of indicators to measure clinical coordination across
levels of care was developed based on the literature review and expert discussions, and in the second phase,
the set was tested in three different healthcare areas.
1. Development of a set of indicators to measure clinical
coordination across levels of care
Identification of indicators: literature review
The study was based on the conceptual framework for
analysing the performance of integrated healthcare networks [6, 10], which identifies two types of clinical coordination across levels of care (clinical information and
clinical management) and five dimensions (transfer of information, use of information, care coherence, follow-up
across levels and accessibility across levels). A systematic
Aller et al. BMC Health Services Research (2015) 15:323
review of literature was undertaken to identify previously
developed indicators. A computerised search of the following bibliographic databases was conducted: Pubmed,
Social Science Citation Index, Science Citation Index,
ECONLIT, CINAHL and LILACS, in addition to standard
internet search engines such as Google. The search strategy included a combination of descriptors and keywords
relating to clinical coordination (‘coordination of care’ or
associated key terms with similar meaning), levels of care
(‘primary care’, ‘secondary care’, ‘hospitalization’, ‘interface’,
‘cross-level’ or associated terms) and measurement tools
(‘measure’, ‘indicator’ or associated key terms), making use
of the Boolean operator ‘AND’. References from retrieved
studies were also screened for possible omissions....
Purchase answer to see full
attachment