Chapter 13
VIGNETTE
How well does an agency do its job? Are police really protecting society or having an impact on crime?
How can we answer these questions? Can we judge—or measure by some yard- stick—how effective
our local police department is in reducing or preventing crime? Finally, if we develop a method to
evaluate the effectiveness of our local police department, can we use the measure to improve
effectiveness? These questions are addressed in this chapter. But we need to start with a brief exposé
on how agencies can easily misuse efforts to track productivity and how society can reap harmful
consequences.
A large city recently went to a data-driven method to measure the effectiveness of their crime-fighting
efforts. Each district was responsible for driving down crime by getting tough on criminals and increasing
the arrest rate in the district. This is a simple approach dictated by a commonsense belief that making
more arrests will act as a deterrent. Because the expectation is that increasing arrests would deter
crime, crime rates should decrease. Cutting down crime rates is the program goal, and the measure for
success is an observable measure of a reduction in reported crime rates. The program was implemented in a district plagued with high rates of poverty, drug use, and gangs. After a few months, it
appeared that the arrest rate increased and reported crime rates fell accord- ingly. To the distant
observer, city council members, the mayor, the chief of police, and other constituents, the get-tough
program is a raging success. Not so fast, my friends!
Most criminal justice practitioners will tell you they can generate numbers and data almost at will. In
policing, if a district commander treats the troops well, they will try and protect the commander. If a
respected district commander is subjected to pressure to produce numbers, the line officers will find a
way to produce the numbers. Going back to our example, it was discovered that, indeed, the police
officers working in the field were getting the numbers their district commander needed. First, it is not
difficult to increase the number of individuals arrested per time period (the arrest rate), especially in the
crime, drug, and gang-ridden area described. It is psychologically easy to arrest young people with
whom the police have always had a conflict relationship and who are considered on the margins of
criminality. The justifications for arrest are also bountiful and range from suspicion of drug dealing,
resisting arrest, assault of a police officer, obstruction of justice, and consuming alcohol in public to a
host of—the independent var- iable, as it were.
Altering the crime rate is a bit more complicated. In this case, the reported crime rate was being used.
To contrive a decrease in reported crime, the district personnel found ways of making it more difficult
for citizens in their district to report crimes. There are a number of ways of doing this. One method is to
require individuals to travel to the district center station to report crimes. Once there, citizens reporting
crimes can be made to wait in lines and fill out lengthy documents. Worse, citizens reporting crimes can
be subjected to subtle hazing by officers at the district center. A former offender from the area, for
example, came in to report that his car was stolen. As the story goes, the officers at the district mocked
the ex-offender, suggesting that a thief had a lot of nerve complaining to the police that he had been
robbed. Another accused him of stealing his own car for insurance fraud and threatened to arrest him.
The ex-offender withdrew the complaint and left the district offices. Not all hazing and harassment was
this harsh, but overall the number of individuals from the district who came in to report a crime went
down over time. In other words, as a result of the district’s unethical and rude treatment of the local
citizens, the willingness of inhabitants in the district to file complaints went down and so did the
reported crime rate. Here, we have it: Police behav- ior changed to increase arrest rates. The reduction
in the reported crime rate was also a result of a change in police behavior and activities. The community
members within the district suffered disrespectful treatment by their police officers, crimes were not
investi- gated, and offenders committing the unreported crimes were not apprehended. One might
argue, however, that if the aggressive increase in arrests served to reduce the actual crime rate, the lawabiding member of the district benefited. But did the actual crime rate in the district go down as a result
of the more aggressive policing? There is no way to tell if the actual crime rate was reduced or, for that
matter, increased using the contaminated data contrived in this case.
For many people, the very concept of organization implies purpose, and the question of how well
purposes are met is central to understanding organizations. Organizational effectiveness is thus a central
theme in both the pragmatically oriented literature on management and the theoretically oriented liter- ature of organizational
behavior. For many managers, determining effectiveness involves identifying the criteria with which to
assess effectiveness, measuring these criteria, and weighing the various outcomes. Implicit in these
steps, how- ever, are important theoretical questions, such as: Effectiveness for whom? How are
outcomes to be measured? What is a good outcome? Such inescapable ques- tions illustrate the
complexity of the concept of organizational effectiveness.
That complexity is evident in most discussions of organizations (see Peters and Waterman, 1982). For
example, Tayloristic managers might cringe at a policy that 3M Company finds central to its
effectiveness. At 3M, some employees are expected to steal company time and material for their own
creative enterprises in the hopes that this theft will produce marketable innovations. (Those sticky PostItTM notes illustrate the potential for success in this approach.) Managers at Ford also grappled with
definitions of organizational effectiveness when, in the late 1970s, they allegedly used a cost-benefit
analysis to decide not to recall Pintos, even though they knew the faulty gas tank design was linked to
fires and the subsequent deaths of some of their customers (Cullen, Maakestad, and Cavender, 1987).
In criminal justice, the question of effectiveness is equally complicated. For example, what criteria for
effectiveness should drive prison policies on overcrowd- ing? In the mid-1980s, the Illinois Department
of Corrections granted massive numbers of “good-time” deductions, thus permitting the early release of
thousands of prisoners before the courts intervened to stop the policy. Although the depart- ment
argued that the policy was necessary for the effective management of the prison population,
prosecutors argued that it violated corrections’ fundamental pur- pose of protecting the public from
convicted criminals (Austin, 1986).
The Florida Department of Corrections faced a similar situation when it retro- actively rescinded goodtime credits for prisoners, thereby delaying the release of some offenders and reincarcerating others
when it was found that they were let out too soon. Some offenders went to court, and the Supreme
Court ruled that such a policy of rescinding good-time credits was in violation of the Constitution’s ex
post facto clause prohibiting, in effect, retroactive punishment. The result was the release of hundreds
of felons, some of whom were serious offenders. Arguing from a public protection perspective, some
politicians advocated that a constitu- tional amendment be passed to prevent the provision of goodtime credits to pris- oners on the basis of advancing public protection as the primary purpose of criminal
sentencing (CNN, 1997).
The complexity of the effectiveness issue is also seen in then-federal prosecutor Rudolph Giuliani’s
support of a three-year prison sentence in the largest case of insider stock trading. In late 1987, after an
investigation lasting nearly two years, Ivan Boesky was convicted of illegally making hundreds of millions
of dollars by trading stocks based on insider information not available to the public. Giuliani defended
the sentence, which made Boesky eligible for parole after one year, by pointing out that Boesky had
cooperated with the investigation and had provided information useful in several other cases. He argued
that cooperation is necessary in such complex cases and that a stiffer sentence may have sent the wrong
message to other stock traders considering cooperating with the prosecution.
Who is the final arbiter of an organization’s effectiveness? In a large industrial and commercial city, the
city council members threatened police executives and the union with across-the-board wage
reductions if the downtown shopping area was not cleared of what the merchants considered a crime
wave that was driving custo- mers away. The goal for the police was not particularly clear, but police
were expected to drive the criminal element from the downtown area. However, the goal for the
merchants was to see more customers; for the city council, their goal was to receive fewer complaints
from the merchants. The city council also wanted more businesses to open in the downtown area to
increase the city’s tax base. These are not goals that police executives would articulate, but they were
imposed upon the police department by economic and political forces. Consider Chapter 3 on the environment, and you can easily understand how external entities can determine organi- zational mandates
and goals and assess effectiveness separate from the goals and priorities of the organization. Moreover,
public groups such as local merchants can also declare criminal justice organizations effective or
ineffective without having any knowledge about the actual performance of criminal justice agencies on
which they depend. In this chapter, we examine the questions posed by these examples. We begin by
defining effectiveness and noting the political consequences of this definition. We then focus on
theories of organizational effectiveness, paying special attention to the limitations of the models used
frequently in organizational assess- ments. After examining a variety of methods for assessing
effectiveness, the chapter ends with a discussion of key issues to consider in determining the
effectiveness of criminal justice organizations.
WHAT IS ORGANIZATIONAL EFFECTIVENESS?
In the literature on organizations, the term effectiveness has been used in many ways. Most commonly,
effectiveness refers to the degree of congruence between organi- zational goals and some observed
outcome. This definition, however, masks many complicated concerns. For example, some have argued
that organizational survival is the best indicator of effectiveness (Hannan and Freeman, 1977). Others
have focused on adaptability to the environment rather than simply on survival. Most scholars focusing
on organizational goals have also argued that effectiveness is a mul- tidimensional concept and have
advocated the use of multiple measures to assess it. The literature on public administration argues that
social indicators, such as quality of life, an increase in a community’s ascetic values, the potential for
growth, sustainabil- ity, and so on, are ultimate goals for all public organizations (Lee, Johnson, and Phillips, 2008). To complicate the picture further, postmodernists argue that effective policing should be
victim centered and judged by the quality and quantity of service rendered to victims (Clark, 2005). In
the literature on organizations, then, effective- ness remains a largely ambiguous and ill-defined
concept. Some scholars have even questioned the value of the concept in the scientific study of
organizations (Pfeffer, 1977b). Few scholars, however, would doubt the value of the concept for
management.
Cameron (1981) identifies three reasons that the concept of organizational effectiveness remains
muddled. First, there are important differences in the way scholars have conceptualized organizations.
Some have suggested that organizations
are best viewed as rational entities pursuing goals. In this view, a police department may be viewed as
attempting to control all crime. An organization, however, might also be viewed as responding to
strategic constituencies. In this view, police man- agers may be most concerned with their impact on
property crime in a business district. Or police organizations may be viewed as primarily meeting the
needs of their members through pay schedules or shift and holiday assignments.
A second but related reason for the confusion surrounding the concept of effectiveness is the
complexity of organizations. To the extent that organizations pursue goals, those goals are often
complex, multiple, and conflicting. This com- plexity prohibits the identification of specific indicators of
effectiveness that can be applied across organizations.
Third, the confusion has been enhanced by the fact that researchers have often used different,
nonoverlapping criteria, thus limiting the accumulation of empirical evidence about organizational
effectiveness.
Scholarly discussions of effectiveness do make one thing clear. Effectiveness is not a single
phenomenon. Organizations can be effective or ineffective in a number of different ways, and these
ways may be relatively independent of one another. There is, however, little agreement on the specific
criteria that should be considered in examining organizational effectiveness. In a review of the research,
Campbell (1977) identified thirty different criteria that have been seriously proposed as indices of
organizational effectiveness. The list includes productivity, efficiency, employee absenteeism, turnover,
goal consensus, conflict, participation in decision making, stability, and communications. In criminal
justice, it is easy to imagine as long a list of idiosyncratic measures: crime rates, arrest rates, conviction
rates, sentences, victim sat- isfaction, incapacitation, recidivism, humaneness, attention to legal rights,
worker sat- isfaction, and increasing budgets. Obviously, such measures may often be independent or
even conflicting.
In light of ambiguities about a general definition of organizational effectiveness, perhaps the best
approach is to first address the question of why we try to assess the effectiveness of organizations.
Scholars give many answers to that question. They may be interested in accounting for the growth or
decline of organizations; they may want to investigate interactions between organizations and their
environments; or they may seek to understand the antecedents of effectiveness.
For managers, however, the answer is straightforward. Beliefs about effec- tiveness influence how
organizations are managed. Notions of effectiveness undergird many management decisions, and
effectiveness studies can have direct and tangible consequences for organizations and their members.
In criminal justice organizations, those consequences may be felt in a variety of areas, including budget,
personnel, and even mission. Treatment programs may be dismantled if they do not lower recidivism
rates. Civilian staff may replace sworn officers in traffic control and other assignments when cost effectiveness is considered (Harring, 1982). Managers or their subordinates may be fired in the face of
indicators of ineffectiveness. Prison wardens may resign fol- lowing disturbances or escapes; police
chiefs may be forced out by dissatisfied officers (Mastrofski, 1991). Managers may even redefine their
goals in response to effectiveness studies. In the mid-1970s, treatment came to be regarded as
ineffective, and departments of corrections redefined their mission by emphasiz- ing incapacitation and
punishment. Prompted by the same research, probation agencies took on responsibilities for victim
services, pretrial supervision, and increased surveillance of offenders (Petersilia and Turner, 1993). Some
have sug- gested the relevance of a crime-control approach to making probation and parole more
effective (Petersilia, 1995).
Concern with effectiveness, in short, can often lead to the redistribution of resources within and across
organizations. In a completely rational model of organizations, changes in budget, personnel, or mission
may appear to be logical consequences of efforts to assess and improve effectiveness. As we pointed
out in Chapter 12, however, rationality has its limits within organizations. Conflicting goals, inadequate
information, and the need to satisfyce rather than optimize limit organizational rationality. Under these
circumstances, effectiveness can be viewed as being subject to the same bounded rationality as decision
making. Effectiveness, therefore, might best be understood as a normative, value-laden concept used to
distribute resources between and within organizations.
For many criminal justice organizations, conflicting goals and inadequate infor- mation are no small
concerns. Take, for example, the microscope under which many prisons are now viewed. During the
1990s, many states saw massive expansion and growth in the number of prisons they managed. Prison
expansion became the norm for many states trying to get control of a bourgeoning prison population,
yet at the same time both critics and supporters were raising questions about the effective- ness of such
efforts. Some states, California, for example, passed laws that required the Department of Corrections
to take aggressive steps in its treatment of drug offenders. Under pressure from multiple interests, the
rational pursuit of public safety through imprisonment was questioned by some. And efforts to redirect
the effectiveness ques- tion toward what was actually being done to assist offenders and away from
simply warehousing them became important. Societal protection is still important, but treat- ment
expectations became normative as well. Prison effectiveness was being assessed on how prisons fit into
the larger picture of changing criminal behavior.
This question became so significant in the state of California that the Department of Corrections was
renamed the Department of Corrections and Rehabilitation in 2005, reflecting a changed expectation
that prisons were to do more to rehabilitate criminals. Questions of whether or not this was possible in
prison settings were raised by some critics who thought that departments of corrections were biting off
more than they could possibly chew and that when prisoners were not rehabilitated, the departments
would be held responsible (Stojkovic, 2005). Similar concerns have been raised by those who question
police strategies on how they fit into the larger question of societal protection and community wellbeing (see Oettmeier and Wycoff, 1998).
Effectiveness studies, like all evaluation research, take place within a political context. According to
Weiss (1972), this context intrudes in three ways. First, the organizations, programs, or offices are the
creatures of political decisions. They have been proposed, created, funded, and staffed through political
pro- cesses. Second, the results of effectiveness studies feed into the political processes that sustain or
change the organization. Third, the studies are political themselves
because they involve implicit statements about the legitimacy of goals and inter- ests within the
organization (Lovell, 2004). In this regard, McCann (2004) sug- gests that a systems model of
effectiveness requires understanding that environmental and political constraints ultimately define
effectiveness. A paroled child molester, for example, who recidivates may cause heated criticism of the
system while other forms of recidivism may go unnoticed.
Appreciation of the complexity of organizations and the political context of evaluation highlights one
important question that undergirds all discussions of organizational effectiveness: effectiveness for
whom? Regardless of the theory of effectiveness under consideration and regardless of how data may
be gathered and analyzed, this question remains relevant.
Many studies of effectiveness adopt the perspective of the organization’s dominant coalition by
reflecting the interests of those in power. As Hall and Tolbert (2005:288) note, effectiveness “lies in the
eye and mind of the beholder, with the important qualification that some beholders are more powerful
than others.” Police managers, for example, may argue that arrest rates are the best indicator of
effectiveness. Corrections managers may regard low levels of inmate violence and few escapes as
indicators of effectiveness. Other constituencies, however, may have alternative views. Internal
constituencies led by union stew- ards may base an assessment of effectiveness on working conditions.
This was the case in 1979 in the largest prison guard strike in history. New York corrections officers had
to be replaced by the National Guard when they walked off the job to protest a perceived lack of control
over inmates and their low status within the organization (Jacobs and Zimmer, 1983). Similar views have
been expressed by the California Correctional Peace Officers Union about the safety and welfare of their
members, who work in a highly volatile and tense environment where overcrowding and inmate
violence have escalated and the Department of Cor- rections has been unable to convince the public
that additional monies should be set aside to build more prisons (Gomez, 1996).
One of the most significant cases in the history of prisoner litigation also illustrates the importance of
the question of effectiveness for whom and the importance of power in determining whose view
prevails. Prior to the case of Ruiz v. Estelle (1980), the Texas Department of Corrections (TDC) was
widely regarded as highly effective based on its low costs, low incidence of reported violence, and the
general cleanliness of its institutions. A combination of internal and external constituents, however, saw
the matter differently. Inmates, prison reform lawyers, and the federal court came to regard the TDC as
grossly ineffec- tive. As one lawyer noted, “While corrections in Texas may be cheap in some senses, the
system exacts intolerable costs to the human rights of the citizens in its custody, and its unlawful
practices must be remedied” (quoted in Martin and Ekland-Olson, 1987:111). These differing views
figured prominently in the lon- gest and most expensive prisoners’ rights trial to date, a case that, after
long and bitter battles, led to the near-total reorganization of the TDC. The case was finally resolved
when the presiding judge, William Wayne Justice, threatened to impose fines of up to $24 million a
month against the Department of Correc- tions for contempt and failure to comply with court orders.
The case was settled
between the judge and the TDC after progress had been made to rectify the unconstitutional conditions
within Texas prisons. Judge Justice’s ruling suggested that the Texas prisons were effective to the extent
to which inmates received proper food. The importance of the question of perspective is also
demonstrated in a study of the use of telephones to arrange bail for pretrial inmates. Here, another
powerful external constituency was involved. An experiment in the Tombs, a detention prison in Lower
Manhattan, proved that many inmates could raise bail money simply by being given access to
telephones. The social scientists conducting the study felt that increased availability of telephones not
only would benefit inmates but would also increase organizational effectiveness by reducing crowding
and saving large sums of money on pretrial detention. Implementation of increased access to
telephones was resisted, however, when prosecutors intervened. This external constituency opposed
the policy because it was seen as weakening their position in plea bargaining. Because detained
arrestees are more likely than released arrestees to plead guilty, the prosecutors attempted to block
increased access to telephones (Lenihan, 1977).
These examples illustrate varying perspectives on organizational effectiveness. Dominant coalitions,
powerful internal constituencies, and powerful external constituencies may all have different ideas as to
what makes for an effective orga- nization. Because those perspectives may lead to different
distributions of resources in organizations, it is critical to understand whose perspective underlies any
discussion of the effectiveness of an organization. Often discussions on orga- nizational effectiveness
within criminal justice organizations frame the debate on how measures of organizational activity are
limited and provide no new insights on how effectiveness can be addressed. Where there has been
improvement in examining organizational effectiveness within criminal justice organizations, it has been
led by the police field.
Sparrow (1999) discusses how issues of police effectiveness can be examined by posing different
questions concerning police operations. Take, for example, the question of police efficiency. Sparrow
suggests that the traditional police department responds to this question by examining detection and
arrest rates as the measures of police performance. Sparrow offers an alternative view by fram- ing the
question differently. Under a community policing model, this question would be answered by examining
the degree to which the community is free from crime and disorder. Effectiveness, therefore, is
dependent on the perspec- tive offered and what questions of interest are advanced. For some, arrest
rates reflect some measure of activity among police; for others, unless this activity is tied to larger
concerns, such as the absence or presence of public disorder, it has limited utility when discussing
organizational effectiveness (Oettmeier and Wyc- off, 1998).
Similarly, Dalton (2004) argues that reconceptualizing police strategies can be effective in the reduction
of homicides. Based on an initiative entitled Project Safe Neighborhoods (PSN), the operating strategy is
to get police, community organizations, and academic researchers working together to analyze how
they can effectively address the homicide question. The idea is to ask the right ques- tions about
homicides. For example, how big an impact do we expect in our
efforts? How long will it take to see a measurable impact on homicides through directed efforts? Can we
really accomplish a significant reduction of homicides? Do we really want to reduce homicides and what
does that actually entail? Answers to these questions reveal how we understand police effectiveness.
Police organizations are not the only criminal justice entities where these questions are being asked.
Changing the nature of questions and answers con- cerning organizational effectiveness can be found in
probation and parole orga- nizations, prison systems, and court operations as well.
Writers and researchers during the 1990s began to raise questions on how organizational effectiveness
can be assessed differently. The emphasis has been on changing the way in which we understand and
gauge performance measures and organizational effectiveness within criminal justice organizations (see
U.S. Bureau of Justice Statistics, 1993; High Intensity Drug Trafficking Area (HIDTA), 2006b). A particular
emphasis has been on distinguishing organiza- tional outputs (e.g., arrests) from organizational
outcomes (e.g., crime reduction), with the goal of improving employee performance evaluations and
organiza- tional effectiveness. Virtually all state and federally funded projects now require some
attention to how an agency will measure effectiveness, with a heavy reliance on outcome measures.
Only when outcomes are assessed will we be able to answer if organizational efforts are effective.
THEORIES OF ORGANIZATIONAL EFFECTIVENESS
Hannan and Freeman (1977) point out that some theoretical perspective must underlie any discussion of
effectiveness. Even the question of whether an organization is regarded as succeeding or failing will
depend on theory. In this section, we review the major theoretical perspectives on the assessment of
organizational effectiveness.
The Goal Model
The goal model is the most common theoretical perspective on effectiveness and, as Hall and Tolbert
(2005) suggest, it is both simple and complex. In its simplest form, the goal model defines effectiveness
as the degree to which an organization realizes its goals (Etzioni, 1964:8). The model posits that
organizations can be understood as rational entities. In using this perspective, evaluators assume that an
organization’s goals can be identified, that organizations are motivated to meet those goals, and that
progress toward them can be measured. Evaluating companies by their profits is, perhaps, the most
obvious example of this approach. In criminal justice, such measures of effectiveness as arrest rates,
conviction rates, and recidivism all reflect the goal model.
There are some difficulties with this model. As we noted in Chapter 12, research has revealed the
limitations of the rational model of organizations. Many commentators have also noted the difficulties
involved in defining an organization’s goals (Etzioni, 1960; Simon, 1964); most organizations have
multiple and, frequently, conflicting goals. Even manufacturing firms must balance quantity with quality
goals and concern for short-term profits with long-term considerations. The situation is still more
complicated in criminal justice. Police departments are charged with controlling crime but must also
ensure due process. They also generate revenue through enforcement practices, reduce fear of crime,
maintain order, and satisfy their employees, as well as pursue many other goals. Identification of some
primary goal or goals is clearly a difficult task and one that again raises the question of effec- tiveness for
whom.
Nevertheless, all public organizations have numerous goals (Hannan and Freeman, 1977:111). In his
study of street-level bureaucracies, Lipsky (1980) considered the impact of these goals on effectiveness.
He argues that one of the characteristics of public organizations is that their conflicting goals reflect
conflicts absorbed by the organization from society at large. For example, the public gen- erally supports
services for welfare recipients, but the same public argues for reductions in welfare rolls and cutbacks in
services. The public also wants to see offenders rehabilitated but at the same time wants prison to be, at
least, uncom- fortable. One implication of Lipsky’s argument is that public organizations are designed to
be ineffective when effectiveness is ascertained by a broad-based goal model.
A second problem with the goal approach also relates to the question of what goals should be
considered but poses that question differently. Perrow (1961) distinguishes between official goals and
operative goals. Official goals are generally for public consumption and can be found in annual reports
and broad policy statements. Such goals as “to serve and to protect,” however, pro- vide little guidance
for what goes on in an organization on a daily basis. Opera- tional goals are generally derived from
official goals but tell us exactly what the organization is trying to do.
The difference between official and operative goals is illustrated in Sykes’s (1958) study of Trenton
Prison. He argues that because we know little of the technology needed to “treat” offenders and
because punishment must be tem- pered with humaneness, prison cannot accomplish either official
goal. Instead, the regimen of incarceration reveals an operative goal of simply retaining custody through
benign means. The Texas prison system confronted the same dilemma that Sykes describes but
apparently resolved it in a different fashion. Although neither treatment nor punishment were actively
pursued, the TDC did follow a strict and sometimes brutal regimen directed at maintaining order (see
DiIulio, 1987). Considering only official goals invokes unrealistic standards and ignores goals that are
actually being pursued. Considering only operative goals, however, makes it impossible to compare
effectiveness across organizations.
A third problem with focusing on organizational goals relates to the conse- quences of measuring goal
attainment. On one hand, this approach means that behavior that is not viewed as relating to goals is
not measured and, therefore, is not viewed as contributing to effectiveness. Police officers’ compassion
toward victims of crime or a judge’s exhortations to an impressionable juvenile go unrecognized if
effectiveness is measured by arrest statistics or cases processed. On the other hand, the effectiveness
criteria selected may go further and actually alter desirable behavior that is not recognized in the
measurement process. While studying an employment agency, Blau (1964) observed that the choice of
evalu- ation criteria had a dramatic effect on behavior within the organization. When the agency was
evaluated on its job placement rate, employment counselors shifted their focus from clients who were
difficult to place to clients who were the most likely to find work and who may even have been
successful without the agency. Measuring goal attainment, then, not only leaves some activity within an
organization unrecognized but may also narrow activity so that only those goals whose attainment is
measured are met.
A final concern about the goal model of effectiveness deals with the relation- ship between goal
attainment and consequences for the organization. In public organizations, this relationship is not at all
straightforward. Lipsky (1980:35), for example, suggests that the demand for services in street-level
bureaucracies will always increase to meet (or exceed) supply. He illustrates the point with the example
of a health care clinic forced to move out of a poor neighborhood in an effort to control the demand for
services. The more successful the clinic was at providing services, the greater was the demand. The
evidently bottomless demand necessitated either cutbacks in the quality of services or making services
difficult to obtain by increasing transportation problems for clients. The attain- ment of goals thus led to
drastic changes for the organization. Similarly, Wilson (2000) argues that the question of effectiveness is
always a sensitive issue for pub- lic bureaucracies because much of what they do cannot be understood
from a purely “market perspective.” Whereas goal attainment may have negative con- sequences for
some organizations, failure may not only have negative conse- quences; it may, in fact, also have some
positive consequences. For example, it is unlikely that a police department’s budget will actually be
reduced while crime rates are increasing! Likewise, when Martinson (1974) and others declared correctional treatment a failure in the mid-1970s, prison populations and resources for prisons began to
soar. When some goals are considered, then, prisons may look ineffective, but the consequences may be
positive for the organization.
Despite the complexities of the goal model, a goal-model-based program considered successful, called
Compstat (computer + statistics), was initiated by the New York City Police Department in 1995.
Compstat is considered a man- agement tool that imposes responsibility for crime analysis and crime
reduction upon district commanders and clarification of mission and goals at the district level. The
system also forces clarification of the mission and operational goals within each district. District
commanders are required to analyze crime patterns and reallocate resources to reduce crime by some
percentage estimate. What is taking place is a management method to improve the effectiveness of
police work and measure effectiveness on the basis of the extent to which set goals are achieved. For
example, crime analysis may reveal that burglaries peak at cer- tain times of day or during certain
months and in particular geographical areas. District commanders then assign more resources to the
area identified at the time and dates burglaries spiked the past year as shown by analysis. Based upon
the analysis, estimates of how many burglaries can be prevented with appropriate planned strategies
are made. The estimate becomes the measurable goal through which effectiveness of the program is
estimated. Scholars have argued that Compstat is an effective management tool that does alter police
structures and practices such that police operations can measure reduced crime (Willis, Mas- trofski,
and Weisburd, 2003). However, the drop in crime in New York City was accompanied by a similar drop in
crime rates in big cities across the country during the time period that Compstat was being applied.
The goal model, then, is a complex framework in which to consider orga- nizational effectiveness.
Despite its limitations, however, the assessment of effec- tiveness continues to be largely a process of
identifying goals, measuring them, and comparing the results against some standard. The reader,
however, should be aware of the problems and limitations of this perspective.
Some Alternatives to the Goal Model
The goal model is a broad and complex means of examining organizational effectiveness. In response to
problems with this goal approach, several other models have been developed that view effectiveness
differently. One of these, the internal process model, is consistent with human relations perspectives in
orga- nizational analysis (see Likert, 1967; Perrow, 1986). It argues that effective orga- nizations are
those in which there is little internal conflict, where information flows easily both horizontally and
vertically, and where internal functioning is smooth and characterized by trust and benevolence toward
individuals (Cameron, 1981). To the extent that this model is concerned with morale within an
organization, it may be seen as simply focusing on a limited set of goals. Such a narrow focus, however,
is not without its benefits, especially in fields like crim- inal justice, when agreement on other goals may
be difficult to reach.
Another perspective has been described as the counter-paradigm to the goal model (see Hall and
Tolbert, 2005). Participant-satisfaction or strategic-constituency models are not concerned with
questions of morale as the names may suggest. Instead, they view effective organizations as serving the
interests of key constituencies (see Cameron, 1981; Hall, 2002), which may include resource providers,
suppliers, users of an organization’s products, or even clients in social service agencies. Effective
organizations are able to maintain the contributions of these constituencies. For example, this model
might high- light the importance of good relations between the police and the prosecutor’s office or
might explain why public defenders often maintain good relationships with their supposed adversaries
in the courtroom. The model may also explain why some organizations fail. The Willowbrook School on
Staten Island, a school for the mentally challenged, was closed in the late 1970s after failing to satisfy
parents and the courts (Rothman and Rothman, 1984). An investigative reporter had sneaked into the
school and revealed deplorable conditions on the local tele- vision station. Eventually, a parents’ group
was organized, and it successfully fought the institution.
Another view of organizational effectiveness incorporates many of the ele- ments in the approaches
already discussed. Steers (1977) describes this process approach. Under this model, effectiveness is
described as a process rather than an
end state, as might be the case under the goal model. The process approach con- sists of three related
components: goal optimization, a systems perspective, and an emphasis on behavior within
organizations. Goal optimization refers to the need to balance goals and thus to optimize multiple goals
rather than to fully achieve a particular one. A systems view incorporates concerns for changes in an
organization’s environment. The behavioral emphasis suggests attention to the pos- sible contributions
of individual employees to organizational effectiveness. Under this model, then, the effective
organization is one in which goals are responsive to the environment, optimization of multiple goals is
pursued, and employees all contribute to meeting those goals. Bolman and Deal (2003) refer to the
value of such a perspective as providing multiple views, or “frames,” from which the question of
organizational effectiveness can be addressed.
One last important theory of organizational effectiveness is a substantial devia- tion from the others.
Yuchtman and Seashore (1967) developed the system resource model from an empirical investigation of
the effectiveness of seventy- five independent insurance agencies. In this model, organizations are not
assumed to possess goals, nor is goal accomplishment a relevant consideration. Instead, an organization
is effective to the extent that it can obtain needed resources from its environment. As Yuchtman and
Seashore note, the effectiveness of an organization can be defined as the “ability to exploit its
environment in the acquisition of scarce and valued resources to sustain its functioning” (1967:893).
Thus, whereas the goal model emphasizes output, the system resource model is concerned with inputs.
This difference in orientation can produce useful insights in areas where the goal model may lead to
confusion. For example, under the goal model, the failure of probation agencies to rehabilitate clients
may be regarded as ineffectiveness. The system resource model, however, would lead to the conclusion
that these same agencies have been highly effective because they were able to change their mission and
attract resources for custody-oriented surveillance programs, such as intensive supervision or electronic
home monitoring. Likewise, if running safe and humane prisons was a goal of the Louisiana Corrections
Department, then a 1975 court deci- sion ordering sweeping reforms indicates that the organization was
ineffective. However, then-Corrections Secretary C. Paul Phelps said that “the court order was the best
thing that ever happened to corrections in [the] state” (cited in Rideau and Sinclair, 1982:47). Such a
proclamation is understandable from a system resource perspective because the court order gave the
department considerable political power and financial resources to make needed improvements
(Wilkinson, 2006). Similar conclusions have been generated by those involved in local correc- tions and
the management of jail facilities (Artison, 1996). This may be true because organizational cultures—
beliefs, values, and unwritten rules—will work against methods of measuring performance and staff
behaviors that may require any cultural changes. Staff may simply produce numbers and reports needed
to satisfy the evalu- ation process without making any substantive behavioral or outcome changes (the
saying used to describe this process is “give to Caesar what is Caesar’s”). Therefore, it is important to
create a culture that supports productivity. Moreover, business orga- nizations whose cultures are both
consistent and adaptable are high performers (Kotrba et. al., 2012).
METHODS OF ASSESSING EFFECTIVENESS
Reviewing a variety of theoretical perspectives on effectiveness is useful because it not only points out
the limitations of the goal model but also provides alterna- tive ways of considering organizations. In
examining studies of effectiveness, however, it is clear that the goal model dominates efforts to assess
organizations. Studies based on this model involve the identification and measurement of some goal or
goals. Most frequently, this type of study has used a method referred to as variable analysis.
Sophisticated studies of this type try to examine causal links in the attainment of some goal. For
example, they may examine the contribution of training or supervision style to job satisfaction. Before
discussing some of the pragmatic uses of this design, it is important to note that other types of effectiveness studies are possible.
Perrow (1977) describes two alternatives to variable analysis. In gross- malfunctioning analysis, the
target of inquiry is failed or failing organizations. The analysis may examine the reasons behind a
commercial bankruptcy, the disap- pearance of a social service program, a police department
reorganization, or a major prison riot. Perrow describes gross-malfunctioning analysis as reflecting a
“primordial” concern with effectiveness because it deals with basic questions of outcome. He argues
that the subtleties of complex goals or goal displacement become irrelevant when organizations fail
dramatically. Gross mismanagement, Perrow suggests, is easy to spot, and understanding it is a useful
guide to improv- ing organizations.
One example of gross-malfunctioning analysis can be seen in the report of the National Advisory
Commission on Civil Disorders (1968), also known as the Kerner Commission. The group was formed to
investigate the causes of the urban riots that occurred in 1967 in major cities across the country,
including Los Angeles, Newark, Detroit, and New York. The commission found that African Americans in
each of the cities complained of police misconduct— harassment, brutality, and even the improper use
of deadly force. Although many causes of the riots were cited in the report, these practices, along with
aggressive patrol practices in urban ghettos, were seen as major contributing fac- tors to the unrest.
Law-and-order candidates for the presidency rejected its con- clusions (Cronin, Cronin, and Milakovich,
1981:66), but the Kerner Commission maintained that at the height of the civil rights movement, police
organizations continued to rely on enforcement strategies reflecting policies of racism and neglect.
Similar findings have been offered by subsequent commis- sions examining the role the police played in
urban disturbances (Christopher Commission Report, 1991; Fire and Police Commission, 1991).
The history of Texas prisons also illustrates the potential benefits of gross- malfunctioning analysis.
Texas prisons, too, failed to adapt to a changing envi- ronment. Prisoner litigation cost millions of
dollars, was associated with increased violence and instability, and ultimately led to the reorganization
of the prison system. Martin and Ekland-Olson’s (1987) history of the litigation makes it clear that the
policies of harassing inmate litigants and their lawyers and of ignor- ing or violating court orders
exacerbated problems for the prison system. In addition, Baro (1988) argues that much of the poor
performance of the Texas Department of Corrections can be tied to the inadequate development of
“political technologies” to cope with changing environmental conditions, notably the growing influence
of the courts in the day-to-day operations of the Texas prisons.
Perrow’s (1977) second alternative to variable analysis is called revelatory anal- ysis. Whereas variable
analysis seeks to answer the question of how well some goal is being met, revelatory analysis asks who
is getting what from an organization. In other words, revelatory analysis directly addresses the question
of effectiveness for whom by investigating how organizations are used by groups inside and outside
those organizations. Prisons, then, may be effective by virtue of the employment opportunities they
provide in rural areas. This fact explains why many rural communities have actively sought to attract
these institutions, which seem relatively ineffective by a simple variable analysis. Likewise, Perrow
suggests that organizations can be effective at meeting the individual needs of employees. A police
department, for example, may be regarded as effective by some because it offers a work schedule of
four ten-hour days per week, which allows officers to maintain second jobs or engage in their favorite
hobbies. In a variable analysis, employee morale may be seen as significant because of its assumed
effect on goals such as productivity. In a revelatory analysis, however, morale may be regarded as
significant in and of itself.
VARIABLE ANALYSIS IN CRIMINAL JUSTICE
In the assessment of organizational effectiveness, variable analysis refers to research designs that
attempt to measure the attainment of some goal. Measurement of some outcome variable is often
accompanied by investigation of the relationship between that outcome and independent variables.
These studies, then, not only lead to general statements about effectiveness based on goal attainment
but also pro- vide information on what may contribute to effectiveness and thus on how effec- tiveness
may be enhanced. For example, studies using crime rates as measures of police effectiveness may
examine the relationship between that dependent variable and independent variables, such as police
expenditures, the numbers of personnel, or the intensity of investigation (Rosenfeld, 2006; Wycoff,
1982).
Variable analysis is the most common approach to studying effectiveness in criminal justice. In this
section, we examine five issues critical to these assess- ments and reveal the complexity of this
approach.
What Domain of Activity Is the Target of the Assessment?
This question recognizes that organizations have multiple goals and that an assessment of effectiveness
may not deal with all of them. As with all organiza- tions, we could assess the effectiveness of criminal
justice organizations in provid- ing a safe and comfortable working environment for workers or
managers, or we could assess their effectiveness at garnering or spending budgetary resources.
There are also many activities unique to criminal justice organizations. For exam- ple, Wycoff (1982)
focuses on the effects of crime control activity by the police. Vanagunas (1982), however, argues that
only a small amount of police activity deals with crime-related events and that, for the “consumer” of
police services, problems unrelated to crime are more frequent and more important than crimi- nal
problems. He suggests using a human services model in the evaluation of the police, which would
include evaluating conflict-reduction efforts and emergency services. Mastrofski and Wadman (1991)
argue that a distinction must be made between performance appraisal and performance measurement.
The former refers to the processes central to the evaluation of an individual’s performance, whereas the
latter refers to the relationship between performance and actual goal accom- plishment. The latter
determination is often difficult for criminal justice admin- istrators to make. Again, the central issue is
what goal(s) is/are being assessed in the evaluation process.
Oettmeier and Wycoff (1998) propose a three-dimensional model to perfor- mance measurement
within police organizations under a community policing model. This model suggests that the community
policing model can evaluate along individual, group, and organizational dimensions to ascertain how
well the community policing model is achieving its objectives and goals. In addition, Oettmeier and
Wycoff employ the imagery of a cube to describe how these three dimensions can also be addressed by
examining the incidents, patterns, and problems in the community and how the department responds
through activities, programs, or strategies. Taken together, the model offers twenty- seven different
cubes that reflect the goals and responses of the department when operating under a community
policing model.
Such questions about the activity or activities being assessed are central. Assessments of courts may
focus on efficiency in the processing of cases or equity in the dispensation of sentences (see Goodstein
and Hepburn, 1985; Hardy, 1983). Likewise, corrections programs have often been assessed on their
ability to change offender behavior through rehabilitation, but some observers have suggested focusing
on fairness (Fogel and Hudson, 1981; Logan, 1993; Stojkovic, 2005). Attention in the 1980s turned to
deterrence (Phillips, McCleary, and Dinitz, 1983), incapacitation (Greenwood, 1982), and crime control
(Petersilia, 1995). The National Institute of Justice released a cost-benefit study that analyzed the
financial savings brought about by crime reductions due to imprisonment (Zedlewski, 1987), even
though many have questioned the primary assumptions associated with the findings (Zimring and
Hawkins, 1995). In addition, the complexity of the variables must be assessed. In the case of prisoner
rehabilitation, it is difficult to assess the long chain of antecedents that are related to individual change
in an offender. Many activities that affect personal rehabilitation, such as employ- ment and family
relations, for example, reach well beyond the scope of cor- rectional institutions (Stojkovic, 2005).
The selection of the domain of activity is a significant step in the evaluation of effectiveness. That
selection bears directly on many of the issues we have dis- cussed—most notably, the question of
effectiveness for whom, the fact that
goals often conflict, and the tendency for effectiveness criteria to influence behavior within
organizations.
What Do the Variables Mean?
After some domain of activity is selected, the next important consideration is validity, or finding
variables that actually provide measures of effectiveness in the selected domain of activity. The problem
is not a simple one. For example, recidivism rates have often been used as a measure of the
effectiveness of rehabil- itation programs. In fact, Martinson’s (1974) famous critique of correctional
treatment was based on the programs’ failure to reduce recidivism rates. As he summarized his findings:
“With few and isolated exceptions, the rehabilitative efforts that have been reported so far have had no
appreciable effect on recidi- vism” (1974:49). That summary influenced the move away from a
rehabilitation model and toward a just desserts, or punishment, model.
Work Perspective: PERSPECTIVES IN UNDERSTANDING ORGANIZATIONAL EFFECTIVENESS
I was a Special Agent in Charge (SAC) for over a decade for the U.S. Environmental Protection Agency
(EPA), Office of Criminal Enforcement Criminal Investigation Division (CID). Here, we have a blending of
two agencies. The EPA was concerned with promulgating regulations that would be imposed upon
businesses in order to minimize the disposal of pollutant emissions to protect the environment. The EPA
was made up of scientists and attorneys. My branch, the Criminal Investigation Division, was the law
enforcement branch of the EPA. We were a bunch of cops, and our job was to investigate possible
criminal violations of EPA regu- lations, make arrests, and get convictions—pretty standard stuff that
cops are good at doing. Just as any other law enforcement agency, CID special agents began their
careers in law enforcement under the tutelage of a field training officer (FTO) who was responsible for
showing new agents the ropes. The FTO’s duty is to advise new agents of the many land mines a new
special agent could encounter if he/she was not made aware of the dos and don’ts contained in what
became known as the “Agents’ Manual.” This was our Bible, our road map to success as law
enforcement officers. Do everything the manual advised; you would be considered a good agent and
prom- ised longevity in your career. As you can imagine, the Agents’ Manual was a compilation of
instructions on every- thing from drafting affidavits for the application of a search warrant, surveillance
techniques, use of body wiretaps and phone-monitoring systems, how to write reports, rules surrounding the discharge of firearms, proper procedures for arrests, and an almost endless list of
instructions. The FTO was with you until he/she thought you had mastered the tools necessary to
perform your duties as outlined in your job description as a Government Service level five Agent (GS-5).
These job descriptions outlined the requirements up through the higher pay grade of GS-13. Ordinarily,
you started as a GS-5 or GS-7 and spent an average of two years with the FTO. Your organizational
effectiveness during these early years was assessed by your FTO based on his/ her opinion of your
performance and upon your compliance with instructions in the Agents’ Manual. Your FTO’s assessment of your performance was reported up through the chain of command.
However, for the journeyman agent, his/her effective- ness was measured by success in the field. This
simply meant the number of active cases you were carrying, how many cases had been opened and then
closed, number of cases not prosecuted, the number of defendants, convic- tions, fines, restitution, and
probation. While not readily measurable but still of major significance, was the size and complexity of
the cases. This meant something in those early years. Even though you can’t realistically put a numerical
weight on complex cases, we all knew that com- plex cases take more time and effort than routine
cases, and complex cases were subjectively weighed more heavily than routine cases when considering
the effectiveness of an agent.
So we counted activities, but we also allowed for sub- jective measures, if you will, regarding the
complexity of cases; frankly, when judging the effectiveness of an agent, we considered such things as if
the agent showed up, showed up on time, showed up prepared, and was a team player. But, as time
moved on, numbers gained para- mount importance, “bean counting” encroached more heavily in the
process, and the way we gauged effective- ness changed. We were required to report all of the activities that could be easily reported with numbers. Based on the numbers, statistics would be compiled,
forwarded to Washington, DC, tabulated and compared with the other fif- teen regions within EPA\CID,
and given to the director of CID, who would pass the numbers further up the chain of command. The
level of effectiveness of EPA\CID would now be compared with all of the other entities within EPA, one
of the largest federal agencies. In CID, we could have lived with just counting any and all activities to
satisfy the new demands for numbers. If I may be sarcastic for a moment, we would have had no
problem including just how many sheets of paper it would take to write a report—or how many typos
there were in the average report. The problem was that we were now being asked to include in the
individ- ual effectiveness reports of each agent the extent to which their activities protected the
environment from pollutants. We understood that the public and Congress had a right to know—in
addition to the arrest and prosecution of indivi- duals who were poisoning our environment—the extent
to which we were keeping pollutants from the environment with our police activities. This new
component of effective- ness would also reflect upon CID as well as individual agents.
This, of course, was not good news for CID agents and managers. We were cops, not scientists.
Measuring pollution, which often is stated in parts per million or billion, was not something agents had
been trained to do. Law enforcement agencies know about search warrants, arrests, convictions; we can
identify and record most haz- ardous substances that are recovered, such as barrels of toxic materials.
However, we did not have a clue about translating all of that into a set of numbers to reflect the
impact of our police activities upon the environment. This is the work of a person with a strong scientific
background. The agency recognized this and hired scientists to work with CID agents and mid-managers.
The scientists created a form for us to fill out to explain how much pollution was kept from the
environment as a result of our policing activi- ties. Even with the help of the scientists and the form, the
process was confusing and chaotic; often the form was filled out with such wild estimates because
accuracy was not possible. We all knew that the information provided on the form was probably not
valid. What we did accomplish was to create a system that always showed positive esti- mates on the
impact our activities had on the environment.
We, in the CID, had no problem understanding the need to report more abstract results to Congress and
the public. The EPA/CID was charged with keeping the environ- ment clean and safe, after all. All of the
members of CID took great pride in making arrests and convicting individuals that were criminal in the
way they managed pollutants. But the agency’s top management lapsed on this issue. In my experience,
effective management requires a top-down, bottom-up approach in which top managers establish the
mission, set priorities, compose general operating princi- ples, obtain resources, etc. The general plan of
top man- agers is passed to middle managers who have the autonomy to implement and carry out the
mission within their district and the context of their field work. Poorly run agencies are top down only—
that is, direct orders for field agents are composed and communicated from the top with- out regard to
the realities of the field. This is what happened with the rise of numerical accountability and the
imposition of scientific outcome measures on our rather traditional law enforcement agency. Had the
leaders of EPA in Washington told us about their need to provide estimates or data on the effectiveness
of our law enforcement efforts to protect the environment, we could have worked out a system that
might have been of real value.
In fact, however, there is every reason to question the use of recidivism as a satisfactory measure of
rehabilitation. As Maltz (1984) points out, without paying attention to how programs are expected to
affect recidivism, it is impossible to tell just what recidivism rates measure. They may also be measuring
the effects of punitiveness and, therefore, special deterrence rather than rehabilitation. In other
words, “it may not be possible to disentangle the effects of the carrot (rehabilita- tion) from those of the
stick (special deterrence)” (Maltz, 1984:11).
The situation is equally complex with other measures of criminal justice effectiveness. For example,
Chicago police and prosecutors were criticized for failing to successfully prosecute a large number of
people arrested for drug offenses. It was later reported that the primary motive for the drug possession
arrests was to get gang members off the streets for brief periods of time. The police were using the
charges in much the way they had once used charges of public intoxication. It was argued, therefore,
that successful prosecution was not an appropriate criterion for effectiveness because failure to
prosecute did not mean what the critics suggested (Chicago Tribune, 1987). (Whether arrests should be
used in this way raises a completely different question about effectiveness criteria.) Reported crime,
arrest rate, and clearance rate are the most frequently employed outcome measures in assessments of
the police. Problems with the validity of these measures, however, have been well documented (see
Wycoff, 1982). In one series of case studies, McCleary, Nienstedt, and Erven (2004) demonstrated that
official crime statistics may sometimes be the result of organizational behavior that has little to do with
effectiveness. In one of the cases, the authors explained a major drop in Uni- form Crime Reports (UCR)
burglary rates in one city by a significant change in investigation and recording procedures. The city
changed procedures to require investigation of burglaries before they were recorded by the UCR clerks
rather than have investigations follow the official recording. With the change in procedures, many
events that would have been recorded as bur- glaries based on the initial patrol officers’ reports were
not viewed as meeting UCR definitions by the investigating detectives.
In a second case study, a “crime wave” coincided with the retirement of a police chief of long tenure.
The study argues that the chief had “wished” crime rates down by rewarding district commanders who
produced low UCR rates. That “wish” was then passed down through the ranks. With the chief ’s retirement, the hierarchical authority patterns within the department disintegrated, and crime rates rose.
Finally, McCleary, Nienstedt, and Erven found another unsuspected source of a crime wave. In one city,
the task of directly supervising police dispatchers was removed from shift sergeants. Without the
experience and protection of the sergeants, dispatchers began to send police officers to respond to
many calls that otherwise would have been handled informally. Department statisticians experi- enced
the increased dispatching of officers as a crime wave.
One final example illustrates the political issues inherent in defining effec- tiveness criteria. Morash and
Greene (1986) report two studies that came to radi- cally different conclusions about the effectiveness
of female police officers. The curious thing is that both studies were done by the same consulting firm in
the same city (Philadelphia) in the same year. In the first study, criteria for effective- ness were
developed by using sophisticated techniques to find consensus among police administrators. In that
study, resolving problems without arrests was an indicator of effectiveness, and female officers were
found to be as effective as
their male counterparts. Later, as a response to court proceedings and the city administration’s concern
with violent confrontations, a second study was con- ducted in which the decision to arrest was viewed
as an indicator of effectiveness. By this criterion, women were not found to be as effective as male
officers. The studies illustrate that not only do organizational factors influence outcomes but also that
the way those outcomes are viewed is the result of value judgments and political decisions.
How Are the Variables Measured?
The problems just discussed relate to how effectiveness criteria are conceptu- alized and understood.
Another aspect of the problem involves the measure- ment of the variables. The question is not
necessarily one of inaccurate measurements. Although computational errors do occur, the significant
prob- lem is the choices made among measures that may all be mathematically cor- rect. Some
measures may be more suitable than others to the specific effectiveness problem being addressed. For
example, in calculating crime rates in an urban area, considering only the number of crimes per 100,000
residents ignores the fact that many potential victims (and offenders) com- mute into the city every day.
The literature on measuring recidivism provides many illustrations of the importance of knowing how
outcome variables are measured. Although there are many examples of inappropriate models and
formulas used in recidivism research, even mathematically correct equations may produce widely
different results. Recidivism can be defined as the proportion of some specified group of offenders who
fail, according to some criteria, within some specified time. At a minimum, then, measuring recidivism
for some group of offenders requires the identification of a failure criterion and a follow-up period.
Failure may involve anything from rearrest to a return to prison with a new sentence, and the follow-up
period may be anything from a few months to many years. Recidi- vism rates can thus vary greatly
according to the failure criterion and the length of the follow-up period. The easier it is to fail and the
longer the follow-up, the higher will be the recidivism rate. To illustrate the point, Hoffman and StoneMeierhoefer (1980) computed recidivism rates on a sample of released prison inmates using various
failure criteria and follow-up periods. When return to prison was the criterion for failure and a one-year
follow-up was used, the recidivism rate was 8.7 percent. When arrest was used as the failure criterion
with a six-year follow-up, the same sample produced a recidivism rate of 60.4 percent.
An inquiry into effectiveness by Wilkins (1976) also demonstrates the com- plexity of using recidivism
measures. The Maryland Institute for Defective Delinquents at Patuxent, a well-known treatment
facility, reported a recidivism rate of 7 percent using rearrest with a one-year follow-up. This rate was
well below the 65 percent reported by similar facilities. When Wilkins examined the data, he found that
the institute had used a unique way of figuring recidi- vism. The treatment program involved a period of
institutionalization followed
by three years of outpatient supervision and services. The institute counted an offender as a recidivist
only if the juvenile was not returned to the institution during the three-year outpatient phase and was
later arrested. Those who were returned to the institute during their period of outpatient services were
not counted among the recidivists. Wilkins showed that the likelihood of anyone from any program
failing after three years was about 7 percent and that Patuxent did no better than other institutions
when the three-year outpatient period was included in the analysis.
Petersilia and Turner (1993) also found very high rates of failure among offenders on intensive
probation and parole due to the nature of the interaction between offender and agent. Increased
surveillance and supervision of offenders led to an increase in “technical violations,” which caused many
offenders to be sent back to jail or prison. The authors suggest that different ways of assessing intensive
probation and parole supervision be developed in addition to the tradi- tional measure of recidivism.
The research underscores two important points about the measurement of variables. Those conducting
studies of effectiveness need to be aware of the implications of selecting different ways of measuring
outcomes, and the reader of reports on effectiveness must understand precisely how the outcome
variables were measured.
Alternatives to Outcome Measures
Until now, our discussion has focused primarily on outcome variables as indica- tors of effectiveness.
Crime rates, arrest rates, convictions, and recidivism are all measures of the outcome of organizational
activity in criminal justice. There are, however, many limitations to such variables. They are complicated
and expen- sive to measure. They can often be assessed only long after some activity has taken place.
And many factors outside the organization may influence them. It is easy to see, for example, that police
can have only a limited impact on crime rates and that many factors other than prison may contribute to
an ex-offender’s return to crime (Stojkovic, 2005).
In response to the problems of outcome measures, many managers and researchers have turned to
measures of process or structure as indicators of effectiveness (Scott, 1977). Process measures are
measures of the activities assumed to cause effectiveness within organizations. For example,
assessments of probation and parole agencies have incorporated variables, such as the length of time
probation/parole officers spend in supervision of each case or the length of time they spend in face-toface contact with parolees. Judges might examine the length of time a case takes to go from indictment
to final dispo- sition, and police managers often examine numbers of traffic stops made by officers. The
advantage of these measures is that they are easy to collect and are likely to be easier than outcome
measures to influence directly. A disad- vantage lies in the fact that they may not be as highly related to
outcome measures as is often assumed.
Structure measures are still further removed from outcomes. These variables measure organizational
features or participant characteristics that are presumed
to have an impact on effectiveness. Courts have used the ratio of corrections officers to inmates as a
measure of inmate safety in prison. Likewise, the level of training among staff has been used in assessing
the effectiveness of many crim- inal justice agencies. One way of viewing these measures is to regard
them as measures of organizational inputs that serve as surrogate measures for outputs.
Using Multiple Measures of Effectiveness
There has been considerable criticism of evaluation efforts that use single measures as indicators of
effectiveness (see Chen and Rossi, 1980). Whether structure, process, or outcome variables are used,
single measures have often been regarded as too simplistic and often uninformative. Single-variable
mea- sures assess achievement of only one goal and are, therefore, based on a lack of appreciation of
the complexity of multiple goals within organizations. Fur- thermore, single-measurement analyses
inevitably conclude that a goal either has been reached or has not been reached. A program is thus
regarded as a success or as a failure. Such a conclusion provides a static, one-shot view of effectiveness
and little information on how an organization might change or improve.
Multigoal/multimeasure designs give a more comprehensive view of organiza- tional effectiveness than
do single measures. These designs utilize a variety of measures to assess the achievement of multiple
goals. In doing so, they permit examination of the effectiveness of different domains of organizational
activity and examination of the relationship between the achievement of various goals. These models
can also view effectiveness as an ongoing activity and thus provide information about how an
organization can improve. Research measuring the effectiveness of a multi-jurisdictional drug task force
considered arrest rates and quality cases brought to prosecutors as goals (Smith et al. 2000). One may
argue, however, that arrests are a process measure to an ultimate goal of reducing the rate of drug use.
Blomberg (1983) provides an example of the multigoal/multimeasure approach in assessing the
effectiveness of juvenile diversion programs. These pro- grams have often been evaluated from a singlegoal perspective. They also have often been regarded as failing because the goal of diverting nonserious
offenders from the criminal justice system has been displaced as the diversion programs became an
add-on intervention rather than an alternative. In other words, through net widening, diversion
programs have sometimes come to serve clients who would not even have been arrested if the diversion
program did not exist. Where evaluations have focused on rearrest data, these single-outcome
measures have also not shown these programs to be effective.
Blomberg is critical of the simplicity of such evaluations. As an alterna- tive, he suggests a
multigoal/multimeasure approach that incorporates at least three broad measurements. First, measures
of structure identify the types of youth served by the diversion programs, including such variables as
age, eth- nicity, social status, and offense history. Second, measures of services provided by the
programs can distinguish between various types and intensities of
programs. Finally, outcome measures, such as rearrest data, should be included. Taken together, these
variables recognize that different types of programs may serve different types of youth. The evaluation,
therefore, addresses the question of what works for whom and thus provides informa- tion for program
improvement.
Another model of multigoal/multimeasure evaluation is an assessment of the juvenile corrections
system in Massachusetts (Coates, Miller, and Ohlin, 1978). Researchers from the Harvard Center for
Criminal Justice carried out the evalu- ation of the community-based system that replaced juvenile
institutions after they were closed by Jerome Miller, the commissioner of youth services, in the early
1970s. The study incorporated a variety of structure, process, and outcome variables.
The assessment began with a theoretical model of community-based correc- tions as running along a
continuum from institutionalization to normalization. Variables such as the types of relationships
between staff and inmates and the types and extent of links to the community, were used to distinguish
between facilities that created social climates more like traditional correctional institutions and facilities
that created social climates that were more open and approximated normal life. The researchers found
that some of the programs had social climates and links to the community that nearly duplicated those
of institutions. Most often, these programs focused on changing offenders’ values rather than reintegrating them into the community. This structural dimension was then examined to see how it related to
process and outcome variables.
The process variables were decisions made to place youth in the various types of programs. The
researchers found that youth with more extensive records were more likely to be placed in the less open
programs. Thus, many youth did not benefit from the new treatment programs and those most needing
the inno- vative programs were least likely to get them.
The study also incorporated a variety of outcome measures. Cost and recidi- vism outcomes were
examined, as were a variety of attitudes in the short and long term. Attitudes were examined at release
from the program and again at six months after release. The evaluation revealed that the open, or
normalized, programs had the most positive effects but that these modest program effects diminished
when the youth returned to their homes.
The complex design of the Massachusetts evaluation permitted conclusions that went far beyond
commenting on the success or failure of the community- based programs. The study showed that most
youth offenders could be handled in the community-based system. Another finding was that
“community based” was an inadequate description and masked a wide range of programs. Yet another
finding was that the more open programs were more likely to be suc- cessful. The evaluation also
produced significant recommendations for program improvement, which ranged from increasing links to
the community to strength- ening advocacy work and follow-up in the community.
These five issues illustrate the complexity of variable analysis in the assess- ment of organizational
effectiveness. At first glance, these issues may appear to deal with narrow technical problems. Careful
consideration, however, reveals that they get to the heart of understanding organizational effectiveness
under the goal model. Attention to these issues will require that decisions be made about what goals
are relevant to effective organizations, how attainment of those goals can be assessed appropriately,
and what benefits their assessment can have for members of the organization.
Attention to the issues will also suggest that the assessment of effective- ness can be an ongoing activity
that provides information for the continuing change and improvement of organizations. Skogan (1996)
suggests that any evaluation of a program must have a basic understanding of the “logic model of the
program.” This logic model has four specific components: inter- vention (level of effort involved),
context (surroundings and circumstances where the intervention is being placed), mechanism (how the
program is to affect the outcome), and, finally, outcomes (anticipated outcomes of the pro- gram). By
following this logic model, it is possible to make more definitive statements concerning program
effectiveness. Other researchers have noted how assessments of program effectiveness have become
critical for criminal justice administrations; they no longer are simply a luxury but a requirement. (See
Fyfe et al. [1997] for a discussion of program effectiveness as it relates to police performance issues.)
ETHICAL CONSIDERATIONS IN ASSESSING ORGANIZATIONAL EFFECTIVENESS
We began and now end this chapter by thinking about the potential for abuse in efforts to measure
effectiveness. As any practitioner can testify, numbers and data can be produced to give the appearance
of compliance to rules or to achieving organizational goals. The motivation to fudge data exists in
practitioners if they are cynical about the value of attempts to evaluate their work, mistrust the evaluation process or evaluators, or fear that they will somehow be made to look bad or harmed in some
manner as a result. Moreover, most experienced practitioners know better than most of us the
complexity of their world and the gulf between official and operational goals. As a result, they will doubt
that they can be evalu- ated fairly. Hence, to protect themselves, they may feel it necessary to manipulate data or the process to their benefit. Real or perceived competition among practitioners or
supervisors if fostered through an evaluation process can also give rise to fudging data or altering
reports. Moreover, because criminal justice practi- tioners work outside the immediate purview of their
supervisors, they have the freedom to play out their roles in such a manner to produce the appearance
of “good results.” Police officers, for example, have unfettered opportunities to avoid making arrests or
to make an arrest when it is not necessary. Corrections officers may choose to report or not report drug
use on the part of inmates depending on longer-term consequences to them or their warden if formal
records indicate increasing or decreasing incidents of drug use by inmates. Hence, the motivation to
contaminate an official feedback loop is potent, and
the opportunity is endemic to the structure of criminal justice agencies. Hence, these issues must be
addressed before the initiation of any formal effectiveness evolution is implemented.
First, any evaluation process must be seen as making sense to those who will eventually be judged.
Including the rank-and-file in designing planned effectiveness research can help in this regard as well as
increase credibility toward the evaluators. Also, executives or evaluators need to set realistic goals by
which to judge effectiveness. Being measured against unrealistic goals will justify falsifying data in the
minds of many. Next, organizational members need to feel that heads are not going to roll and egos are
not going to be bruised—at least badly—as a result of the process and outcomes. In addition,
organizational members need to believe that they are being judged within their domain and that the set
of circumstances unique to those domains is being considered. But the major consideration for agency
members will be how the results of the evaluation will impact their careers or affect their professional
reputations.
Chapter 14
VIGNETTE
Mental health courts dedicated to processing mentally ill offenders are finally a reality. The advent of
criminal courts for offenders who are mentally ill is a major step in bridging the gap between the
criminal justice system and mental health systems (Redlich et al. 2006). Beginning in the early 1970s,
criminal defendants who suffered from mental illness were becoming a problem for law enforcement,
courts, local jails, and prisons because they were released en masse from state mental institutions. Prior
to the late 1960s until the 1970s, mentally ill citizens who became a behavioral problem for families,
communities, or law enforcement ended up fairly routinely in state mental institutions. The problem of
manag- ing mentally ill citizens began to shift from state mental hospitals—often referred to as insane
asylums—to a system of community mental health. The shift was a result of our col- lective observation
during the previous decade that citizens placed in “insane asylums” had few civil rights and would be
kept in the asylum until they were diagnosed as “sane” by staff psychiatrists. Individuals were sent to
insane asylums by criminal courts if it was determined that the individual was incapable of standing trial
due to their illness, and the person was to be returned to the courts after they were cured. Offenders
considered criminally insane— not guilty by reason of insanity—were also sent to asylums by criminal
courts. People also ended up in asylums through a civil commitment by a family or a probate court if it
was determined they were sufficiently mentally ill as to require institutionalization.
In the 1960s—the decade in which individual civil rights were being emphasized—all public institutions
were being scrutinized. It became clear to civil rights activists that inmates in our mental institutions
were being managed in an arbitrary and capricious man- ner, often kept in custody for years beyond a
sentence for a “sane” criminal offender. Fur- ther investigations revealed that the conditions in the
institutions were abysmal, often leading to cruel treatment of inmates. The law changed as a result of
our changing view of mentally ill citizens as well as our emphasis on civil rights, and it became more
difficult to place individuals into mental institutions and almost impossible to keep them there for any
length of time. This solution basically closed mental institutions and released patients/ inmates back to
their communities. But we then had mentally ill individuals in our communi- ties rather than kept away
from us in an asylum. Now what?
A community mental health system was created to provide treatment and assistance to the mentally ill
now living in their communities. However, the system was dramatically underfunded and mental health
workers, unlike criminal justice professionals, were never trained to work with criminal offenders who
also suffered from mental illness. In addition, criminal justice practitioners were trained to work with
criminal offenders but did not have a clue about working with individuals who suffered from mental
illness. Hence, we had a classic performance gap: Mental health workers would not provide services for
criminal offenders who were mentally ill. They considered this group of individuals as the responsi- bility
of police and the criminal justice system. Conversely, police who encountered men- tally ill individuals as
offenders, whether the individuals were acting out their delusions in public or committing serious
crimes, had no idea how to manage them and received little help from the experts, the mental health
workers.
Individuals with mental illness were a real problem for jail personnel. They had no idea how to handle
individuals who were acting bizarrely—talking to God or other invisible con- tacts, for example. The days
when a mentally ill arrestee or inmate could be easily transferred to a mental institution were over. As a
result, mentally ill offenders and inmates received very little mental health treatment, and jail officers
coped the best they could. And, of course, more and more, mentally ill citizens found their way into the
criminal justice system and became a major problem for those managing local jails and prisons.
Gradually, changes in the system began to take hold. Cross-training between criminal justice
practitioners became common; large jails and prisons gradually established a mental health treatment
process for inmates. In an odd way, jails and prisons took the place of the old asylums, certainly an
unintended con- sequence of earlier changes. Finally, the system has established courts that deal
exclusively with mentally ill offenders. The courts and their probation staff are all trained to deal with
men- tally ill individuals and find the best path to deal with their problems in the long term rather than
resorting to traditional probation services or confinement to jails and prisons. Change happens—
especially after a problem gets everyone’s attention and it becomes apparent to decision makers that
we are not expending our resources wisely.
This chapter focuses on organizational change within the criminal justice system and its organizations. It
is suggested that readers review the first several pages of Chapter 3, The Criminal Justice System in its
Environment, before reading this chapter. Organizational change and its attendant concepts and
theories cover minor procedural changes within an agency as well as sweeping reforms that change the
philosophy and operations of an entire system. At one extreme, changes can take place in an agency
because of internal decisions; at the other extreme, major system-wide change is typically the result of
reform movements that emanate from cohesive groups in society at large. Historically, such
organizational change takes place within the context of general social changes that create, or are
created by, new perspectives, ideas, or paradigms that force the organization to adapt to the external
forces (Schein, 2004). Henry (2007:78) suggests that public agencies staffed by professionals change
when there is a perceived loss of status. Public agen- cies protected by patrons—such as lobbyists or
legislators—change when they lose their patrons’ support. However, routinized public agencies that
have followed the same processes and routines for years on end have little capacity for change, and
they change only after “expectations about what they should be doing increase in a sustained way over
time.” For better or worse, it seems that the criminal justice system and its agencies often fall into this
category. However, change has occurred in the criminal justice system, at times as a result of the need
to adapt. In addition, change is more likely to occur in the face of a crisis that affects an agency or the
system (Rochet, Keramidas, and Bout, 2008).
The criminal justice workforce is rapidly becoming more diverse with the inclu- sion of minorities and
women. This change, which itself will impact the system, has its roots in the earlier civil rights
movement that, in turn, produced legislation ensur- ing minorities and women an equal right to
employment opportunities, which again led to affirmative action requirements and programs. Rothman
(1980) argues that the prison reforms that took place at the beginning of the last century were the
natural consequence of the Progressive era of 1890 to 1920, during which all social institutions were
being questioned and changed. The perceived need for reform and change in the criminal justice
system, however, preceded the Progressive era; attempts to bring reform to the criminal justice system
began at least as early as the mid-1800s, when prison reform was pursued in New York and
Pennsylvania. In essence, our present parole system, which is now being challenged and has been
eliminated in some states, was developed in the late 1800s to emulate the “success- ful” system of
penology developed by Sir Walter Crofton in Ireland (Barnes and Teeters, 1959). In 1870, penologists
from across the United States met in Cincinnati at the National Congress of Penitentiary and
Reformatory Discipline. The goal of this congress’s dedicated members was to reform and reorganize
the existing American penal system. In 1931, the National Commission on Law Observance and
Enforcement published the fourteen-volume Wickersham Commission Report, which provided
recommendations to improve the ability of our criminal justice system to manage crime and
delinquency.
The President’s Commission of 1967 was followed in 1973 by massive volumes from the National
Advisory Commission on Criminal Justice Standards and Goals on recommendations to improve police,
courts, corrections, and the juvenile justice system. The commission was provided with $1.75 million
through the Law Enforcement Assistance Administration (LEAA), which had been created in the 1960s to
respond to increasing crime, civil and racial distur- bances, and looting during riots. The unrest, both civil
and political, shed doubt on the ability of criminal justice institutions to impose law and order,
rehabilitate and control offenders, and, in general, impose social control in a just and efficient manner.
Because change and innovation were seen as desperately needed throughout the system, millions of
dollars in grants were provided to state and local criminal justice agencies through LEAA to assist them
in making their operations effective and efficient, in the hope that the standards and goals promulgated
by the National Advisory Commission would be implemented.
The effectiveness and practices of the criminal justice system are continually being challenged by society
or the political and legal systems. During the 1980s, successful civil rights litigation in federal courts left
the corrections systems unpro- tected by the traditional “hands-off” policy of federal courts and opened
the prison system to an avalanche of inmates’ rights litigation (Bailey and Hayes, 2006). Passage of the
Violence against Women Act in 1994 and antidrug legislation during the 1980s not only created a more
direct federal response to crime, it also led to significant changes across the components of the criminal
justice system. Police agencies are being restructured through the implementation of foot patrol units or
team polic- ing. The practice of indeterminate sentencing and parole has been challenged by both liberal
and conservative members of the public (Cullen and Gilbert, 1982; Tonry, 2004), and traditional judicial
sentencing practices have been changed dra- matically by legislated sentencing guidelines that are being
challenged by court appeals and pressure groups that oppose the harshness the guidelines imposed on
selected cases. Currently, change is being sought in the juvenile justice system by academics and
practitioners who favor the concept of restorative justice for juvenile offenders (Bazemore and Schiff,
2004). In short, the perceived need for reform within the criminal justice system is an ever-present
constant, and both substantive and symbolic changes have been made over time to meet that need.
During the last decade, private sector services and private prison corporations have sprung up and the
system is still struggling to adapt to the intrusion of market forces into the pri- marily exclusively public
sector operations of criminal justice agencies (Mills et al., 2012; Taxman et al., 2009). Change
manifested in simple agency alterations or in major reforms may be purposive or crescive (Warren,
1977). Crescive change is inadvertent or unplanned, independent of an organization’s control, and may
come about in spite of organizational efforts at self-direction. Crescive change can result from
environmental influences on an organization or from internal organiza- tional conflict (see Chapters 3
and 11 on organizational environments and conflict). As suggested earlier, what we refer to as crescive
change is often imposed upon criminal justice agencies only after a long period of sustained doubt about
what the system should be doing (Henry, 2007). The striking example of crescive change being imposed
upon aspects of law enforcement agencies stems from the 9/11 Commission Report in response to the
destruction of the World Trade Center in New York City on September 11, 2001. The commission
discovered what was well known by law enforcement practitioners and criminal justice academics: Law
enforcement agencies, especially intelligence-gathering agencies, such as the FBI and CIA, kept crucial
information to themselves and had informal policies not to share information with other agencies. The
commission recommended that such practices be immediately altered for the sake of national security.
In addition, the Department of Homeland Security (the Homeland Security Act, 2002) was formed, in
part, to impose a coordinated, intelligence-gathering effort on federal agencies. Also, as a result of the
commission’s recommendations, the Intelligence Reform and Terrorism Prevention Act was passed in
2004 in an effort to create an information- sharing environment for data on terrorism (Henry, 2007).
However, policies, rou- tines, and attitudes of agencies will have to change to meet the new demands.
Purposive change results from conscious, deliberate, and planned efforts by organizational members,
typically managers. It may be a response to changing envi- ronmental conditions or pressures, to
internal conflict, or to organizational members’ perceived needs to change or improve aspects of their
system. Crescive and purpo- sive changes are obviously not mutually exclusive processes. Purposive
change repre- sents an “intervention into a flow of events that will in any case result in change....
Consequently, the decision is not whether or not there will be change, but rather what one’s part will be
in shaping or channeling inexorable change” (Warren, 1977:10). The Patriot Act is a current example of a
mix of crescive and purposive changes and fits into our discussion on the national response to terrorist
threats. It is beyond the scope of this chapter to consider every possible change within an organi- zation.
We are concerned here primarily with purposive, or planned, change as it applies to the agencies of the
criminal justice system. Change can take place at every level of an organization and in every nook and
cranny. We discuss organiza- tional change as a general phenomenon rather than its innumerable
specific applica- tions in every facet of a criminal justice agency or system. As we have seen in moving
from the twentieth century into the twenty-first century, many changes have occurred within criminal
justice organizations. All of these changes have altered, to some degree, how criminal justice agencies
are responding to crime.
WHY CHANGE OCCURS
As we have said, change can emanate from inside or outside an agency’s environ- ment. To the extent
that members are aware of environmental changes or internal conflict that may affect the agency’s
operations or outputs, the agency may enter into deliberate or planned change efforts. Before
executives actively make some form of change effort, however, they must first perceive a need for
change—that “something is broke and needs to be fixed.” When an agency is performing improp- erly or
below capacity, it is suffering a performance gap that may be recognized by agency executives,
personnel, clients, or other constituencies (Downs, 1967:191): “Whenever an official detects some
performance gap between what he is doing and what he believes he ought to be doing, he is motivated
to search for alternative actions satisfactory to him.” Therefore, a change effort will usually be initiated
in an agency only after officials perceive a performance gap or are convinced that a performance gap
exists. The need for change may also be a result of environmental changes, such as changes in public
perceptions of what agencies should be doing or how agencies should carry out their mission or duties.
As discussed in Chapter 3, environmental forces, such as technological changes, demographic changes,
or social or political pressures, may impose on agencies at least a need for change. For agen- cies of the
criminal justice system, their task environments—new legislation, court decisions, and so on—will have
the strongest impact (Henry, 2007) on the need to change. A performance gap may also be produced
internally by employee turnover, internal structural or tech...
Purchase answer to see full
attachment