Unformatted Attachment Preview
FBI's CIO Change Over and Decentralized IT Structure
In reviewing the FBI's failure with VCE, FBI Director Robert Mueller admitted, "We lackes all sets
in our personnel such as qualified software engineering, program management, and contraca
management. We also experienced a high turnovor in Trilogy program managers and che non
mation officers. During the three years of milogy's development, the FBI had five diligent CIOS
as shown in the following table. The high turnoversato created an additional obstacle to dining
obstan
system requirements and setting goals,
CIO
Torm
Bob Dies
November 2001-May 2002
Mark Tanner
May 2002 July 2002
Darwin John
July 2002-May 2003
Wilson Lowery Jr.
May 2003--December 2003
Zalmai Azmi
December 2003-present
In addition, the FBI's IT management was so decentralized that when Zalmai Azmi arrived,
the FBI did not have a centralized budget for IT management As CIO, Azmi managed a whop-
ping budget of $5,800. Azmi instituted measures to centralize IT decision making while finally get-
ting the organization to produce an enterprise architecture. Azmi has also focused on boosting
the skill level of IT personnel
. Suce
In March 2006, after wasting over $100 million and five years, the FBI awarded Lockheed
Martin a $305 million contract to create a new case management system. The project is under
intense scrutiny as Congress, IT professionals, and the media wonder whether Azmi's changes will
be enough to ensure success. Some lessons have been learned. The project is to be deployed
in four phases over six years, and the system will rely in part on commercial, off-the-shelf software.
Yet the question remains: what other lesson does the FBI need to learn before it can obtain the
IT system needed to "connect the dots" and possibly prevent another September 11?
Discussion Questions
1.
2.
3.
Which of the actions recommended in Table 3-7 were not performed on this project?
What decisions did the FBI and SAIC make during development that increased risk? How
can these be avoided in the next project that the FBI takes on?
What changes should the FBI make to its IT organization to improve its ability to handle large
IT projects?
FBI CIO Zalmai Azmi has focused on building IT project management skills within the
agency. Should the FBI consider outsourcing IT project management instead?
4.
Project Manag
FBI's CIO Change Over and Decentralized IT Structure
In reviewing the FBI's failure with VCE, FBI Director Robert Mueller admitted, "We lackes all sets
in our personnel such as qualified software engineering, program management, and contraca
management. We also experienced a high turnovor in Trilogy program managers and che non
mation officers. During the three years of milogy's development, the FBI had five diligent CIOS
as shown in the following table. The high turnoversato created an additional obstacle to dining
obstan
system requirements and setting goals,
CIO
Torm
Bob Dies
November 2001-May 2002
Mark Tanner
May 2002 July 2002
Darwin John
July 2002-May 2003
Wilson Lowery Jr.
May 2003--December 2003
Zalmai Azmi
December 2003-present
In addition, the FBI's IT management was so decentralized that when Zalmai Azmi arrived,
the FBI did not have a centralized budget for IT management As CIO, Azmi managed a whop-
ping budget of $5,800. Azmi instituted measures to centralize IT decision making while finally get-
ting the organization to produce an enterprise architecture. Azmi has also focused on boosting
the skill level of IT personnel
. Suce
In March 2006, after wasting over $100 million and five years, the FBI awarded Lockheed
Martin a $305 million contract to create a new case management system. The project is under
intense scrutiny as Congress, IT professionals, and the media wonder whether Azmi's changes will
be enough to ensure success. Some lessons have been learned. The project is to be deployed
in four phases over six years, and the system will rely in part on commercial, off-the-shelf software.
Yet the question remains: what other lesson does the FBI need to learn before it can obtain the
IT system needed to "connect the dots" and possibly prevent another September 11?
Discussion Questions
1.
2.
3.
Which of the actions recommended in Table 3-7 were not performed on this project?
What decisions did the FBI and SAIC make during development that increased risk? How
can these be avoided in the next project that the FBI takes on?
What changes should the FBI make to its IT organization to improve its ability to handle large
IT projects?
FBI CIO Zalmai Azmi has focused on building IT project management skills within the
agency. Should the FBI consider outsourcing IT project management instead?
4.
Project Manag
92
Contractor Responsibilities
The defining and changing of systems requirements was not the only area in which SAIC tales
to insist that the fel follow sound software engineering practices, SAIC let the Fers inapen
enced IT team make several risky decisions, the worst of which was the mendes les autorer
Rather than implement a phased migration in which parts of the new system could be tested and
repaired until the whole system was deemed fully functional, the Fei planned to suitcairaan
their old ACS system to VCF overnight, with no overlapping period,
University of Pennsylvania Computer Science Professor Matt Blaze was on the National
Research Council (NRC) committee that reviewed the VCF project in 2004. He said the NPC cm-
mittee was horrified when it learned of the planned flash cutover, "1 remember thinking, said
Blaze, "that that would be a very good day for a crime spreers
Former SAIC vice president David Kay said, "SAIC was at fault because of the usual contrase
tor reluctance to tell the customer
, "You're screwed up. You don't know what you're doing
Yet
, SAIC'e mistakes may have gone beyond this problem, In response to the tighter deadlines
and the demanded changes, Sald brought more employees into the project, including. Den bites
and adopted a risky parallel development approach, SAIC personnel working on VCF increased to
250, a possible violation of what is known as Brook's Law. This popular axion states that“Adding
manpower to a late software project makes it later” The reasoning behind the law is that the time spent
communicating to new, inexperienced team members outweighs the time it would have taken to com-
ploto the tasks at hand. In addition, to decrease development time; SAIC di fided its staff into eight,
development teams working in parallel
on different parts of the software. These parts would be inte-
grated in the final phase of development. As later project reviews noted, this approach resulted in a
failure to apply coding standards uniformly across the product.
When the original contract was signed, the Trilogy project was a relatively minor endeavor
for the SAIC, Established in 1969, SAIC today is one of the largest science and technology gov-
ernment contractors. For the FBI alone, SAIC had developed CODIS, a national DNA data-
base used by law enforcement agencies around the world, and NICS, the national criminal
background check system. The IT revolution of the 1990s and the September 11 attacks cre-
ated a boom in the industry, SAIC's business, Income, and staff expanded rapidly just before and
after SAIC was awarded the contract, as shown in the following table.
Year
SAIC Revenue
SAIC Employees
*243,000
20
WWV 862.000
965
1980
$150,894, 000
3,300
TONS
WRO
5709
1990
X1 billion
11,449
1995
Million
17,853
2000
X5.5 billion
39,000
2005
423000
apter 3
The target dates of the Trilogy project's other two components were pushed up as well. The
overall costs of the project spiraled from $379.8 million to $581.1 million.'
Trilogy Costs
Based on Glen A. Fine's statement before the Senate Committee on Appropriations
91
2000
$379.8 million
Early 2002
$458 million
Late 2002
$568.7 million
2003
$581.1 million
In addition, SAIC estimates that it put $3.9 million of its own money into the project.
No Blueprint, Changing Specifications, and Micromanagement
In March 2002, Sherry Higgins, FBI's new director of the newly created Office of Program
Management, asked Special Agent Larry Depew to become the VCF project manager. Depew was
a technophile who had programmed his own case management database to handle an investi-
gation in the early 1990s. Yet, he had no experience in IT project management. He had been one
of a team of seven agents to evaluate SAIC's original Web interface, and had recommended
sacking it.
Depew organized a team of FBI agents to work with SAIC engineers and specify require-
ments for the VCF. For six months, Depew's team of FBI subject experts met with SAIC engi-
neers to define and redefine user needs through a software development process called Joint
Application Development (JAD). But the JAD approach went horribly awry as FBI agents over-
stepped their boundaries, dictating details that should have been decided by experienced
engineers. For example, agents went as far as proposing a design for a portion of the interface.
Depew, who led the sessions, would decide what was inside or outside the scope of the project.
The result was an unwieldy, 800-page system requirement document.
Unfortunately, the FBI's attempts to define and redefine system requirements did not stop
there. As SAIC delivered parts of the product, FBI's team of agents demanded more design
changes in a "we-will-know-it-when-we-see-it" approach. SAIC would then retroactively make
these changes to related parts of the software. Inevitably, this process led to inconsistent imple-
mentation of the altered system requirements.
"This cycle was repeated over and over again and prevented SAIC from defining system
acceptance criteria and suitable test standards," said SAIC Executive Vice President and
General Manager Arnold Punaro. 24
Due to the time pressure created by the September 11 attacks, SAIC agreed to go forward
without an enterprise architecture-a comprehensive blueprint that describes how the current
and future structure of an organization, its IT systems, and its processes align with strategic goals.
The decision turned the project into a high-risk venture.
"Here is where SAIC made honest mistakes," admitted Punaro. "We should have made known
that this approach was too ambitious."
Project Manage
90
focused public attention on the urgent need to overhaul its antiquated information sharing
technology
Four years in the making, VCF was one-third of a larger IT modernization initiative known as
Trilogy. Its other two components consisted of a hardware and software upgrade and the con-
struction of secure LAN and WAN networks. In May 2001, the FBI outsourced these two compo-
nents of Trilogy to DynCorp,
which finally completed the project in April 2004, 22 months late and
$138 million over budget. As a result, the FBI did not have the technological infrastructure in
place to deploy VCF until 2004.
The Department of Justice had deemed the project too large to be handled by one contractor,
so in June 2001, the FBI outsourced the development of VCF to Science Applications International
Corporation (SAIC), a San Diego-based science and technology firm that primarily services U.S.gov.
ernment agencies. In December 2003, after scrambling to meet its deadline for the project, SAIC deliv-
ered 700,000 lines of unworkable code. The FBI rejected it upon delivery.
Over the next few months, the FBI assembled a list of nearly 400 problems. When SAIC
announced that it would fix the problems for an additional cost of $56 million, the FBI refused
the offer. Instead, in June 2004, the agency began testing a scaled-down version of the prod-
uct in its New Orleans field office. The testing checked the VCF's ability to provide electronic
approval for documents uploaded into the old ACS system. The FBI also hired the Aerospace
Corporation for an additional $2 million to review the project. By early 2005, the agency realized
that nothing was salvageable and accepted Aerospace's recommendations to scrap the whole
project.
"There is a long history of failures in large software projects, especially when you're convert-
ing an existing system,” said Steve Bellovin, Professor of Computer Science at Columbia
University. “This is almost a textbook example of how to not do it."22
In answering the question "Where do we go from here?" in his testimony before Congress,
FBI Director Robert Mueller Ill stated that "we will take with us a number of valuable 'lessons
learned.""23 The Aerospace report, as well as several earlier audits and studies, pointed out
exactly what these lessons should be and where the FBI and SAIC went wrong.
Ever-Expanding Scope, Time Crunch, and Spiraling Costs
In June 2001, when the FBI awarded the contract to SAIC, the purpose of the project was to update
and consolidate the ACS and four other important FBI investigative applications. These updates
were intended to allow agents to access these five integrated applications through the Internet.
Following September 11, Mueller recognized that the original scope of the project would not allow
the FBI to fulfill new objectives under its counterterrorism mandate. The FBI, an organization that
collected information after a crime, now had to catch terrorists before the crime. In December
2001, Mueller asked SAIC to abandon the development of the Web interface and begin work on
a new case management system from scratch. The FBI decided not to incorporate off-the-
shelf products, which not only lengthened production time but magnified the number of bugs in the
final product.
In addition, following September 11, the FBI was under the gun to produce the new case man-
agement system as quickly as possible. The FBI abandoned the project's three-year time frame
and moved up final deadlines. The deployment of VCF was divided into two phases, with dead-
lines set for December 2003 and June 2004. With six months of development time already lost to
the Web interface, SAIC had only 22 months to develop a much more extensive product.
Chapter 3
Case Study
The FBI Stumbles Developing a Virtual Case File System
In March 2005, the FBI shelved a $170 million software development project designed to improve
the efficiency and effectiveness of its investigations. The software, called Virtual Case File (VCF),
was to serve three functions: manage investigative records, share and analyze information, and
provide electronic approval for the flow of paperwork. While the FBI already had taken action to
replace its cumbersome and obsolete Automated Case Support (ACS) system with VCF by
2000, the September 11 terrorist attacks exposed the FBI's inability to "connect the dots" and
Project Management