13 ■ ■ ■ Project Evaluation and Control Chapter Outline PROJECT PROFILE New York City’s CityTime Project INTRODUCTION 13.1 CONTROL CYCLES—A GENERAL MODEL 13.2 MONITORING PROJECT PERFORMANCE The Project S-Curve: A Basic Tool S-Curve Drawbacks Milestone Analysis Problems with Milestones The Tracking Gantt Chart Benefits and Drawbacks of Tracking Gantt Charts 13.3 EARNED VALUE MANAGEMENT Terminology for Earned Value Creating Project Baselines Why Use Earned Value? Steps in Earned Value Management Assessing a Project’s Earned Value 13.4 USING EARNED VALUE TO MANAGE A PORTFOLIO OF PROJECTS PROJECT PROFILE Earned Value at Northrop Grumman 13.5 ISSUES IN THE EFFECTIVE USE OF EARNED VALUE MANAGEMENT 13.6 HUMAN FACTORS IN PROJECT EVALUATION AND CONTROL Critical Success Factor Definitions Conclusions Summary Key Terms Solved Problem Discussion Questions Problems Case Study 13.1 The IT Department at Kimble College Case Study 13.2 The Superconducting Supercollider Case Study 13.3 Boeing’s 787 Dreamliner: Failure to Launch Internet Exercises MS Project Exercises PMP Certification Sample Questions Appendix 13.1 Earned Schedule Notes Chapter Objectives After completing this chapter, you should be able to: 1. Understand the nature of the control cycle and four key steps in a general project control model. 2. Recognize the strengths and weaknesses of common project evaluation and control methods. 3. Understand how Earned Value Management can assist project tracking and evaluation. 4. Use Earned Value Management for project portfolio analysis. 5. Understand behavioral concepts and other human issues in evaluation and control. 6. From Appendix 13.1: Understand the advantages of Earned Schedule methods for determining project schedule variance, schedule performance index, and estimates to completion. 431 432 Chapter 13 r Project Evaluation and Control PROJECT MANAGEMENT BODY OF KNOWLEDGE CORE CONCEPTS COVERED IN THIS CHAPTER 1. Control Schedule (PMBoK sec. 6.7) 2. 3. 4. 5. Control Costs (PMBoK sec. 7.4) Earned Value System (PMBoK sec. 7.4.2.1) Forecasting (PMBoK sec. 7.4.2.2) Performance Reviews (PMBoK sec. 7.4.2.4) PROJECT PROFILE New York City’s CityTime Project “… virtually all of the $600 million that the City paid to SAIC for CityTime was tainted, directly or indirectly, by fraud.” In announcing charges filed against several contractors and project overseers for New York City’s troubled CityTime automated payroll and time keeping project, Manhattan U.S. Attorney Preet Bharara placed the blame for the massive cost overruns squarely at the feet of corrupt project managers and their shockingly brazen money skimming practices. In 1998, when former mayor Rudolph Giuliani announced the creation of the CityTime project for New York City’s employees, he was seeking to automate an outdated, paper-based time card and payroll system used by some 300,000 city employees. Over the years, this record-keeping system could not keep up with the large employee base, leading to allegations of waste and routinely falsified time sheets, all of which cost the city millions a year. CityTime was supposed to fix this problem by automating and updating the old paper-based methods with the latest electronic, integrated information system. When the city authorized the project, the winning bid was awarded to Science Applications International Corporation (SAIC) for $63 million over a five-year period. SAIC’s job as prime contractor was to oversee the development of the new computer-based system and support the migration of all record-keeping and payroll processes to the new system. By the time the dust settled, New York City had spent over $720 million on a system that took 10 years to install. Worse, much of the money appears to have been siphoned off by a string of “consultants” and subcontractors, who all used the project to get rich. How did things get so bad? Federal prosecutors have said that nearly the entire sum the city spent on the project was stained by an epic fraud that involved hundreds of contractors, systemic overbilling, and an international moneylaundering conspiracy. FIGURE 13.1 New York City Source: Janniswerner/Fotolia Introduction 433 The project involved numerous stakeholders, all with differing motives and all convinced that others were using the project to push their own agendas. For example, as early as 2003, unions representing city employees were complaining about the new system and the inflated salaries of numerous consultants and officials involved in the project. Still, nothing happened. That was because CityTime had powerful supporters who included the new mayor, Michael Bloomberg; his budget chief Mark Page; and Joel Bondy, the director of the Office of Payroll Administration (OPA). These officials saw the project as a way to control both overtime and pension costs and regarded union complaints as the predictable anger of city employees being stripped of their ability to game the system. Official oversight for this project rested with a three-person panel including Mark Page, a representative for Comptroller Bill Thompson, and Joel Bondy, who led the Office of Payroll Administration up until he was fired. Despite this oversight, Attorney Baharara says the private contractors and consultants were able to manipulate the terms of their contracts and inflate the costs eleven times the original estimate. Their biggest coup was getting the project terms changed from a fixed-price contract to a fixed-price level of effort contract. This meant that in future, the city would be held responsible for all cost overruns, which quickly became enormous. Around this time, allegations of fraud and widespread theft were gaining ground. Mark Mazer, for example, was hired in 2004 to streamline and rescue the project, which had been falling behind its original schedule. He is accused of taking more than $25 million in kickbacks in addition to his $4.4 million salary from the CityTime project. His job? Ironically, Mazur was the outside contractor hired by the city to keep a close eye on the other outside contractors who were performing the work; a classic case of the fox being set to guard the chickens! Meanwhile, the prime CityTime contractor, Science Applications International Corp. (SAIC), received more than $600 million from the city. SAIC’s project chief, Gerard Denault, is accused of taking $9 million in kickbacks. SAIC’s systems engineer, Carl Bell, took more than $5 million and pleaded guilty to doing so. Much of the bribe money flowed through Technodyne, a subcontractor hired by SAIC and headed by an IndianAmerican husband and wife team, Reddy and Padma Allen. SAIC paid $450 million to Technodyne, and the Allens responded with a series of illegal cash payments for Denault, Bell, and Mark Mazer, according to the government. In return, investigators say, these individuals worked together to pad the CityTime payroll and prolong the project. The Allens shipped at least $50 million to companies they controlled in India. Part of it was then wired back to the United States to shell companies controlled by Denault and Bell. Mazer was paid off through a larger web of shell companies. Mazer then funneled millions of dollars stolen from the city to bank accounts overseas. Investigators found hundreds of thousands of dollars stashed in safe-deposit boxes around the city that were part of the ill-gotten gains. Things finally began to unravel in 2010 (over 10 years into the project) when an unhappy CityTime consultant who had been fired went to the city’s Department of Investigation and blew the whistle. The Department of Investigation began its own probe and then notified federal authorities who were brought into the case. Federal indictments were returned in 2010 and the cases were finally brought to trial in 2013. Before he left office, Mayor Michael Bloomberg demanded that SAIC repay the city more than $600 million spent on the scandal-plagued CityTime payroll technology project. In 2012, SAIC came to an agreement with New York City to repay $500 million to avoid federal prosecution. As the U.S. Attorney noted in her indictment, “… the corruption on the CityTime project was epic in duration, magnitude, and scope. As alleged, CityTime served as a vehicle for an unprecedented fraud, which appears to have metastasized over time.” Ironically, a computer system that was developed to make sure that city employees did not cheat on their time card hours resulted in one of the biggest cases of graft, cheating, and outright theft that New York City has ever seen. Project controls and oversight for the project simply did not exist as city officials adopted an attitude that “outside contractors can do it better.” If by “doing it better,” officials meant “are more capable of theft,” then they may have been right. It seems that the echoes of the CityTime scandal are finally starting to fade. Of the 11 people arrested in connection with the fraud and corruption, by 2014, eight had been convicted, the Allens had fled back to India with at least $35 million, and one died prior to the start of his trial. Most recently, a federal judge in Manhattan sentenced three men, including Mark Mazur, to 20 years in prison for their roles in the scandal-ridden project, and he also sharply criticized New York City’s contracting procedures for what he called a lack of “adequate and effective oversight.”1 INTRODUCTION One of the most significant challenges with running a project has to do with maintaining an accurate monitoring and control system for its implementation. Because projects are often defined by their constraints (i.e., budget and schedule limitations), it is vital that we ensure they are controlled as carefully as possible. Project monitoring and control are the principal mechanisms that allow the project team to stay on top of a project’s evolving status as it moves through the various life cycle stages toward completion. Rather than adopting a “no news is good news” approach to monitoring and control of projects, we need to clearly understand the benefits that can be derived from careful and thorough status assessments as the project moves forward. In order to best ensure that the project’s control will be as optimal as possible, we need to focus our attention on two important aspects of the monitoring process. First, we need to identify the 434 Chapter 13 r Project Evaluation and Control appropriate cues that signal project status as well as understand the best times across the project’s life cycle to get accurate assessments of its performance. In other words, we need to be fully aware of the what and when questions: What information concerning the project should be measured, and when are the best times to measure it? Our goal is to have a sense of how to develop systematic project control that is comprehensive, accurate, and timely. Put another way, when we are responsible for a multimillion-dollar investment in our organization, we want to know the status of the project, we want that information as soon as we can get it, and we want it to be as up-to-date as possible. 13.1 CONTROL CYCLES—A GENERAL MODEL A general model of organizational control includes four components that can operate in a continuous cycle and can be represented as a wheel. These elements are: 1. Setting a goal. Project goal setting goes beyond overall scope development to include setting the project baseline plan. The project baseline is predicated on an accurate Work Breakdown Structure (WBS) process. Remember that WBS establishes all the deliverables and work packages associated with the project, assigns the personnel responsible for them, and creates a visual chart of the project from the highest level down through the deliverable and task levels. The project baseline is created as each task is laid out on a network diagram and resources and time durations are assigned to it. 2. Measuring progress. Effective control systems require accurate project measurement mechanisms. Project managers must have a system in place that will allow them to measure the ongoing status of various project activities in real time. We need a measurement system that can provide information as quickly as possible. What to measure also needs to be clearly defined. Any number of devices will allow us to measure one aspect of the project or another; however, the larger question is whether or not we are getting the type of information we can really use. 3. Comparing actual with planned performance. When we have some sense of the original baseline (plan) and a method for accurately measuring progress, the next step is to compare the two pieces of information. A gap analysis can be used as a basis for testing the project’s status. Gap analysis refers to any measurement process that first determines the goals and then the degree to which the actual performance lives up to those goals. The smaller the gaps between planned and actual performance, the better the outcome. In cases where we see obvious differences between what was planned and what was realized, we have a clear-cut warning signal. 4. Taking action. Once we detect significant deviations from the project plan, it becomes necessary to engage in some form of corrective action to minimize or remove the deviation. The process of taking corrective action is generally straightforward. Corrective action can either be relatively minor or involve significant remedial steps. At its most extreme, corrective action may even involve scuttling a nonperforming project. After corrective action, the monitoring and control cycle begins again. As Figure 13.2 demonstrates, the control cycle is continuous. As we create a plan, we begin measurement efforts to chart progress and compare stages against the baseline plan. Any indications of significant deviations from the plan should immediately trigger an appropriate response, 1. Setting a goal 4. Taking action and recycling the process 2. Measuring progress 3. Comparing actual with planned FIGURE 13.2 The Project Control Cycle 13.2 Monitoring Project Performance 435 leading to a reconfiguration of the plan, reassessment of progress, and so on. Project monitoring is a continuous, full-time cycle of target setting, measuring, correcting, improving, and remeasuring. 13.2 MONITORING PROJECT PERFORMANCE As we discovered in the chapters on project budgeting and resource management, once we have established a project baseline budget, one of the most important methods for indicating the ongoing status of the project is to evaluate it against the original budget projections. For project monitoring and control, both individual task budgets and the cumulative project budget are relevant. The cumulative budget can be broken down by time over the project’s projected duration. The Project S-Curve: A Basic Tool As a basis for evaluating project control techniques, let us consider a simple example. Assume a project (Project Sierra) with four work packages (Design, Engineering, Installation, and Testing), a budget to completion of $80,000, and an anticipated duration of 45 weeks. Table 13.1 gives a breakdown of the project’s cumulative budget in terms of both work packages and time. As we discussed in Chapter 8, this type of budget is referred to as a time-phased budget. To determine project performance and status, a straightforward time/cost analysis is often our first choice. Here the project’s status is evaluated as a function of the accumulated costs and labor hours or quantities plotted against time for both budgeted and actual amounts. We can see that time (shown on the x, or horizontal, axis) is compared with money expended (shown on the y, or vertical, axis). The classic project S-curve represents the typical form of such a relationship. Budget expenditures are initially low and ramp up rapidly during the major project execution stage, before starting to level off again as the project gets nearer to its completion (see Figure 13.3). Cumulative budget projections for Project Sierra shown in Table 13.1 have been plotted against the project’s schedule. The S-curve figure represents the project budget baseline against which actual budget expenditures are evaluated. Monitoring the status of a project using S-curves becomes a simple tracking problem. At the conclusion of each given time period (week, month, or quarter), we simply total the cumulative project budget expenditures to date and compare them with the anticipated spending patterns. Any significant deviations between actual and planned budget spending reveal a potential problem area. Simplicity is the key benefit of S-curve analysis. Because the projected project baseline is established in advance, the only additional data shown are the actual project budget expenditures. The S-curve also provides real-time tracking information in that budget expenditures can be constantly updated and the new values plotted on the graph. Project information can be visualized immediately and updated continuously, so S-curves offer an easy-to-read evaluation of the project’s status in a timely manner. (The information is not necessarily easily interpreted, however, as we shall see later.) Our Project Sierra example (whose budget is shown in Table 13.1) can also be used to illustrate how S-curve analysis is employed. Suppose that by week 21 in the project, the original budget projected expenditures of $50,000. However, our actual project expenditures totaled only $40,000. TABLE 13.1 Budgeted Costs for Project Sierra (in thousands $) Duration (in weeks) Design 5 10 6 2 Engineer 4 15 20 25 8 8 8 4 20 Install Test 30 35 40 45 2 6 4 2 Total 6 Total 6 6 8 12 28 8 6 4 2 Cumul. 6 12 20 32 60 68 74 78 80 80 436 Chapter 13 r Project Evaluation and Control Cumulative Cost ($ in thousands) 80 60 40 20 5 10 15 20 25 30 35 40 45 Elapsed Time (in weeks) FIGURE 13.3 Project S-Curves In effect, there is a $10,000 budget shortfall, or negative variance between the cumulative budgeted cost of the project and its cumulative actual cost. Figure 13.4 shows the tracking of budgeted expenditures with actual project costs, including identifying the negative variance shown at week 21. In this illustration, we see the value of S-curve analysis as a good visual method for linking project costs (both budgeted and actual) over the project’s schedule. S-Curve Drawbacks When project teams consider using S-curves, they need to take the curves’ significant drawbacks as well as their strengths into consideration. S-curves can identify positive or negative variance (budget expenditures above or below projections), but they do not allow us to make reasonable Cumulative Cost ($ in thousands) 80 60 $10,000 Negative Var. 40 20 5 10 15 20 25 30 35 40 Elapsed Time (in weeks) Cumulative Budgeted Cost Cumulative Actual Cost FIGURE 13.4 Project Sierra’s S-Curve Showing Negative Variance 45 13.2 Monitoring Project Performance 437 interpretations as to the cause of variance. Consider the S-curve shown in Figure 13.4. The actual budget expenditures have been plotted to suggest that the project team has not spent the total planned budget money to date (there is negative variance). However, the question is how to interpret this finding. The link between accumulated project costs and time is not always easily resolved. Is the project team behind schedule (given that they have not spent sufficient budget to date) or might there be alternative reasons for the negative variance? Assume that your organization tracks project costs employing an S-curve approach and uses that information to assess the status of an ongoing project. Also assume that the project is to be completed in 12 months and has a budget of $150,000. At the six-month checkup, you discover that the project S-curve shows significant shortfall; you have spent far less on the project to date than was originally budgeted. Is this good or bad news? On the surface, we might suppose that this is a sign of poor performance; we are lagging far behind in bringing the project along and the smaller amount we have spent to date is evidence that our project is behind schedule. On the other hand, there are any number of reasons why this circumstance actually might be positive. For example, suppose that in running the project, you found a cost-effective method for doing some component of the work or came across a new technology that significantly cut down on expenses. In that case, the time/cost metric may not only be misused, but might lead to dramatically inaccurate conclusions. Likewise, positive variance is not always a sign of project progress. In fact, a team may have a serious problem with overexpenditures that could be interpreted as strong progress on the project when in reality it signals nothing more than their inefficient use of project capital resources. The bottom line is this: Simply evaluating a project’s status according to its performance on time versus budget expenditures may easily lead us into making inaccurate assumptions about project performance. Another drawback with using S-curves to update a project’s progress is that they are providing “reactive” data; that is, we plot the results of expenditures after the money has already been spent. S-curves are a tracking tool that are visually appealing but do not allow the project team to anticipate or take proactive actions because the information only arrives after the fact. Likewise, S-curves do not allow the team to forecast project expenditures or other performance metrics to completion. We know how much we have spent to date, but we only have the initial budget to suggest how much we should have spent and how much we are likely to spend in the future. Once significant variances start appearing in the budget, the question of what our final, likely project costs will be is extremely difficult to determine. Milestone Analysis Another method for monitoring project progress is milestone analysis. A milestone is an event or stage of the project that represents a significant accomplishment on the road to the project’s completion. Completion of a deliverable (a combination of multiple project tasks), an important activity on the project’s critical path, or even a calendar date can all be milestones. In effect, milestones are road markers that we observe on our travels along the project’s life cycle. There are several benefits to using milestones as a form of project control. 1. Milestones signal the completion of important project steps. A project’s milestones are an important indicator of the current status of the project under development. They give the project team a common language to use in discussing the ongoing status of the project. 2. Milestones can motivate the project team. In large projects lasting several years, motivation can flag as team members begin to have difficulty seeing how the project is proceeding overall, what their specific contribution has been and continues to be, and how much longer the project is likely to take. Focusing attention on milestones helps team members become more aware of the project’s successes as well as its status, and they can begin to develop greater task identity regarding their work on the project. 3. Milestones offer points at which to reevaluate client needs and any potential change requests. A common problem with many types of projects is the nature of repetitive and constant change requests from clients. Using project review milestones as formal “stop points,” both the project team and the clients are clear on when they will take midcourse reviews of the project and how change requests will be handled. When clients are aware of these formal project review points, they are better able to present reasonable and well-considered feedback (and specification change requests) to the team. 438 Chapter 13 r Project Evaluation and Control 4. Milestones help coordinate schedules with vendors and suppliers. Creating delivery dates that do not delay project activities is a common challenge in scheduling delivery of key project components. From a resource perspective, the project team needs to receive supplies before they are needed but not so far in advance that space limitations, holding and inventory costs, and in some cases spoilage are problems. Hence, to balance delays of late shipments against the costs associated with holding early deliveries, a well-considered system of milestones creates a scheduling and coordinating mechanism that identifies the key dates when supplies will be needed. 5. Milestones identify key project review gates. For many complex projects, a series of midterm project reviews are mandatory. For example, many projects that are developed for the U.S. government require periodic evaluation as a precondition to the project firm receiving some percentage of the contract award. Milestones allow for appropriate points for these reviews. Sometimes the logic behind when to hold such reviews is based on nothing more than the passage of time (“It is time for the July 1 review”). For other projects, the review gates are determined based on completion of a series of key project steps (such as the evaluation of software results from the beta sites). 6. Milestones signal other team members when their participation is expected to begin. Many times projects require contributions from personnel who are not part of the project team. For example, a quality assurance individual may be needed to conduct systems tests or quality inspection and evaluations of work done to date. If the quality supervisor does not know when to assign a person to our project, we may find when we reach that milestone that no one is available to help us. Because the quality assurance person is not part of the project team, we need to coordinate her involvement in order to minimize disruption to the project schedule. 7. Milestones can delineate the various deliverables developed in the Work Breakdown Structure and thereby enable the project team to develop a better overall view of the project. We then are able to refocus efforts and function-specific resources toward the deliverables that show signs of trouble, rather than simply allocating resources in a general manner. For example, indications that the initial project software programming milestone has been missed allow the project manager to specifically request additional programmers downstream, in order to make up time later in the project’s development. Figure 13.5 gives an example of a simple Gantt chart with milestones included. The milestones in this case are simply arbitrary points established on the chart; we could just as easily have placed them after completed work packages or by using some other criteria. Problems with Milestones Milestones, in one form or another, are probably the simplest and most widely used of all project control devices. Their benefits lie in their clarity; it is usually easy for all project team members to relate to the idea of milestones as a project performance metric. The problem with them is that, like FIGURE 13.5 Gantt Chart with Milestones Source: MS Project 2013, Microsoft Corporation 13.2 Monitoring Project Performance 439 S-curves, they are a reactive control system. You must first engage in project activities and then evaluate them relative to your goal. If you significantly underperform your work to that point, you are faced with having to correct what has already transpired. Imagine, for example, that a project team misses a milestone by a large margin. Not having received any progress reports until the point that the bad news becomes public, the project manager is probably not in a position to craft an immediate remedy for the shortfall. At this point, the problems are compounded. Due to the delay in receiving the bad news, remedial steps are themselves delayed, pushing the project even farther behind. The Tracking Gantt Chart One form of the Gantt chart, referred to as a tracking Gantt chart, is useful for evaluating project performance at specific points in time. The tracking Gantt chart allows the project team to constantly update the project’s status by linking task completion to the schedule baseline. Rather than monitor costs and budget expenditures, a tracking Gantt chart identifies the stage of completion each task has attained by a specific date within the project. For example, Figure 13.6 represents Project Blue, involving five activities. As the project progresses, its current status is indicated by the vertical status bar shown for Thursday, July 24. To date, activity A (Licensing Agreement) has been 100% completed, while its two subsequent tasks, Specification Design and Site Certification, are shown as having progressed proportionally by the identified tracking date. That is, activity B (Specification Design) is rated as 57% completed, and activity C (Site Certification) as 80% completed. Activities D and E have not yet begun in this example. It is also possible to measure both positive and negative deviations from the schedule baseline with the tracking Gantt chart. Let us suppose, using our Project Blue example, that activity B remains approximately 57% completed as of the baseline date indicated. On the other hand, activity C has not progressed as rapidly and is only 20% completed as of the July 24 date. The chart can be configured to identify the variations, either positive or negative, in activity completion against the project baseline. These features are demonstrated in Figure 13.7, showing the current date for the project and the delay in progress on activity C. FIGURE 13.6 Assessing Project Blue’s Status Using Tracking Gantt Chart Source: MS Project 2013, Microsoft Corporation FIGURE 13.7 Tracking Gantt with Project Activity Deviation Source: MS Project 2013, Microsoft Corporation 440 Chapter 13 r Project Evaluation and Control Benefits and Drawbacks of Tracking Gantt Charts A key benefit of tracking Gantt charts is that they are quite easy to understand. The visual nature of the feedback report is easy to assimilate and interpret. This type of control chart can be updated very quickly, providing a sense of real-time project control. On the other hand, tracking Gantt charts have some inherent drawbacks that limit their overall utility. First, although they may show which tasks are ahead of schedule, on schedule, and behind schedule, these charts do not identify the underlying source of problems in the cases of task slippage. Reasons for schedule slippage cannot be inferred from the data presented. Second, tracking control charts do not allow for future projections of the project’s status. It is difficult to accurately estimate the time to completion for a project, particularly in the case of significant positive or negative variation from the baseline schedule. Is a series of early finishes for some activities good news? Does that signal that the project is likely to finish earlier than estimated? Because of these drawbacks, tracking charts should be used along with other techniques that offer more prescriptive power. 13.3 EARNED VALUE MANAGEMENT An increasingly popular method used in project monitoring and control consists of a mechanism that has become known as Earned Value Management (EVM).* The origins of EVM date to the 1960s when U.S. government contracting agencies began to question the ability of contractors to accurately track their costs across the life of various projects. As a result, after 1967, the Department of Defense imposed 35 Cost/Schedule Control Systems Criteria that suggested, in effect, that any future projects procured by the U.S. government in which the risk of cost growth was to be retained by the government must satisfy these 35 criteria.2 In the more than four years since its origin, EVM has been practiced in multiple settings, by agencies from governments as diverse as Australia, Canada, and Sweden, as well as by a host of project-based firms in numerous industries. Unlike previous project tracking approaches, EVM recognizes that it is necessary to jointly consider the impact of time, cost, and project performance on any analysis of current project status. Put another way: Any monitoring system that only compares actual against budgeted cost numbers ignores the fact that the client is spending that money to accomplish something—to create a project. Therefore, EVM reintroduces and stresses the importance of analyzing the time element in project status updates. Time is important because it becomes the basis for determining how much work should be accomplished at certain milestone points. EVM also allows the project team to make future projections of project status based on its current state. At any point in the project’s development, we are able to calculate both schedule and budget efficiency factors (the efficiency with which budget is being used relative to the value that is being created) and use those values to make future projections about the estimated cost and schedule to project completion. We can illustrate the advance in the project control process that Earned Value Management represents by comparing it to the other project tracking mechanisms. If we consider the key metrics of project performance as those success criteria discussed in Chapter 1 (schedule, budget, and performance), most project evaluation approaches tend to isolate some subset of the overall success measure. For example, project S-curve analysis directly links budget expenditures with the project schedule (see Figure 13.8). Again, the obvious disadvantage to this approach is that it ignores the project performance linkage. Project control charts such as tracking Gantt charts link project performance with schedule but may give budget expenditures short shrift (see Figure 13.9). The essence of a tracking approach to project status is to emphasize project performance over time. Although the argument * Note that Earned Value Management (EVM) is used interchangeably with Earned Value Analysis (EVA). EVA is an older term, though still widely in use. EVM has become increasingly common and is used within many project firms. 13.3 Earned Value Management 441 Cost Project S-Curves Performance FIGURE 13.8 Monitoring Project Performance (S-Curve Analysis) FIGURE 13.9 Monitoring Project Performance (Control Charting) Schedule Cost Performance Schedule Tracking Control Charts (e.g., Gantt Charts) Cost Earned Value FIGURE 13.10 Performance Schedule Monitoring Project Performance (Earned Value) could be made that budget is implicitly assumed to be spent in some preconceived fashion, this metric does not directly apply a link between the use of time and performance factors with project cost. Earned value (EV), on the other hand, directly links all three primary project success metrics (cost, schedule, and performance). This methodology is extremely valuable because it allows for regular updating of a time-phased budget to determine schedule and cost variances, as identified by the regular measurement of project performance (see Figure 13.10). Terminology for Earned Value Following are some of the key concepts that allow us to calculate earned value and use its figures to make future project performance projections. Many of these definitions are taken from the Project Management Institute’s Project Management Body of Knowledge (PMBoK), 5th Edition. PV EV Planned value. The authorized budget assigned to scheduled work. At any given moment, planned value defines the physical work that should have been accomplished to that point in time. It can also be thought of as a cost estimate of the budgeted resources scheduled across the project’s life cycle (cumulative baseline). In older terminology, PV used to be referred to as BCWS (Budgeted Cost of Work Scheduled). Earned value. This is a measure of the work performed expressed in terms of the budget authorized for that work. This is the real budgeted cost, or “value,” of the work that has actually been performed to date. In older terminology, EV used to be referred to as Budgeted Cost of Work Performed (BCWP). 442 Chapter 13 r Project Evaluation and Control AC SV CV SPI CPI BAC EAC Actual cost of work performed. This is the realized cost for the work performed on an activity during a specific time period. It is the cumulative total costs incurred in accomplishing the various project work packages that EV measured. In older terminology, AC used to be referred to as Actual Cost of Work Performed (ACWP). Schedule variance. This is a measure of schedule performance expressed as the difference between the earned value and the planned value, or EV – PV. It is the amount by which the project is ahead or behind the delivery date at a given point in time. Cost variance. This is a measure of cost performance expressed as the difference between the earned value and the actual cost of work performed, or EV – AC. It is the amount of budget deficit or surplus at a given point in time. Schedule Performance Index. The rate at which project performance is meeting schedule expectations up to a point in time. SPI is expressed as the earned value to date divided by the planned value of work scheduled to be performed (EV/PV). This value allows us to calculate the projected schedule of the project to completion. Cost Performance Index. The rate at which project performance is meeting cost expectations during a given period of time. CPI is expressed as the earned value divided by the actual, cumulative cost of the work performed to date (EV/AC). This value allows us to calculate the projected budget to completion. Budgeted cost at completion. This value represents the total budget for a project. Estimate at completion. The expected total cost of completing all work on the project. This is the projected (forecasted) total cost based on project performance to that point in time. It is represented as the sum of actual costs (AC) plus an estimate to complete all remaining work. Creating Project Baselines The first step in developing an accurate control process is to create the project baselines against which progress can be measured. Baseline information is critical regardless of the control process we employ, but baselines are elemental when performing EVM. The first piece of information necessary for performing earned value is the planned value, that is, the project baseline. The PV should comprise all relevant project costs, the most important of which are personnel costs, equipment and materials, and project overhead, sometimes referred to as level of effort. Overhead costs (level of effort) can include a variety of fixed costs that must be included in the project budget, including administrative or technical support, computer work, and other staff expertise (such as legal advice or marketing). The actual steps in establishing the project baseline are fairly straightforward and require two pieces of data: the Work Breakdown Structure and a time-phased project budget. 1. The Work Breakdown Structure identified the individual work packages and tasks necessary to accomplish the project. As such, the WBS allowed us to first identify the individual tasks that would need to be performed. It also gave us some understanding of the hierarchy of tasks needed to set up work packages and identify personnel needs (human resources) in order to match the task requirements to the correct individuals capable of performing them. 2. The time-phased budget takes the WBS one step further: It allows us to identify the correct sequencing of tasks, but more importantly, it enables the project team to determine the points in the project when budget money is likely to be spent in pursuit of those tasks. Say, for example, that our project team determines that one project activity, Data Entry, will require a budget of $20,000 to be completed, and further, that the task is estimated to require two months to completion, with the majority of the work being done in the first month. A time-phased budget for this activity might resemble the following: Activity Jan Feb Data Entry $14,000 $6,000 ... Dec Total -0- $20,000 Once we have collected the WBS and applied a time-phased budget breakdown, we can create the project baseline. The result is an important component of earned value because it represents the standard against which we are going to compare all project performance, cost, and schedule 13.3 Earned Value Management 443 TABLE 13.2 Percentage of Tasks Completed for Project Sierra (in thousands $) Duration (in weeks) Design 5 10 6 2 Engineer 4 15 20 25 30 35 40 45 % Comp. 100 8 Install 8 8 4 20 Test 100 6 50 2 6 4 2 Total 6 6 8 12 28 8 6 4 2 Cumul. 6 12 20 32 60 68 74 78 80 0 data as we attempt to assess the viability of an ongoing project. This baseline, then, represents our best understanding of how the project should progress. How the project is actually doing, is another matter. Why Use Earned Value? Let us illustrate the relevancy of EVM using our Project Sierra example. Return to the information presented in Table 13.1, as graphically represented on the project S-curve in Figure 13.3. Assume that it is now week 30 of the project and we are attempting to assess the project’s status. Also assume that there is no difference between the projected project costs and actual expenditures; that is, the project budget is being spent within the correct time frame. However, upon examination, suppose we were to discover that Installation was only half completed and Project Testing had not yet begun. This example illustrates both a problem with S-curve analysis and the strength of EVM. Project status assessment is relevant only when some measure of performance is considered in addition to budget and elapsed schedule. Consider the revised data for Project Sierra shown in Table 13.2. Note that as of week 30, work packages related to Design and Engineering have been totally completed, whereas the Installation is only 50% done, and Testing has not yet begun. These percentage values are given based on the project team or key individual’s assessment of the current status of work package completion. The question now is: What is the earned value of the project work done to date? As of week 30, what is the status of this project in terms of budget, schedule, and performance? Calculating the earned value for these work packages is a relatively straightforward process. As Table 13.3 shows, we can modify the previous table to focus exclusively on the relevant information for determining earned value as of week 30. The planned budget for each work package is multiplied by the percentage completed in order to determine the earned value to date for the work packages, as well as for the overall project. In this case, the earned value at the 30-week point is $51,000. Now we can compare the planned budget against the actual earned value using the original project budget baseline, shown in Figure 13.11. This process allows us to assess a more realistic determination of the status of the project when the earned value is plotted against the budget baseline. Compare this figure with the alternative method from Figure 13.4, in which a negative TABLE 13.3 Calculating Earned Value (in thousands $) Planned % Comp. Earned Value 8 100 8 Engineer 28 100 28 Install 30 50 15 Test 14 0 0 Design Cumul. Earned Value 51 444 Chapter 13 r Project Evaluation and Control Cumulative Cost ($ in thousands) 80 Project Baseline 60 Earned Value 40 20 0 FIGURE 13.11 5 10 15 20 25 30 35 Elapsed Time (in weeks) 40 45 Project Baseline, Using Earned Value variance is calculated, with no supporting explanation as to the cause or any indication about whether this figure is meaningful or not. Recall that by the end of week 30, our original budget projections suggested that $68,000 should have been spent. Instead, we are projecting a shortfall of $17,000. In other words, we are showing a negative variance not only in terms of money spent on the project, but also in terms of value created (performance) of the project to date. Unlike the standard S-curve evaluation, EVM variance is meaningful because it is based not simply on budget spent, but value earned. A negative variance of $10,000 in budget expenditures may or may not signal cause for concern; however, a $17,000 shortfall in value earned on the project to date represents a variance of serious consequences. Steps in Earned Value Management There are five steps in Earned Value Management (EVM): 1. Clearly define each activity or task that will be performed on the project, including its resource needs as well as a detailed budget. As we demonstrated earlier, the Work Breakdown Structure allows project teams to identify all necessary project tasks. It further allows for each task to be assigned its own project resources, including equipment and materials costs, as well as personnel assignments. Finally, coupled with the task breakdowns and resource assignments, it is possible to create the budget figure or cost estimate for each project task. 2. Create the activity and resource usage schedules. These will identify the proportion of the total budget allocated to each task across a project calendar. Determine how much of an activity’s budget is to be spent each month (or other appropriate time period) across the project’s projected development cycle. Coupled with the development of a project budget should be its direct linkage to the project schedule. The determination of how much budget money is to be allocated to project tasks is important. Equally important is the understanding of when the resources are to be employed across the project’s development cycle. 3. Develop a “time-phased” budget that shows expenditures across the project’s life. The total (cumulative) amount of the budget becomes the project baseline and is referred to as the planned value (PV). In real terms, PV just means that we can identify the cumulative budget expenditures planned at any stage in the project’s life. The PV, as a cumulative value, is derived from adding the planned budget expenditures for each preceding time period. 13.3 Earned Value Management 445 Project Budget Management Reserve Estimate at Completion (EAC) Cumulative Cost Budget at Completion (BAC) Planned Value (PV) Actual Cost (AC) Earned Value (EV) Assessment Date Time FIGURE 13.12 Scheduled Completion Date Earned Value Milestones 4. Total the actual costs of doing each task to arrive at the actual cost of work performed (AC). We can also compute the budgeted values for the tasks on which work is being performed. This is referred to as the earned value (EV) and is the origin of the term for this control process. 5. Calculate both a project’s budget variance and schedule variance while it is still in process. Once we have collected the three key pieces of data (PV, EV, and AC), it is possible to make these calculations. Remember from earlier in the chapter that the schedule variance is calculated by the simple equation SV = EV - PV, or the difference between the earned value to date minus the planned value of the work scheduled to be performed to date. The budget, or cost, variance is calculated as CV = EV - AC, or the earned value minus the actual cost of work performed. A simplified model that fits the principal elements of earned value together (PV, EV, AC, BAC, and EAC) is shown in Figure 13.12. The original baseline data, comprising both schedule and budget for all project tasks, is indicated by the planned value (PV) line and budget at completion (BAC) estimate for the project. Notice that PV follows a standard S-curve outline. Actual costs at the time of this assessment (when the calculations are being made) are shown on the AC line, which has been steadily tracking above the planned value to date. Earned value is illustrated as a line below the PV baseline, suggesting that the project’s current earned value is below expectations. The dotted line represents the forecast for project performance to completion (EAC). Note that the line is tracking well above the project’s planned value, suggesting that based on current performance, which drives projections to completion, the project is likely to finish over budget and past the scheduled completion date. We will see how these EVM calculations and forecasted projections to completion are actually calculated in the next section. Assessing a Project’s Earned Value Table 13.4 presents the first components of a calculated earned value analysis on Project Mercury.3 This project has a planned seven-month duration and a $118,000 budget. The project began in January and we are interested in calculating its earned value as of the end of June. For simplicity’s sake, the total work packages for this project are only seven in number. If we know the amount budgeted for each work package and when that work is slated to be done, we can construct a budget table similar to that shown in Table 13.4. Notice that each work package has a fixed budget across a number of time periods (e.g., Staffing is budgeted to cost $15,000 and is to be performed almost equally across the months of January and February, while Blueprinting begins in March, with $4,000 budgeted to be spent, and concludes in April with $6,000). 446 Chapter 13 r Project Evaluation and Control TABLE 13.4 Earned Value Table (end of June) with $6,000 for Project Mercury (in thousands $) Activity Jan Feb Staffing 8 7 Mar Apr May June July Plan % Comp. Value 15 100 15 Blueprinting 4 6 10 80 8 Prototype Development 2 8 10 60 6 Full Design 3 Construction 8 10 21 33 7 2 30 32 25 8 10 10 0 0 5 20 0 0 Transfer Punch List 15 118 ©= Monthly Plan 8 7 6 17 10 55 15 Cumulative 8 15 21 38 48 103 118 Monthly Actual 8 11 8 11 10 30 0 Cumulative Actual 8 19 27 38 48 78 44 If we plot the expenses across each month of the project completed to date (January through June), we find that we can determine the amount budgeted and, through gathering some information from the project team and the accountant, the actual amount spent each month. These sets of figures are added to the bottom four rows of the table. For example, note that by March, we had planned to spend $21,000 in project budget on activities to date. Our actual cumulative costs were $27,000. The obvious question is: Is this good news or bad news? On the surface, we might conclude that it is bad news because we have overspent our budget. However, recall that the chief problem with S-curve methodology is that it only considers actual costs versus planned costs. This simply is not sufficient information for us to make any real determination of the status of the project. The key pieces of information that allow us to identify earned value are included in the righthand columns. We are very interested in determining the current status of the project based on the number of tasks completed over the time budgeted to them. Therefore, the last columns show the planned expenditures for each task, the percentage of the tasks completed, and the calculated value. Value in this sense is simply the product of the planned expenditures and the percentage of these tasks completed. For example, under the work package Blueprinting, we see that this activity was given a planned budget of $10,000 across two months total. To date, 80% of that activity has been completed, resulting in $8,000 in value. If we total the columns for planned expenditures and actual value (EV), we come up with our project’s planned budget ($118,000) and the value realized at the end of June ($44,000). We now have enough information to make a reasonable determination of the project’s status using Earned Value Management. The first number we require is the planned value (PV). This value can be found as the cumulative planned costs at the end of the month of June ($103,000). We also have calculated that the earned value for the project to date (EV) totals $44,000. The schedule variances that are of interest to us are the Schedule Performance Index (SPI) and the estimated time to completion. The SPI is determined by dividing the EV by the PV. Table 13.5 shows this calculation ($44,000/103,000 = .43). With the SPI, we can now project the length of time it should take to complete the project. Because the SPI is telling us that we are operating at only 43% efficiency in implementing the project, we take the reciprocal of the SPI multiplied by the original project schedule to determine the projected actual time frame to completion for the project (1/.43 * 7 = 16.3 months). The bad news is: It appears that as of June, we cannot expect to complete this project for an additional 10 months; we are running more than nine months behind schedule. How about costs? Although we are running more than nine months late, can we make similar projections about the project in terms of how much it is projected to finally cost? The answer, according to EVM, is yes. Just as we can determine schedule variances, we can also compute cost 13.3 Earned Value Management 447 TABLE 13.5 Schedule Variances for Project Mercury EVM Schedule Variances Planned Value (PV) 103 Earned Value (EV) 44 Schedule Performance Index EV/PV = 44/103 = .43 Estimated Time to Completion (1/.43 * 7) = 16.3 months TABLE 13.6 Cost Variances for Project Mercury EVM Cost Variances Cumulative Actual Cost of Work Performed (AC) 78 Earned Value (EV) 44 Cost Performance Index EV/AC = 44/78 = .56 Estimated Cumulative Cost to Completion (1/.56 * $118,000) = $210,714 variances, as long as we have two very important pieces of data—the cumulative actual cost of work performed (AC) and the earned value (EV). The earned value figure has already been calculated ($44,000), and now we turn back to Table 13.4 to locate the AC. The cumulative actual cost at the end of June is $78,000. This figure is our AC and is entered into Table 13.6. As we did in calculating schedule variance, we calculate cost variance by dividing the EV by AC, or $44,000/78,000 = .56. That is the Cost Performance Index (CPI) for this project. Determining the projected cost of the project involves taking the reciprocal of the CPI multiplied by the original project budget ($118,000). The bad news is: Not only is this project well behind schedule, but it also is projected to end up costing more than $210,000, a significant cost overrun. Finally, we can plot these variance values graphically, showing the difference between EV (earned value) and PV and AC (see Figure 13.13). The intriguing result of this example suggests how misleading simple S-curves can sometimes be. For example, in this case, we have discovered a difference at the end of June of $25,000 between the AC ($78,000) and PV ($103,000). Although the analysis at that point showed that we had underspent our budget slightly, the results were actually more serious when viewed from the perspective of earned value by the end of June ($44,000). In reality, the schedule and cost variances were much more severe due to the lag in earned value on PV 100 90 Budget in Thousands ($) 80 AC 70 60 50 EV 40 30 Slippage 20 10 0 April FIGURE 13.13 May June Earned Value Variances for Project Mercury 448 Chapter 13 r Project Evaluation and Control TABLE 13.7 Calculating Cumulative CPI for Trend Analysis EV EVC AC ACC CPI CPIC July $27,500 $ 27,500 $20,000 $ 20,000 1.38 1.38 August $58,000 $ 85,500 $62,000 $ 82,000 0.94 1.04 September $74,500 $160,000 $69,000 $151,000 1.08 1.06 October $40,000 $200,000 $35,500 $186,500 1.13 1.07 TABLE 13.8 Calculating Cumulative SPI for Trend Analysis EV EVC PV PVC SPI SPIC July $27,500 $ 27,500 $30,000 $ 30,000 0.92 0.92 August $58,000 $ 85,500 $60,500 $ 90,500 0.96 0.94 September $74,500 $160,000 $75,000 $165,500 0.99 0.97 October $40,000 $200,000 $37,500 $203,000 1.07 0.99 the project, as calculated by the percentage completion of all scheduled tasks. This example clearly shows the advantages of earned value for more accurately determining actual project status as a function of its three component pieces: time, budget, and completion. Earned value can be employed to measure trends in project performance. Trends are important because we want more than simple, one-time assessments of our project’s status; we want to be able to determine whether CPI and SPI are trending up, down, or remaining stationary. So, for example, it is possible to develop cumulative values for CPI and SPI as follows: Cumulative CPI = Cumulative EV/Cumulative AC, or: CPIC = EVC/ACC Likewise, Cumulative SPI = Cumulative EV/Cumulative PV, or: SPIC = EVC/PVC Let us see how these values can be derived in an actual example. Suppose we collected cost data for our project over four months as shown in Table 13.7. Our results suggest that after July, CPI dropped significantly for the month of August, before trending upwards in each of the final two months. Further, cumulative CPI (CPIC) continued to track above the threshold 1.0 level and in recent months has shown a steady positive trend. Now, using the same data, let’s develop the cumulative SPI (SPIC) table for this example. Suppose that our data is as shown in Table 13.8. We can see from the calculation of SPIC that the project’s schedule performance has been steadily improving and by the end of this four-month segment, has improved from 0.92 to nearly 1.0. Cumulative SPI and CPI values are important for updating overall project performance to a master EVM report, rather than relying on a series of discrete “snapshots” of the project at different points in time. We can also perform Earned Value Management using MS Project 2013. Suppose that we wished to track Project Atlas, shown in Figure 13.14. Notice that as of August 14, the project is beginning to show some signs of delay. By this point, we should have completed four of the six work packages, and yet Testing, for which Stewart is responsible, is only now getting under way. From a monitoring and control perspective, the question we want to answer is: How does EVM indicate the potential delays in our project? Suppose that, in addition to regularly updating the baseline schedule, we have been tracking the costs associated with each of the work packages and have found, as Figure 13.15 shows, that we have spent all budgeted money allotted to the work packages of Design, Engineering, and 13.3 Earned Value Management 449 FIGURE 13.14 Sample Gantt Chart for Project Atlas Showing Status on August 14 Source: MS Project 2013, Microsoft Corporation FIGURE 13.15 Sample Cost Report for Project Atlas on August 14 Source: MS Project 2013, Microsoft Corporation Supplier Qualification. We have only spent $288 of our Testing budget. These are the actual cost values (AC) for these activities. We now have sufficient updated information to determine the earned value for Project Atlas as of August 14. Figure 13.16 shows an example of an earned value report generated by MS Project 2013 for our Project Atlas.* In addition to providing the key metrics of PV, EV, and AC (see footnote), the report generates both schedule and cost variances. Schedule variance (SV) is simply the difference between earned value and planned value, while cost variance (CV) is the difference between earned value and actual cost. The Estimate at Completion (EAC) column shows the expected total cost of the project to completion based on performance across the various tasks up to the status date. Note that for Project Atlas, we are currently projecting schedule and cost variances, suggesting that our project is over budget and behind schedule. In fact, the EAC demonstrates that based on the progress made by August 14, this project is expected to cost $7,180 to completion. * MS Project 2013 uses the term BCWS (Budgeted Cost of Work Scheduled) for planned value (PV), BCWP (Budgeted Cost of Work Performed) for earned value (EV), and ACWP (Actual Cost of Work Performed) for actual cost (AC). As Figure 13.16 demonstrates, MS Project 2013 employs older terms in conjunction with the newer terminology that has been updated by the Project Management Institute’s PMBoK. 450 Chapter 13 r Project Evaluation and Control FIGURE 13.16 Earned Value Report for Project Atlas on August 14 Source: MS Project 2013, Microsoft Corporation 13.4 USING EARNED VALUE TO MANAGE A PORTFOLIO OF PROJECTS Earned Value Management can work at the portfolio level as well as with individual projects. The process simply involves the aggregation of all earned value measures across the firm’s entire project portfolio in order to give an indication as to the efficiency with which a company is managing its projects. Table 13.9 gives an example of a portfolio-level Earned Value Management control table that identifies both positive and negative cost and schedule variances and, based on these evaluations, projects the cost to completion of each current project.4 Other useful information contained in the Portfolio Earned Value Management table includes the total positive variances for both budget and schedule, as well as a determination of the relative schedule and cost variances as a percentage of the total project portfolio. In the example shown in Table 13.7, the company is running average cost and schedule variances on its projects of 7.34% and 6.84%, respectively. The use of Earned Value Management for portfolio tracking and control offers top management an excellent window into the firm’s ability to efficiently run projects, allows for comparisons across all projects currently in development, and isolates both the positive and negative variances as they occur. All of this is useful information for top-level management of multiple projects. TABLE 13.9 Project Portfolio Earned Value (in thousands $) Project Alpha Beta PV EV Time Var ($) Var AC Cost Var ($) Var + Plan Est. at Completion 91 73 -18 18 83 -10 10 254 289 130 135 5 0 125 10 0 302 280 Gamma 65 60 -5 5 75 -15 15 127 159 Delta 25 23 -2 2 27 -4 4 48 56 84 82 -2 2 81 1 0 180 178 395 373 Epsilon Total Schedule Variance 391 27 Relative Schedule Variance 27/395 = 6.84% Total Cost Variance 29 Relative Cost Variance 29/395 = 7.34% 962 13.4 Using Earned Value to Manage a Portfolio of Projects 451 PROJECT PROFILE Earned Value at Northrop Grumman “There comes a time to shoot the engineers and get on with production.” This statement, commonly voiced in defense industry companies, refers to the engineers’ tendency to continually improve but never complete a project. The penchant for continual “tinkering” has enormous implications for companies that live or die by their ability to effectively and efficiently implement their projects. The type of work defense contractors perform further complicates the problem. There is a standing requirement that a company must meet the government’s stringent cost and quality control tests as it brings projects through the development cycle. In an effort to regain control of the project development process, defense contractor Northrop Grumman has been committed to the use of Earned Value Management for a number of years. Northrop Grumman, one of the world’s leading defense contractors (see Figure 13.17), has been using Earned Value Management as a key component of its approach to better project tracking and control. Because of the numerous projects the company routinely undertakes, its annual operating budget for projects runs into the billions of dollars. With dozens of projects under way at any time and enormous capital commitments supporting these ventures, it is imperative that the corporation develop and maintain the most sophisticated project control system possible. Northrop Grumman has selected Earned Value Management as its primary project control device for the following reasons: 1. 2. 3. 4. 5. 6. EVM develops a comprehensive baseline plan for the scope of the program’s work over its entire duration. The system incorporates tools to measure work performance and accomplishments based on objective criteria. EVM analyzes and forecasts the impact of significant variances from the plan. It produces managerial decision-making information in ascending levels of management. EVM provides action plans for corrective actions when something digresses from the baseline plan. All parties involved in the plan agree to and document all changes. The company has developed a four-tier approach for project control using EVM. All projects are classified into one of the following categories, requiring an individualized approach to EVM creation: Tier One is the most stringent because it requires most of the system’s features to be identified. This approach is employed when a contract requires that a large amount of detailed information be produced and reported. Tier Two is similar to Tier One except that the contract requires close management oversight because the project is risky, and there is a heavier burden to meet profit margin goals. Tier Three applies to programs of significant size that are mature and running smoothly. Tier Four applies the benefits of earned value to projects with low administrative costs. FIGURE 13.17 Northrop Grumman’s B-2 Bomber Source: Mark Meyer/The Life Images Collection/Getty Images (continued) 452 Chapter 13 r Project Evaluation and Control Northrop Grumman uses EVM at every stage of a project, from developing the original metrics at the contract proposal stage, to updating them in the form of a full-blown work breakdown structure (WBS) once a project has been won and is under development, to routinely updating the status of project activities on a weekly basis. Over time, the company discovered that monthly reviews were simply too far apart and did not allow for real-time corrective action when it was necessary. Flow of Earned Value System Earned value begins early at Northrop Grumman projects. In fact, there is a “flow” to the EVM system, beginning at the proposal stage, moving into a baseline development phase when a contract has been awarded, and then becoming part of a routine maintenance and data generation phase as the project is in development and moving toward successful completion. Proposal stage. The specifics of the program are determined at this stage. Among the key considerations to be determined is the form of EVM to be applied to the program if the proposal is successful and the contract awarded. Different clients may require different earned value metrics or evaluation windows that have to become part of the proposal. Contract award. When Northrop Grumman is selected as the successful contractor, all critical elements and requirements of the project are defined, including WBS, scope, delivery schedule, target budgets, as well as an earned value plan that will be used as the basis for status measurement and updates across the project life cycle. Baseline stage. Once the preliminary scope and deliverables have been agreed to between the contractor and the client, the detailed planning, project schedule, and formal work authorization is developed. The baseline is created now that key work packages and deliverables are identified and budgets are assigned to create a time-phased project budget. Maintenance phase. Once the project baseline is established and formally signed off by key parties, the project passes to the monitoring and control stage, where the key advantages of EVM are clearly realized. Performance is measured, schedules are updated, and all significant variances are identified and reported. People responsible for the actual performance of the work receive EVM reports at the detailed level and the system is transparent so that government representatives, interested in status updates, can receive real-time data on cost performance (CPI) and schedule performance (SPI) in accordance with contract requirements. Throughout the project life cycle, EVM considerations continue to drive the project forward; shaping the organization of the project, the development of critical planning features, and the manner in which it is controlled. At Northrop Grumman, EVM is not simply an option, but a corporate mandate. The four-tier approach helps the company tailor the system to each new project in order to apply it correctly for maximum benefit, cost control, and corporate profitability.5 13.5 ISSUES IN THE EFFECTIVE USE OF EARNED VALUE MANAGEMENT As with any other metric that helps us understand the “true” status of an ongoing project, the key to effective use of EVM lies in providing accurate and up-to-date information on the project, particularly in terms of the percentage of work packages completed. Because this information is key to determining the earned value at any point in time, the calculated EV is only as accurate as project team members and managers allow it to be through developing and enforcing an honest reporting system. In our Project Mercury example shown earlier (Table 13.4), the percentage completion column included values ranging from 100, 80, 60, 33, 25, to zero. In reality, organizations often adopt a simpler decision rule for assigning completion percentages. Among the more common methods for assigning completion values are the following: 1. 0/100 rule—The simplest and perhaps least effective method requires that a project activity be assigned a value of zero (0) until the point the activity is finished, at which time the value switches to 100%. This rule works best for work packages with very short durations, such as a day or two, but it is not useful for longer work packages because it provides little real-time information on an ongoing basis. It also makes sense for work packages that require vendor deliveries or that depend upon external stakeholders performing required steps. For example, we count a work package as “complete” when the vendor delivers a needed component. 2. 50/50 rule—Under this decision rule, an activity that has been started automatically receives a valuation of 50% completed. That value remains attached to the work package until the 13.5 Issues in the Effective Use of Earned Value Management 453 activity has been completed, at which time it becomes 100% completed. Like the 0/100 rule above, this decision model is used most often for work packages of very short duration. 3. Percentage complete rule—Under the percentage complete rule, the project manager and team members mutually agree on a set of completion milestones, whether they are based on quarters (25%, 50%, 75%, 100%), thirds (33%, 67%, 100%), or some other values. Then, on a regular basis, the status of each in-process work package in the project is updated. A new completion value may or may not be assigned to the package, and then the project’s EVM is updated based on this new information. As noted earlier, the key to making this process work lies in honest appraisal of the status of ongoing activities, based not on time elapsed or budget spent but on actual percentage of the activity completed. An important caveat with the percentage complete rule has to do with the controversy surrounding the level of detail to be used in calculating task value. Critics of earned value argue that unless reasonable gradients of completion are acknowledged and used by all parties, there is a high potential to create misleading information through the earned value analysis. For example, one criticism leveled at EVM argues that excessive levels of detail are dangerous and essentially not interpretable. For example, suppose a project uses completion values based on 10% increments (e.g., 10%, 20%, 30%, etc.). As a practical matter, it is fundamentally impossible to successfully delineate between, say, 30% and 40% completion for most project activities; hence, the use of too much detail is more likely to mislead than to clarify the true status of a project. The chief exception to this difficulty with the project complete rule occurs in projects in which there is a fair degree of prior knowledge of how well delineated the development process is or in situations where it is easier to accurately gauge the amount of work done within any project task. In a simple construction project, for example, where the project steps are well known in advance and rigorously followed, a higher level of detail can be employed. Likewise, in the case of software development where the task consists of writing code, a senior programmer may have an excellent sense of the total number of lines of code needed to complete the task. Therefore, if the total task requires approximately 5,000 lines of code and a programmer completes 500 lines of the program, it would be reasonable to assign a figure of 10% completion of the total task performance requirement. The importance of establishing a reasonable standard for project performance cannot be overemphasized. In the absence of a clear set of guidelines for identifying cutoff points and the appropriate level of detail, it is possible to derive very different conclusions from the same project information. For example, let us revisit the earlier EVM problem shown in Table 13.4. This time, we will use two decision rules as regards the levels of detail for project activities in calculating value and EV. In the first example, shown in Table 13.10, column 1 gives the original calculations, based on the first set of percentage complete values from Table 13.4. In column 2, I have employed a simple decision rule based on three increments (0, 50%, and 100% complete). Column 3 shows a slightly more precise level of detail, employing levels of 0, 25%, 50%, 75%, and 100% complete. I have rounded the original percentage completion values (shown in column 1) to the closest equivalents in the other two alternatives. Note what occurs as a result of using alternative levels of detail; rounding the level of completion values to a simplified 0%, 50%, 100% completion scheme results in significantly different results, both for projecting future project schedule and cost deviations. The original schedule overrun that projected a new completion of 16.28 months has been improved to 12.73 months, or a schedule overrun of only 5.73 months. Likewise, the original earned value budget projection for the project ($210,714) has been reduced to $163,889, for a savings of $46,825 due merely to adopting an alternative level of detail for project activity completion. Similarly, using the level of detail with slightly more gradients (0, 25%, 50%, 75%, and 100%), shown in column 3, and rounding the original values to most closely match this alternative, we discover that the future projections for the project, as developed through the SPI and CPI, are more negative than the originals. The new project schedule is forecast to last 17.5 months and the revised project budget has increased to $226,923, or $16,209 more than our first projection. Even more compelling, the absolute difference between the high and low budget projections is more than $63,000, all due to moving from a three-point level of detail (column 2) to one based on five levels of completion (column 3). Is one approach “more correct” than the other? Absent some decision rule or logic for making these determinations, it is virtually impossible to suggest that one level of detail is more representative of the “true” status of project activity completion. 454 Chapter 13 r Project Evaluation and Control TABLE 13.10 Calculating Project Mercury Earned Value Based on Alternate Levels of Detail (in thousands $) Col. 1 (Original) Col. 2 (0, 50, 100%) Col. 3 (0, 25, 50, 75, 100%) Planned Value % Comp. Value Staffing 15 100 15 100 15 100 Blueprinting 10 80 8 100 10 75 7.5 Prototype Development 10 60 6 50 5 50 5 Full Design 21 33 7 50 10.5 25 5.25 Construction 32 25 8 50 16 25 8 Transfer 10 0 0 0 0 0 0 Punch List 20 0 0 0 0 0 Activity % Comp. 44 Total EV = SPI and Projection to Completion CPI and Project to Completion Value % Comp. 56.5 Value 15 0 40.75 44/103 = .43 56.5/103 = .55 40.75/103 = .40 (1/.43 * 7) = 16.28 mos. (1/.55 * 7) = 12.73 mos. (1/.40 * 7) = 17.5 mos. 44/78 = .56 56.5/78 = .72 40.75/78 = .52 $210,714 $163,889 $226,923 As this chapter has noted, earned value management is not a flawless methodology for project tracking and control, particularly as it pertains to the problems in accurately determining the percentage of work packages completed at any time point during the project’s development. Nevertheless, EVM does represent a significant step forward in allowing project managers and their teams to gain a better perspective on the “true” nature of a project’s status midstream, that is, in the middle of the development and implementation process.6 This sort of real-time information can be invaluable in helping us gain current information and begin to develop realistic plans for correcting any systematic problems with the development process. The more we learn, and the faster we learn it, of a project’s status, the better equipped we will be to take measured and effective steps to get a troubled project back on track. In recent years, developments in project monitoring and control have led to some modifications and enhancements to the standard EVM processes widely in use. For example, an additional criticism of earned value relates to its use of cost data (budget) to assess not only cost performance but also schedule performance. That is, it has been reasonably argued that earned value does a fine job of monitoring cost performance relative to value earned in a project at any given point in time. We have seen how this information can be used for forecasting project costs at completion. However, there is a philosophical disconnect in the idea of using cost data to measure schedule performance; that is, using this same cost information as a means for forecasting schedule performance has been shown to potentially skewed data, leading to falsely positive or negative status assessments the further into a project we get. One solution recommended is the use of Earned Schedule (ES) as a complement to standard EVM analysis.7 ES is developed in detail in the appendix to this chapter. Another criticism of EVM lies in its potential lack of usefulness for some classes of projects that are more complex or involve uncertain technologies, like R&D or radical new product development projects. For these complex product system (CoPS) projects, authors have suggested that standard metrics of project performance may simply be ineffective and instead, propose a variation of EVM referred to as Earned Readiness Management (ERM) that focuses on the maturity of the project and overall system development. Though still in its early stages, ERM shows promise for combining the best features of earned value with a broader perspective needed for these projects.8 13.6 HUMAN FACTORS IN PROJECT EVALUATION AND CONTROL Another recurring problem with establishing accurate or meaningful EVM results has to do with the need to recognize the human factor in all project activity completion projections. That is, there is a strong incentive in most organizations for project team members to continuously report 13.6 Human Factors in Project Evaluation and Control 455 stronger results than may be warranted in the interest of looking good for the boss or sending the right signals about the project’s status. Worse, many times implicit or even explicit pressure may come from the project managers themselves, as they find themselves under pressure from top management to show steady results. Hence, the level of detail controversy is not simply one of accurately matching technical performance on the project to the best external indicator or number of gradients. Often it is also a problem rooted in human behavior, suggesting that excessively fine levels of detail not only may be inappropriate for the types of project activities we engage in, but also may be prone to misuse by the project team. The common feature of control approaches is their reliance on measurable data based on project outcomes; that is, the results of project actions taken in any one time period are collected and reported after the fact. Hence, we determine schedule or cost variance after the information has been collected and reported. Some project management writers, however, have suggested that it is equally essential to maintain a clear understanding of the importance of the management of people in the project implementation process. In other words, the data collected from the various evaluation and control techniques represents important outcome measures of the project; however, comprehensive project control also requires that the project organization employ sufficient process evaluations to determine how the development is progressing. A key component of any process evaluation of project performance must include an assessment of its people, their technical skills, management, teamwork, communication processes, motivation, leadership, and so forth.9 In short, many evaluation and control techniques (such as EVM) will do an excellent job in answering the “what” questions (What is the status of the project? What is our cost efficiency factor? What tasks are currently running late?), but they do not attempt to answer the “why” questions (Why are activities behind schedule? Why is the project team performing at a suboptimal level?). In an effort to provide answers to the “why” questions, work on the human processes in project management has been initiated and continues to be done. Past research examining the impact of human factors on project success bears out the importance of considering the wider “management” challenge inherent in managing projects. For example, early work of Baker and colleagues10 identified a variety of factors that directly predict project success. Included in their list were issues such as: r
Project coordination and relations among stakeholders r
Adequacy of project structure and control r
Project uniqueness, importance, and public exposure r
Success criteria salience and consensus r
Lack of budgetary pressure r
Avoidance of initial overoptimism and conceptual difficulties Their findings bear out the importance of having a clear knowledge of the managerial challenges involved when implementing projects. These findings have been reinforced by other research that has examined a set of both successful and unsuccessful projects across their life cycle.11 The findings of such research are intriguing because of the importance they place on the managerial and human behavioral aspects of project management for project success. As Table 13.11 shows, regardless of whether the project studied was a success or failure, the factors that were of highest importance demonstrate some clear similarities. Issues such as leadership, top management support, team and personal motivation, and client support were consistently linked with project success, suggesting once again that an understanding of the project management process is keenly important for determining the likelihood of a project’s successful outcome. One of the key recurring problems, however, with making wider use of nontechnical information as a method for controlling projects and assessing their ongoing status lies in the question of measurement. Although financial and schedule data can be easily acquired and are relatively easy to interpret, measuring human processes such as motivation level, leadership, top management support, and so forth is highly problematic. As a result, even though a number of project management theorists have accepted the argument for inclusion of human process factors in assessing the status of ongoing projects, there has been little agreement as to how best to make such assessments, interpret the results, and use the findings in a prescriptive manner to improve the project processes. The work of Pinto and Slevin12 addresses the shortcomings with behavioral assessments of project management processes. They formulated the Project Implementation Profile (PIP), a 456 Chapter 13 r Project Evaluation and Control TABLE 13.11 Key Success Drivers and Inhibitors Stage Successful Projects Factors Stage Failed Projects Factors Formation Personal ambition Formation Unmotivated team Top management support Poor leadership Team motivation Technical limitations Clear objectives Funding problems Technological advantage Buildup Team motivation Buildup Personal motivation Unmotivated team Conflict in objectives Top management support Leadership problems Technical expertise Poor top management support Technical problems Main Phase Team motivation Main Phase Unmotivated team Personal motivation Poor top management support Client support Deficient procedures Top management support Closeout Personal motivation Team motivation Closeout Poor control Poor financial support Top management support Unclear objectives Financial support Leadership problems 10-factor instrument that assesses the performance of the project team with respect to 10 critical success factors, that is, those factors they found to be predictive of project success. The advantage of the PIP is that it allows project teams to formally assess their performance on the ongoing project, allowing for midcourse correction and improvement of the management process itself. The 10 critical success factors represent an important, supplemental source of information on the project’s status. Coupled with other types of evaluation and control information supplied through the tracking of cost and schedule variance against the project baseline, project teams can develop a comprehensive vision of the project’s status throughout its development. Critical Success Factor Definitions The 10 critical success factors identified by Pinto and Slevin in formulating the Project Implementation Profile (PIP) instrument are (1) project mission, (2) top management support, (3) project plans and schedules, (4) client consultation, (5) personnel, (6) technical tasks, (7) client acceptance, (8) monitoring and feedback, (9) communication, and (10) troubleshooting. Each of these factors is discussed in more detail in the text that follows. Project mission, the first factor, relates to the underlying purpose for the project. Project success is predicated on the importance of clearly defining objectives as well as ultimate benefits to be derived from the project. Many times, the initial stage of project management consists of a feasibility decision. Are the objectives clear and can they succeed? Project mission refers to a condition in which the objectives of the project are clear and understood, not only by the project team involved, but also by the other departments in the organization. The project manager must be concerned with clarification of objectives as well as achieving broad belief in the congruence of the objectives with overall organizational objectives. Top management support, the second factor, has long been considered of great importance in distinguishing between ultimate success and failure. Project managers and their teams not only are dependent upon top management for authority, direction, and support, but also are the conduit for implementing top management’s plans, or goals, for the organization.13 Further, if the project is being developed for an internal audience (one within the company), the degree of management 13.6 Human Factors in Project Evaluation and Control 457 support for a project will lead to significant variations in the degree of acceptance or resistance to that project or product. Top management’s support of the project may involve aspects such as allocation of sufficient resources (financial, personnel, time, etc.) as well as project management’s confidence in support from top management in the event of a crisis. The third factor, project plans and schedules, refers to the importance of developing a detailed plan of the required stages of the implementation process. It is important to remember, however, that the activities associated with project planning and project scheduling are distinct from each other. Planning, which is the first and more general step in developing the project implementation strategy, is composed of scope definition, creation of a Work Breakdown Structure, and resource and activity assignments. Scheduling is the setting of time frames and milestones for each important element in the overall project. The project plans and schedules factor is concerned with the degree to which time schedules, milestones, labor, and equipment requirements are specified. There must be a satisfactory measurement system to judge actual performance against budget allowances and time schedules. The fourth factor is client consultation. The “client” is anyone who ultimately will be using the product of the project, either as a customer outside the company or as a department within the organization. Increasingly, the need for client consultation has been recognized as important in attempting a system implementation; indeed, the degree to which clients are personally involved in the implementation process correlates directly with variations in their support for projects.14 It is important to identify the clients for the project and accurately determine if their needs are being met. The fifth factor, personnel, includes recruitment, selection, and training of project team members. An important, but often overlooked, aspect of the implementation process concerns the nature of the personnel involved. In many situations, personnel for the project team are chosen with less than full regard for the skills necessary to actively contribute to implementation success. The personnel factor is concerned with developing an implementation team with the ability and commitment to perform their functions. Technical tasks, the sixth factor, refers to the necessity of having not only the required numbers of personnel for the implementation team but also ensuring that they possess the technical skills and the technology and technical support needed to perform their tasks. It is important that people managing a project understand the technology involved. In addition, adequate technology must exist to support the system. Without the necessary technology and technical skills, projects quickly disintegrate into a series of miscues and technical errors. The seventh factor, client acceptance, refers to the final stage in the project development process, at which time the overall efficacy of the project is to be determined. In addition to client consultation at an earlier stage in the system implementation process, it remains of ultimate importance to determine whether the clients for whom the project has been initiated will accept it. Too often project managers make the mistake of believing that if they handle the other stages of the implementation process well, the client (whether internal or external to the organization) will accept the resulting system. In fact, client acceptance is a stage in the project life cycle process that must be managed like any other. The eighth factor, monitoring and feedback, refers to the project control process by which, at each stage of the project implementation, key personnel receive feedback on how the project is progressing compared to initial projections. Making allowances for adequate monitoring and feedback mechanisms gives the project manager the ability to anticipate problems, to oversee corrective measures, and to ensure that no deficiencies are overlooked. Project managers need to emphasize the importance of constant monitoring and fine-tuning project development; tracking control charts and Earned Value Management are excellent examples of the techniques and types of monitoring and control mechanisms necessary to develop a project. Communication, the ninth factor, is not only essential within the project team itself, but—as we discussed in regard to stakeholder management—it is also vital between the team and the rest of the organization as well as with clients. Communication refers both to feedback mechanisms and to the necessity of exchanging information with both clients and the rest of the organization concerning the project’s capabilities, the goals of the project, changes in policies and procedures, status reports, and so forth. Therefore, channels of communication are extremely important in creating an atmosphere for successful project implementation. Troubleshooting is the tenth and final factor of the model. Problem areas exist in almost every project development. The measure of a successful project is not the avoidance of problems, but 458 Chapter 13 r Project Evaluation and Control taking the correct steps once problems develop. Regardless of how carefully the implementation effort is initially planned, it is impossible to foresee every trouble area or problem that can possibly arise. As a result, the project manager must include mechanisms in the implementation plan for recognizing problems and for troubleshooting them when they arise. Such mechanisms make it easier not only to react to problems as they arise, but also to foresee and possibly forestall potential problem areas in the implementation process. Conclusions This chapter has addressed a variety of approaches to project tracking and control. Although most of the models mentioned have many advantages associated with them, project management professionals should be aware of the concomitant problems and shortcomings with these approaches as well. The key to developing a useful project control process lies in recognizing the strengths and weaknesses of the alternative methods and ultimately developing an approach that best suits the organization, the projects undertaken, and the stakeholders of the project. A project control process should be tailored, to the degree possible, to the specific needs, culture, and uses for which an organization intends it. Thus, under some circumstances, a simplified control system may be sufficient for providing management with the types of information they require. Alternatively, some organizations and/or projects will need to employ highly sophisticated control processes because of either the unique nature of their operating processes or the demands that developing projects place on them (e.g., governmental stipulations and mandates).15 The comprehensive and intricate concept of project evaluation and control involves the need to understand alternative evaluation techniques, recognizing their particular usefulness and the types of information they can provide. Ultimately, however, these techniques are merely as good as the project planning process; that is, a good control system cannot make up for inadequate or inaccurate initial plans. Without effective baselines, good project cost estimation and budgeting, and adequate resource commitments, project control simply will not work. However, if the upfront planning has been done effectively, project evaluation and control can work in harmony with the project plans, providing the project team with not only a clear road map to success, but also excellent mileposts along the highway. Summary 1. Understand the nature of the control cycle and four key steps in a general project control model. Accurately evaluating the status of ongoing projects represents a real challenge for project teams and their parent organizations. The process of project control, consisting of a recurring cycle of four steps (setting goals, measuring progress, comparing actual progress with plans, and correcting significant deviations), demonstrates a theoretical framework for understanding the continuous nature of project monitoring and control. 2. Recognize the strengths and weaknesses of common project evaluation and control methods. A number of project evaluation and control techniques exist, from the simplistic to the highly sophisticated. The most basic evaluation process, project S-curves, seeks to reconcile the project schedule baseline with planned budget expenditures. The cumulative project budget, resembling the letter S, creates a schedule/budget relationship that early project monitoring methods found useful as an indicator of expected progress. Unfortunately, a number of problems with S-curve analysis preclude its use as an accurate evaluation and control technique. Other evaluation methods include milestone analysis and tracking Gantt charts. These approaches link project progress to the schedule baseline, rather than the project budget. As with S-curves, milestones and tracking charts have some advantages, but they all share a common drawback: the inability of these methods to accurately assess the status of ongoing activities, and therefore the “true” status of the project, in a meaningful way. Specifically, because these monitoring and control methods do not link schedule and budget baselines to actual ongoing project performance, they cannot offer a reasonable measure of project status. 3. Understand how Earned Value Management can assist project tracking and evaluation. Earned Value Management (EVM) is a powerful tool, developed through a mandate from the federal government, to directly link project progress to schedule and budget baselines. In effect, EVM provides the missing piece of the control puzzle by requiring the reporting of actual project activity status on a realtime basis. Earned Value Management has begun to Solved Problem 459 diffuse more rapidly within ordinary project-based organizations as they increasingly perceive the advantages of its use. 4. Use Earned Value Management for project portfolio analysis. The basic principles that govern the use of earned value on a single project can be applied to a portfolio of projects. Each project is evaluated in terms of the basic efficiency indexes for time and cost, and an overall evaluation can be calculated for a firm’s project portfolio. This portfolio model allows us to determine the overall efficiency with which we manage projects, to see which are ahead and which are behind the firm’s baseline standards. 5. Understand behavioral concepts and other human issues in evaluation and control. A final method for tracking and evaluating the status of ongoing projects lies in the use of alternative control methods, aimed at assessing and managing the “human issues” in project management, rather than focusing exclusively on the technical ones. In other words, EVM and other previously discussed tracking and control mechanisms focus on data-driven measures of performance (budget, schedules, and functionality); but other models that address the managerial and behavioral issues in project management argue that unless we merge these data-driven models with those that assess the project in terms of human interactions (leadership, top management support, communication, and so forth), it is possible to generate a great deal of information on the current status of a project without recognizing the primacy of human behavior in determining the suc...
Purchase answer to see full attachment