Evaluation of Models for
Forecasting the Final Cost of a Project
Ofer Zwikael, Graduate School of Business Administration, Tel Aviv University, Tel Aviv 69978 Israel
Shlomo Globerson, PMP, Graduate School of Business Administration, Tel Aviv University, Tel Aviv 69978 Israel
Tzvi Raz, PMP, Graduate School of Business Administration, Tel Aviv University, Tel Aviv 69978 Israel
■
Abstract
This paper addresses how to estimate the final cost of a project and when the estimate becomes accurate. The performance of
five forecasting models drawn from literature was evaluated with data from a sample of actual projects. A stability analysis was
carried out in order to identify when the forecasts become stable and accurate for the model that emerged as the most accurate.
Keywords: cost performance index; earned value; cost control; forecasting
©2000 by the Project Management Institute — 2000, Vol. 31, No. 1, 53–57 — 8756–9728/00/$5.00 per article + $0.50 per page
A
main concern of every project manager is not to exceed the approved project budget. Indeed, a major
portion of project control efforts is devoted to ensuring that
actual costs do not deviate from planned costs. In reality,
cost overruns are quite common, and, in many business
fields, they are the norm rather than the exception. However, cost overruns are not always the fault of the project
manager or the project team. Often, in high-risk areas that
involve the development of new products, systems, or technologies, projects end up being inadequately funded due to
a variety of reasons. These reasons include lack of knowledge regarding the true extent of the effort involved, inadequate reserves to account for technical challenges, as well as
unwarranted optimism, which sometimes is needed in
order to gain approval for the project to proceed.
If a project is bound to deviate from its approved
budget, then it is helpful to be able to predict the extent of
the deviation. The sooner this information becomes available and the more accurate it is, the more useful it becomes for dealing with the various project stakeholders
and for the organization’s financial planning.
Literature offers several methods for forecasting final
project cost, based on actual cost performance at intermediate points in time. These models vary in terms of their
assumptions. They range from the naive belief that the
project manager will be able to overcome all cost deviations experienced to date and will complete the project
within the original budget, to the pessimistic argument
that deviations will continue to accrue at the rate observed
March 2000
so far. These two extreme approaches, as well as other intermediate methods, are presented in most project management textbooks—for example see Shtub, Bard, and
Globerson (1994). However, there is no agreement regarding which forecasting model is the most accurate.
The research of Fleming and Koppelman (1995a) suggests that the cost performance index (CPI), as calculated according to the earned value method, provides a good basis
for cost forecasting models. The work of Beach (1990) and
Christensen and Heise (1993) indicates that the CPI final
value can be forecasted accurately in the early stages of the
project, possibly as early as after 15% of the project duration.
In this paper, the authors compare the performance of
five models for forecasting final project cost. The comparison is based on data from 12 projects that were carried out
in a high-tech company in Israel during a five-year period.
The analysis identified one model that was clearly superior
to the others, in terms of forecast accuracy. This model was
further examined in order to determine the point in the
project life cycle when the forecast the final cost.
Forecasting Models
The forecasting models selected differ in terms of the assumptions upon which they are based. The following
earned value terminology is used to describe the models.
BCWS Budgeted Cost of Work Scheduled
ACWP Actual Cost of Work Performed
BCWP Budgeted Cost of Work Performed
Project Management Journal
53
CPI
SPI
BAC
EAC
Cost Performance Index (BCWP/ACWP)
Schedule Performance Index (BCWP/BCWS)
Budget at Completion (original planned cost
of the project)
Estimate at Completion (forecasted final cost
of the project).
The five forecasting models along with their respective
sources and assumptions, are listed.
1. Constant budget—This model assumes that all
cost deviations will be corrected by the time the project
is completed (Fleming & Koppelman, 1994), implying
that the final cost will be equal to the planned budget:
EAC=BAC.
2. Constant cost deviation value—This model assumes
that the remainder of the project will be executed according to the original plan (Fleming & Koppelman,
1996), implying that the numerical value of the budget
deviation at the time of the forecast will not change:
EAC = BAC + (ACWP–BCWP).
■
About the Authors
Ofer Zwikael is a Ph.D. student in project management in
the faculty of management at Tel-Aviv University, Israel. He has a B.Sc. in industrial
engineering from Ben Gurion University
and a MBA from Tel-Aviv University. He
has been working in the Israeli Navy for
the past six years.
Shlomo Globerson, PMP, Ph.D., is an internationally known researcher, educator,
and consultant in the fields of project
management and operations management. A professor at the Graduate School
of Business Administration, Tel Aviv University, he is extensively involved in developing new courses and workshops for MBA students,
project managers, and top executives. He teaches project
management courses in MBA programs, and runs project
management workshops around the world. He holds a
Ph.D. degree in industrial engineering from the University
of California at Berkeley and has published over 60 refereed articles and six books.
Tzvi Raz, PMP, holds B.Sc, M.A.Sc, and
Ph.D. degrees in industrial and management engineering. He is on the faculty of
the management of technology program of
the Leon Recanati Graduate School of
Business Administration at Tel Aviv University. Previously, he managed a technology
insertion program at an IBM software development laboratory, and was on the industrial engineering faculties of the
University of Iowa and Ben Gurion University. Dr. Raz is on
the editorial review boards of Computers and Operations
Research, the Project Management Journal, and the International Journal of Industrial Engineering.
54
3. Constant cost efficiency rate—This model assumes
that the cost efficiency achieved so far in the project will
remain through the remaining part (Shtub, Bard, &
Globerson, 1994): EAC=BAC/CPI.
4. Constant cost and schedule efficiency rate—This
model assumes that the final cost is affected by both
the cost efficiency rate and the schedule efficiency rate
(Fleming & Koppelman, 1994): EAC=BAC/(CPI*SPI).
5. Future constant cost and schedule efficiency rate—This
model assumes that the cost deviation for the remainder of
the project is a function of both the cost efficiency rate and
the schedule efficiency rate. This deviation will be in addition to the deviation accumulated so far (Fleming & Koppelman, 1995b): EAC = ACWP+(BAC–BCWP)/(CPI*SPI).
Data
The 12 projects were all fixed-price and relatively low-risk
projects, involving the production of a series of high-tech
products, with little development work. Consequently, the
expenditure rates were relatively stable over time, and there
was a strong incentive not to exceed the planned budget.
All 12 projects were carried out during a five-year period
during 1993–1997. Budget and cost-control data were captured and processed by the same information system under
the supervision of the same financial control manager.
Cost and schedule data for the 12 projects in the
sample appear in Table 1. The average planned cost was
$1.3M, and the average duration was three years, with the
vast majority of the projects ending with cost and schedule
overruns. It is interesting to note that all projects in the
sample were carried out for external customers under a
fixed-price contract. Costs pertaining to scope changes introduced during the project execution were excluded from
the analysis.
The analysis was carried out in the following manner.
Each project was divided into 10 segments of equal duration, each representing 10% of the actual duration. From
historical records maintained in the financial control management system, the cumulative earned value (BCWP),
planned cost (BCWS) and actual cost (ACWP) at the end
of each of the 10 segments were calculated. Next, the performance indices CPI and SPI were calculated for each segment end point, and forecasts were obtained using the five
methods described in the previous section. Since there is
no point in forecasting the total cost at the end of the
project, the end point of the last segment in the analysis
was not included. The data set consisted of five forecasts at
each of the nine intermediate points in time for each of
the 12 projects. With this data, the performance of the five
forecasting models was evaluated by considering the deviation between each forecast and the actual final cost.
Project Management Journal
March 2000
Project
Actual
Cost
(k$)
Cost
Overrun
(%)
Planned
Schedule
(months)
Actual
Schedule
(months)
Schedule
Overrun
(%)
1
2
3
4
898
605
322
613
1,212
871
670
773
35%
44%
108%
26%
21
32
36
43
24
38
43
47
14%
19%
19%
9%
5
6
7
8
291
1,525
585
1,026
277
2,439
767
1,170
-5%
60%
31%
14%
24
50
46
29
24
59
54
30
0%
18%
17%
3%
9
10
2,223
6,077
2,979
6,988
34%
15%
45
44
55
50
22%
14%
353
1,305
1,319
646
2,099
1,733
83%
54%
31%
17
50
36
23
50
41
35%
0%
14%
11
12
Average
Table 1.
Planned
Cost
(k$)
Cost and Schedule Data for the 12 Sample Projects
Model
1.
2.
3.
4.
5.
Constant Budget
Constant Cost Deviation Value
Constant Cost Efficiency Rate
Constant Cost and Schedule Efficiency Rate
Future Constant Cost and Schedule Efficiency Rate
MSE (K $2)
MAD ($)
MAPE (%)
266
141
85
257
166
417
258
178
317
242
27
16
11
20
15
Table 2. Measures of Performance for the Five Forecasting Models
Measures of Forecasting Performance
Three measures of forecasting performance were used to
evaluate the five models:
■ Mean squared error (MSE)—average of the square of
the difference between the forecasted value and the actual value
Mean absolute deviation (MAD)—average of the absolute value of the difference between the forecasted value
and the actual value
■ Mean absolute percent error (MAPE)—average of the
absolute value of the difference between the forecasted
value and the actual value expressed as a percentage of the
actual value.
The average results for the 9 x 12 observations obtained for
each forecasting model appear in Table 2.
■
It is apparent from Table 2 that the worst model is
the one based on the assumption that the project will
eventually recover and will complete within the original budget (#1). The two models that incorporate both
the SPI and the CPI (#4 and #5) were inferior to the
two models based on the CPI only (#2 and #3). This
finding suggests that schedule performance is not truly
relevant to final cost performance. Among the two
March 2000
models based on the CPI only, the one based on the
constant efficiency rate assumption (#3) gave the best
results, while the next best model (#2) is at least 50%
worse, according to any of the three performance measures. Overall, the results of the analysis based on this
sample suggest that the most accurate forecasts are derived under the assumption that the cost efficiency observed from the beginning of the project up to the forecasting moment will stay constant through the end of
the project. In order to apply this model, all we need is
to calculate the value of the CPI, and divide the original
budget by it. Of course, the actual value of the CPI is
likely to change over time, depending on the specific
problems encountered and the corrective actions taken.
Stability Analysis
For the purpose of forecasting the final cost, it is important to know at which point during the life of the project
the CPI is sufficiently close to its final value. In Figure 1,
the deviation of the CPI value calculated at the nine intermediate points during the life of the project from the final
CPI value is plotted. Plot A shows the actual deviation,
Project Management Journal
55
Actual Deviation
0.2
0.1
0.0
-0.1
-0.2
-0.3
-0.4
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
80%
90%
100%
80%
90%
Percentage of Project Duration Elapsed
Figure 1a. Actual Deviation
0.4
MAD
0.3
0.2
0.1
0.0
0%
10%
20%
30%
40%
50%
60%
70%
Percentage of Project Duration Elapsed
Figure 1b. Absolute Deviation
80%
70%
MAPE
60%
50%
40%
30%
20%
10%
0%
0%
10%
20%
30%
40%
50%
60%
70%
100%
Percentage of Project Duration Elapsed
Figure 1c. Absolute Percentage Deviation
Figure 1.
Deviation From the Final CPI Over Time
Plot B shows the absolute deviation, and Plot C shows the
absolute deviation as a percentage of the final value. Each
plot contains 12 data points for each time period, corresponding to the 12 projects in the sample. In many cases,
two or more data values coincided and appear as a single
56
point on the plot. The solid line on each plot connects the
averages calculated for the 12 projects.
In order to study the trend in the CPI deviation, regression lines were fitted to the 12 x 9 points in each
graph. The results are summarized in Table 3. Regardless
Project Management Journal
March 2000
Slope of Linear
Regression Line
Astatistical
Significance
0.12
-0.13
-0.19
0.003
0.000
0.000
Actual Deviation
Absolute Deviation
Absolute Percent Deviation
Table 3. Regression Coefficients and Statistics
of whether we look at the actual deviation, the absolute
value, or the percentage, there is a clear finding: The deviation decreases as we progress toward the end of the
project. This is evident from the fact that all three regression lines are statistically significant. The positive slope
of the regression on the actual deviations reflects the fact
that, in this sample, the deviations are, in general, negative, and that as the project percentage elapsed increases,
they tend to become less negative and come closer to the
final CPI value. The meaning of the negative slopes of
the other two regression lines is also clear: As we get
closer to completion, the magnitude of the forecasting
error, either in absolute or relative terms, decreases.
This finding is hardly surprising. Intuition tells us
that the further along we are in the project, the more
difficult it is to implement corrective actions and to improve cost efficiency. However, it is still important to
determine at which point in the project life cycle the
CPI value and the resulting final cost estimate are sufficiently close to the true values.
Visual examination of the plots in Figure 1 suggests
that this happens at the 60% mark. This finding is more
conservative than those of Beach (1990), who analyzed
data from 700 projects and found that the final cost
overrun will not be less than the overrun after the first
15%, and Christensen and Heise (1993), who reported
that the difference between the CPI at 15% of the duration and the final CPI is no more than 10%. CPI stability depends to a great extent on the quality of the
original budget and the ability of the project manager
to correct deviations and stick to the plan. These two
factors vary widely across industries, companies, and
even teams and individuals. Therefore, the authors feel
quite comfortable with their finding, which, in fact,
states that one has to wait until after 60% of the project
elapses in order to be fairly certain of what the final
cost will be.
Concluding Remarks
In this paper, the issue of how to estimate the final project
cost and when the estimate becomes accurate was addressed. Although the models considered have been mentioned in literature, this is the first empirical study that carried out a numerical comparison. The authors’ analysis
showed clearly that the most accurate estimates are those
March 2000
obtained under the assumption that cost deviations will
continue at a constant rate. The subsequent stability
analysis suggests that the accuracy of the final cost estimate
improves after 60% of the project duration has elapsed.
Organizations involved in multiple projects could
benefit from carrying out the analysis reported here on
their own projects in order to improve cost forecasting capability. The procedure is relatively simple and utilizes
data that should be already available, since it is routinely
collected as part of the earned value control methodology.
Further, the analysis need not be restricted to the five
models studied here and could include any number of segments, either duration based or progress based, as well as
other organization-specific methods and variables.
The analysis was based on a relatively small sample of
projects carried out by a single organization. Although the
results are consistent with intuition and do not negate previously obtained findings, one should be careful before
generalizing them to projects that are very different from
those in the sample. In fact, the same research methodology should be applied to larger, more diverse projects in
order to validate the findings reported here.
References
Beach, C.P. (1990, November). A-12 administrative inquiry.
Navy Memorandum, p. 6.
Chen, M.T. (1996, April). An innovative project report. Cost
Engineering, 38.
Christensen, D.S., & Heise, S.R. (1993). Cost performance
index stability. National Contract Management Association Journal,
25 (1).
Fleming, Q.W., & Koppelman, J.M. (1994, November). The
essence of evolution of earned value. Cost Engineering, 36 (11).
Fleming, Q.W., & Koppelman, J.M. (1995a, May). The earned
value body of knowledge. PM Network.
Fleming, Q.W., & Koppelman, J.M. (1995b, October). Reengineering the earned value process: From government into the private sector. Proceedings of the 26th Annual Project Management Institute 1995 Seminars & Symposium, p. 7–12.
Fleming, Q.W., & Koppelman, J.M. (1996, January). Forecasting the final cost and schedule result. PM Network.
Shtub, A., Bard, J.F., & Globerson, S. (1994). Project management—Engineering, technology and implementation. Prentice Hall, p.
374–497.
Project Management Journal
57
Purchase answer to see full
attachment