A disputation between the authors and the reviewing editor.(Letter to the editor)

Letter to the Editor

Authors Bertisen and Davis are commended for their study of mine project capital cost estimation. They rightly point out that the estimated cost when compared to the actual cost errs usually by underestimating. They analyze the "undershooting" and propose a remedy and also suggest that the persistence of bias is intentional. They say that the estimating by project consulting firms, who do the estimating of the mine capital costs for investment banks, is manipulated in such a way to have the capital costs be less, and so have the NPV be positive, and when ranked against other competitive projects, their project will float to the top of the financing pecking order. A strong statement by the authors, but are their claims justified? Indeed, there are legal ramifications to these disputes, and implicit collusion between the engineering consultants and the project sponsor is suggested by the authors. This last matter is not discussed here.

My counterarguments fall into four points. First, the comparison of the 63 projects between the estimated and actual values overlooks those projects that were estimated but did not achieve success in winning with a favorable NPV, which would lead to a rejection. In conjecture, of course, the actual costs of the unsuccessful projects are never determined. Perhaps the capital cost estimates were too high, and we are unable to determine the overestimating, and the NPV and project cost did not lead to a funded project. There are many reasons for uncompetitive bids. Analysis of estimating performance must consider all jobs, not just the ones that lead to winning projects. While it is appealing to compare the estimated costs to actual costs, comparisons that are straightforward are difficult to achieve. Consulting firms who do "for-hire estimating" typically keep a record of their capture rate, which is the metric of the winning bids to total number of bids. The ratios are typically low, and ranges of 10-25% are not uncommon.

The authors dismiss the impact that design and schedule effects can have upon the actual cost growth. Remember that years may elapse between the time of the estimate and the start and duration of the project. During the pre-work period and construction, there are design changes, which typically lead to contract cost increases. For example, we know of alleged and notorious instances where contractors on defense work were audited on contract increases above the estimate, but when the facts were examined, design change and schedule stretch-out were the contributing reasons.

The worldwide raw data of the 63 projects are points, not distributions. An element is not a "distribution" in the classical statistical sense. An "'element" does not have a mean or variance, a requirement of the central limit theorem (CLT), and invalidates the CLT application to the data. The elements are nonparametric, and thus other procedures are necessary for analysis.

The authors propose a remedy to underestimating. Monte Carlo procedures are given. They suggest that a statistical learning model can add a cost component to the estimate to raise the estimate to a potentially higher value, though it is possible that some estimates could decline. I think that procedures that adjust the estimate are unsound, especially whenever Monte Carlo methods are applied. It must be agreed that not all estimates undershoot the actual cost. But which estimates over- or undershoot the actual cost are not known beforehand. To add a cost to an already low estimate may make it uncompetitive, or to subtract a cost from an already high capital cost can make the difference between the estimate and actual cost wider. I am uncomfortable with techniques that add a Monte Carlo factor to these estimates. Improved professional engineering procedures are preferred.

Phillip F. Ostwald

Emeritus Professor of Mechanical Engineering

University of Colorado at Boulder

Print Get the Mag Weekly Updates