Oxford study on large dams: the right to reply

Although acknowledging the importance of accurately predicting project outcomes, members of the dam industry have expressed their concern about the study, which claims that large hydroelectric dams are unviable and extremely damaging to emerging economies. "Should we build more large dams? The actual costs of hydropower megaproject development" was authored by Atif Ansar, Bent Flybjerg, Alexander Budzier and Daniel Lunn from the University of Oxford, and published in March 2014. It uses statistical techniquies applied to large dams for the first time and aims to give an "outside view" of the subject.

For an overview of the study conclusions see: The outside view: giving a clear picture of large dams?

Adama Nombre, ICOLD President

This study focuses on cost and time overrun without addressing the true challenges. It is suffering from important drawbacks and methodology issues that we will detail.

The existing 50,000 large dams supply 15% of the world's electricity production and provide irrigation water for feeding 800M people. But there are extra needs, as can be seen in many countries of Africa where people are cutting the forest for cooking energy, where they live in darkness and are hit by water-related diseases and malnutrition which result in millions of fatalities each year, mainly women and children

The study sample is also biased and unrepresentative. It is based on a sample of 245 dams, which appears as a total misrepresentation of the 50,000 large dams existing today, as shown in Table 1.

Traditional cost-benefit analysis is not well adapted to large dams which appear however as cost-efficient. Usual cost-benefit analysis, based upon high discount rates, is unfavourable to dams, which can operate for a century with low operation costs. Even with this method, hydropower worldwide is usually the most economic way for power production beyond coal power. A recent study by the International Renewable Energy Agency on the levelised cost of energy shows hydropower to be the least cost option of all the renewable energies. Even the World Commission on Dams concluded: "It is worth emphasising that cost recovery has not been a substantial problem for hydropower projects".

This new study completely ignores the climate change problem and doesn't provide any viable alternative to large dams and hydropower. The authors state that policy makers should prefer energy alternatives that require less upfront outlays and that can be built very quickly. But what would those alternatives be? Fossil fuel plants consuming coal or gas? Without explicitly saying it, the authors use a purely financial reasoning to bring us toward a carbon-emitting electric system. The carbon emissions of fossil fuel plant and the climate change problem are not mentioned in their text.

Applying the unjustified recommendations of Ansar et al. would be disastrous for the poorest countries of Africa, Asia, South and Central America. ICOLD, together with other international scientific institutions, has signed a World Declaration on Water Storage for Sustainable Development (Kyoto 2012), which explains why there is an urgent need to build more water infrastructure for the development and the well-being of the people of the world.

The conclusions of the Ansar report are also unjustified for the very large dams. The basic data (height, construction time etc) of the sample are closer to those of very large dams as financed by global financial institutions. But the average cost overrun of 99% claimed by the paper seems totally unjustified by the six references for this sample: Asian Development Bank (ADB), World Bank, World Commission on Dams, TVA, US Army Corps of Engineers and US Bureau of Reclamation. The paper does not give detailed figures, but the relevant data for those organisations are actually shown in Table 2.

The three other references are concerning 40 dams among the US dams, for which the report claims an average cost overrun of 11%.
Thus, the 99% claimed overrun cannot be explained by the six given references. One explanation could be the inclusion of 20 or 25 dams with a very large cost overrun (about 500% as average) which do not appear in the six references. One of these added dams alone, with a 5000% cost overrun, explains 20% of the 99% claimed. Such an extraordinary case (an actual cost 50 fold the initial evaluation) should not have been included without any justification. The dam is not even named.

ICOLD could give more detailed comments, provided the authors make their data and methods public.

Brian Sadden, Civil Engineering Consultant, US

More careful analysis of this paper is warranted. A quick review shows that some analysis and conclusions appear to be suspect.

Unfortunately the flaws in the analysis are excused by Prof Flyvbjerg's 'outside view' of research, which appears to allow key drivers to be ignored in favour of 'generic risk in a reference class'. It is not clear that the selection of the generic risk has been performed by someone with knowledge of dam development and implementation.

The authors believe that their sample is 'large' at 245 dams (although there are about 90,000 dams in just the US register of dams) and their 'reference class' appears to have been arbitrarily set as projects from the 1930s through 2007. By electing to examine this sample period in history, projects are included that were designed and constructed during some of the most significant events that affected the world economic environment during the 20th century. Just off the top of my head I can think of the following events during the sample period: the Great Depression; World War II; the Korean War; the Vietnam War; Oil shocks in the 1970s and 90s; post-war construction and reconstruction; the creation of the World Bank, ADB etc.; the Cold War; globalisation, the 1998 financial crisis etc.

Such huge economic and financially significant matters are discounted with the phrase "RCF focusses on generic risk inherent in a reference class". But how can one possibly neglect all these global currents in an analysis of major infrastructure projects, many of which are dependent on cross border lending, and implementation schedules that span the various events?

Almost unbelievably, the paper highlights the fact that the mean overrun in cost of the sampled projects is 90% above estimates - the accuracy of or time of which have not been clarified. The highlighting of the mean is sometimes known as 'abuse of statistics'. As all students learn, for positively skewed data, the mean has a higher value than the median. With the highly skewed curve that is presented in the paper - one project has been included that had a cost overrun of 5000% - it is an error to use the mean as much more than an indication of how long the 'tail' of the skewed curve actually extends.

This has got to be the single most important problem with the paper. A more realistic parameter to use would be the median or the mode, and we find that the median shows around 26% cost overrun. It would be interesting to see which 'original cost estimates' were used - some of the projects I have worked on were in gestation (with many false starts) for 30 years or more (during which many cost estimates were assembled), or took five years or more to obtain environmental permits after the decision to build.

If one uses the 50th percentile figures as a baseline for comment then the projected cost overruns for the construction of dams sits nicely between roads (15%), bridges (23%); rail projects (40%); standard civil engineering (3-44%); building (4-51%) and non-standard civil engineering (6-66%). The mean cost overrun of 6% for thermal projects - evidently - must be missing all data on coal fired plants and can only be related to the most recent gas turbine "off the shelf" plants.

Some of the correlations that the authors establish are tenuous. Among them is a derived correlation between the 'length of the dam wall and cost overrun'. This has got to be a case of 'illusory correlation'. I can accept that there is a correlation between cost overrun and unforeseen geological foundation conditions, but not 'length of the dam wall'. Again, this is evidence that in preparing the original paper, not enough research has been performed into actual project implementation in our industry.

To me the conclusions drawn also appear to be highly subjective. It seems that if one adopted a different focus for the 'cherry picking' that seems to have occurred, one could easily have drawn a conclusion from the analysis presented that all dams should be designed and constructed by North American engineers and contractors, since those implemented by this group do not show so much overrun.

Apart from these highlighted examples, there are other factors that Flyvbjerg and his co-authors do not appear to have taken into account. These include the unprecedented rise over the last 20 years in the implementation of dams around the world by mainland Chinese designers and state supported contractors.

Aside from the lack of appreciation in the paper of the ebb and flow of the global financial and economic climate surrounding these large infrastructure works, there is a historical perspective to the development of infrastructure that is missing. The developing world is trying to "catch up" as fast as possible with the economic development of Europe and the West. But we have 300 years start on them and we have experimented with many different ways of building the necessary infrastructure for our current comfortable existence using most of the world's resources and energy. Are we occupying the moral high ground in deciding - after we have all the infrastructure that we need for our enviable standard of living - that others should not also have access to power and water in abundance?

As highlighted by Andy Hughes of the British Dam Society, construction of dams in the developing world is essentially about developing the country - providing water, power, irrigation, flood control etc., in parallel with other basic infrastructure such as roads, railways, telecoms, that are essential to lift the general population out of poverty and starvation. The developed world should be trying to do (or assist in doing) this as fast as possible as a moral obligation and to reduce the world's population (birth rate tends to go down as standard of living rises) and wars.

There are many examples of infrastructure projects that - under modern accounting scrutiny - may have cost more than they should, or been larger than warranted, but which have provided almost ignored "support" to our modern life. A simple example is the Victorian sewerage tunnels in London which were designed and built by Sir Joseph Bazalgette for a population of 4M when London only had 2.5M residents. They have served the capital well (with improvements) for 150 years - supporting a population of 8M - and only now need major expansion.

In the hydro field I could point to Churchill Falls in Labrador which has supplied electricity for the same price per kWh for the last 42 years without any adjustment, even for inflation. Can the same be said of coal fired plants or gas turbines which last less than 25 years before major rebuild? Consider the development of hydro in China - imagine the greenhouse gases developed if China was precluded from developing hydro yet continued to try to provide electrical power to industrialise, and support its population.

Not all is bad about the paper. It does highlight the importance of more robust attempts to predict outcomes (for both cost and schedule) for major infrastructure. And clearly it is important to find out what caused the large cost overruns for the long tail events.
However, I can assure that authors that I (and my colleagues) have been implementing more and more probabilistic risk analysis on the estimate and schedule of our projects since the 1970s. We initially used the work of Professor Chapman at Southampton, and latterly software that can be used in conjunction with estimating software or scheduling software to account for variability in estimating, construction planning and unknown factors that precipitate claims such as unforeseen geological conditions.

Dr Andy Hughes, Vice Chairman of British Dam Society

All I can say is that this paper is based on highly questionable data analysis which results in a totally misleading article. The authors clearly know very little about the subject. I find their descriptions of engineers highly offensive and can hardly be regarded as those that one would expect from a professional academic promoting a balanced review of a subject. In my opinion the article has no value at all.

Roberto Eugênio Bertol, Basic Studies Superintendent, Intertechne, Brazil

"there is indication of projects that overrun original costs by 50 times but the authors don't identify them. "

The authors do not present the basic data of their study (ie project names, locations, original and final budgets) so it is not possible to verify the accuracy of this information that constitutes the basis of the work. For example, there is indication of projects that overrun original costs by 50 times but the authors don't identify them. However I am pretty sure that we, as a technical community responsible for estimating costs, could not be wrong on this scale. So I don't think this should be used to forecast future projects.

Although the article concludes that due to costs and budget overrun dams are not good investments, there is no investigation about the relation of cost and benefits. Even if we believe the costs to be higher there is no discussion about whether or not it is a correct economic decision to build large dams.

I am limited by my own Brazilian experience but over the past 12 years projects, which have mostly been developed by private investors, have consistently been delivered within schedule (normally around six months ahead of schedule) and within budget.

Jayaraman Punidhan, experienced hydropower manager, India

For greater understanding of the value of the benefits provided by a large dam project, we do need to understand the economic parameters. Construction costs usually account for 80% of the project cost and the main issues that can increase costs are geological considerations, scope changes in design and poor execution of contracts. The solution lies in extensive site investigation, appropriate design and the effective preparation and management of contracts.

It is vital that we have as realistic an estimate as possible (so that it doesn't seem expensive at a later stage) and adopt cost control measures during construction. But what we need to stress is that hydropower dams - whether small, big, mega, large or massive - are all a dire necessity for developing countries. We need to proceed with a proactive approach to development. Ignoring large dams would be like ignoring the much needed development of emerging countries.

Developing countries should embark on mega dam construction, especially for hydropower projects. Once built, mega dams can last for more than a century and the operating costs are very low. Yes, large upfront investment is required and there are a lot of financing issues, but you need to look at the benefits that the dam would bring. This includes years and years of cheap affordable electricity, when other renewables like wind only has a useful life of about 25 years, and fossil fuel projects would have long been put up for scrap.