It has become increasingly difficult to predict how the United States’ energy system will evolve, leading to a growing number of projection errors in the Energy Information Administration’s (EIA) Annual Energy Outlook (AEO) and other analyses, according to researchers at Carnegie Mellon University.

“We find that both unpredictability and volatility have been larger in the past decade than in the two and three decades before it,” the researchers said in a study published in the journal Nature Energy.

Retrospective analyses, including the AEO, “raise the question of the extent to which insights into past errors can help predict future errors. In short, will the future be as difficult to predict as the past was? We note that AEO’s own low and high oil price scenarios began to widen substantially in AEO 2006, suggesting higher uncertainty in at least that quantity…

“The stated goal of the AEO is not to forecast the future, but to project the likely development of the U.S. energy system under the policies in place at the time of the study, and assuming there are no major technological breakthroughs. Still, if the AEO is to guide decision-making, we believe it is important to characterize its historical prediction accuracy.”

Unpredictability, measured as “the frequency of extreme errors in AEO projections,” has increased over the past decade, according to the researchers. Those errors were largely under-projections for prices and inflation and over-projections for energy production and consumption, they said.

All over-projected extreme errors in the 30-year study period occurred in 2005-2014 for 10 quantities: production and consumption of natural gas and coal; oil production; electricity sales; total, residential and transportation energy consumption; and gross domestic product.

“This means that the largest over-projected errors in this period for the short-, medium- and long-term for these quantities were larger than any seen in the previous 20 years,” according to the report.

Long-term projections for shale gas and tight oil production in EIA’s most recent AEO, which was released last month, were questioned by some environmental groups.