EIA Warns of Major Errors in its Drilling Data
Frequently faced with sharp criticism from industry and Wall Street for its estimates on everything from supply and demand to natural gas prices, the Energy Information Administration earlier this month publicly admitted there were serious errors over more than a decade in its gas and oil drilling data.
During a recent effort to revise its statistical procedure that generates estimates of actual well completions, EIA detected some "unusual patterns" in its results. "Discrepancies" were uncovered that turned out to be much more serious than first thought. For total wells by year, records were under-reported by more than 2,200 wells, 11% of the 1995 total. And the impact of the errors extended back in time to at least the early 1980's, EIA said in its March Natural Gas Monthly.
"Users of the EIA drilling activity data are therefore advised that the drilling activity data which were published or otherwise distributed by EIA prior to February 1998 are substantially in error," a note in the publication warns.
"I expect there were a lot of financial decisions based on [these incorrect numbers]," said EIA's Phil Shambaugh, co-author of the report that explains the errors. "That's why we wanted to dig in and make sure we got all of the wrinkles out of it and then publicize the fact that, hey, there have been problems with these numbers.
"Whether the [wells] are going up or down, somebody on Wall Street is going to look at that and say 'okay if the [wells] are going up that means that we're going to be increasing supply of domestic hydrocarbons and that will influence the market prices and so on. Or even they could say 'well that means we should invest more in drilling stocks.' I don't know what actual things happened, but its easy to posit a number of scenarios."
EIA said it has corrected the errors, however. The revised series shows some striking differences. It turns out the mid-1990s drilling decline was not as steep as previously indicated. Further in the past, the corrections show drilling activity did attain a peak level in 1981, but the peak was higher than first reported: 91,496 wells as opposed to 90,034. Gas and oil exploratory wells were greatly under-reported in the post-1985 period with more than half of the wells missing in certain years. And success rates measured as the share of successful gas and oil wells relative to the total wells improved greatly as early as 1986 rather than by the mid-1990s.
"I was troubled quite a bit by the difference in the success rates. That's the sort of thing that is so fundamental for our understanding of the industry and how it reacted to the economics of the late 1980s when the industry was going through some very hard times," said William Trapmann, an EIA economist and co-author of the report on the errors. "So our understanding of the impacts of economics as well as the adoption of alternate techniques to try to enhance industry performance - 3-D seismic was adopted in that period - [was not quite correct]. If you look at the success rates using the old data, it looks as though nothing really happened until just a few years ago even though we know anecdotally and otherwise that these technologies were being applied on a widespread basis..
"The data understated by a substantial margin the favorable impact of some of these techniques that the industry was applying to try to deal with the very severe conditions that they found themselves in," he said.
A Simple Misunderstanding?
EIA is quick to point out it doesn't collect the data. Houston-based Petroleum Information/Dwights has been supplying the data to EIA since 1995 for about $15,000/year. Prior to that, PI/Dwights provided the data to the American Petroleum Institute, which then shipped it to EIA.
In trying to avoid placing the blame on its only source of drilling data, Shambaugh attributed the errors to miscommunication particularly following the switch to PI/Dwights. He said "not fully understanding how the private companies were collecting the data and what each variable meant in its full entirety got us into this problem."
"It was just a misunderstanding, I think," said PI/Dwights Pete McClintock, manager of statistics. "I'm not so sure they understood that the data that they were getting, the data that we were selling to the API, was filtered by API specifications.. There were some wells that API didn't want delivered to them. For instance, they didn't want any workovers. They didn't want any wells with less than 50 feet of drilled footage. They didn't want what we called on our system pending wells, wells that didn't have complete information on them. EIA would have wanted those wells if they had realized they weren't getting them. I got the impression after I spoke with the DOE at length that they assumed they were getting all the wells.
"They apparently didn't look at what they got or understood what they got. I wasn't quite sure that these guys had a handle on processing this data and an understanding how this data really works," said McClintock, noting EIA's modeling methodologies are often criticized even by EIA personnel. EIA has to use an estimating model on drilling data because of the time lag in well completion reporting. Despite EIA's claims to the contrary, McClintock said he's not sure the problem has been corrected completely "or if it's a case of even the blind squirrel finds a nut sometimes."
While noting the good work PI/Dwights does, EIA's Trapmann said the biggest problems occurred simply because "we did not get all the records" from PI/D.
"I thought there was a programming error involved. I'm not sure what the misunderstanding would have been. We're not scheduled to get the recompletions, but that in itself would not have been the source of all these missing data. My understanding is that the processing that creates the file that is sent to us was somehow flawed.
"There were a couple months in the 1995-96 period in which is was something like half the records weren't present. That was the worst case. And that's not a misunderstanding about recompletions.
EIA said the problems would not have occurred over such an extended period had it done its own data collection all along. Using an outside source "does not allow complete familiarity with the data, and it hampers efforts to investigate concerns with the data. Second, the well completion data are inherently quite erratic, so even sizable fluctuations are not prima facie a cause for concern." And third, reported data are often inflated because they contain wells that were completed months and years prior to the reporting date. Perhaps worst of all, there is no other timely source for well completion data, EIA said.
But the time and expense of conducting well completion surveys make it unlikely if not impossible for EIA to do the job itself, Shambaugh said.
"We are certainly monitoring the data as we receive it and the numbers that we generate to try to make sure that if something unfortunate does happen then we'll catch it more quickly than this past problem," said Trapmann. "This is an unfortunate thing and kind of embarrassing, but it had to be taken care of. We're hoping we're on top of it now."
©Copyright 1998 Intelligence Press, Inc. All rights reserved. The preceding news report may not be republished or redistributed in whole or in part without prior written consent of Intelligence Press, Inc.