Beware the hype around analytics

As energy traders make greater use of big data, lessons of the past should not be forgotten


Big data is coming to the energy business. And it's coming in a big way.

Over the past year or so, energy firms have significantly stepped up their use of data analytics to squeeze greater profits out of their trading activities, consultants and market participants say.

The reasons for this are twofold. On the one hand, the greater availability of energy-related data and increased computing power have made it possible to develop detailed models and run simulations that were unthinkable just a few years ago. On the other hand, slumping profits have incentivised companies to boost efficiencies. Whether you're a refiner, pipeline operator or commodity merchant shipping cargoes of fuel across the seas, you don't want to leave money on the table by failing to monetise the optionality inherent in your assets and contracts.

But how much value can companies really extract from analytics? I spoke to a former physical oil trader from a Wall Street bank, now largely retired from the business, to hear his perspective.

The former trader conceded that data-driven insights had utterly transformed once-opaque markets. For instance, many years ago he used to visit shipping agents in places such as Ventspils in Latvia, and Novorossiysk, Russia, to learn when tankers were set to come and go – sometimes encouraging the agents' cooperation with a bottle of vodka. That information would help the bank's traders develop a picture of global oil flows.

Nowadays, anyone with a Thomson Reuters Eikon screen can pull up a digitised map that uses ship-tracking data to show where oil tankers are located at any point in time.

But the former trader says it's still hard to extract actionable information from the huge amount of data out there. Worse, having access to that data and to fancy analytics tools can breed overconfidence in one's modelling. "There are limits to how useful it can be and what kind of people can take advantage of it," he says.

History offers plenty of examples in which faith in quantitative models led to disaster. After Hurricane Katrina hit the US Gulf Coast in 2005, for instance, certain US hedge funds reportedly lost in excess of $250 million due to positions that would have profited from narrowing spark spreads – the differential between the price of natural gas and the price of electricity produced from that gas. The trades were underpinned by expectations around the correlations between power and gas prices. But when, in the wake of Katrina, gas prices spiked and electricity prices didn't, the hedge funds were hammered with steep losses.

Admittedly, the new wave of analytics doesn't seem to be driven by firms seeking to place large, speculative bets. There's a big difference between a refiner conducting rigorous analysis to figure out the optimal facility to which to ship a cargo or crude and a hedge fund crunching numbers to develop a trading strategy for US natural gas markets.

But the same lessons still apply: it is important to treat data-driven insights with a healthy dose of scepticism, either because of potential quality problems lurking in the data or the risk that one's model could be tripped up by unexpected events.

Companies pushing for ever-greater use of analytics nowadays should approach the task with humility. Or else big data could lead to big trouble.

Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.

To access these options, along with all other subscription benefits, please contact or view our subscription options here:

You are currently unable to copy this content. Please contact to find out more.

Chartis Energy50 2023

The latest iteration of Chartis' Energy50 2023 ranking and report considers the key issues in today’s energy space, and assesses the vendors operating within it

2021 brings big changes to the carbon market landscape

ZE PowerGroup Inc. explores how newly launched emissions trading systems, recently established task forces, upcoming initiatives and the new US President, Joe Biden, and his administration can further the drive towards tackling the climate crisis

You need to sign in to use this feature. If you don’t have a account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an individual account here