Lack of data strategy could cause costly mistakes

As demand for commodities data soars, firms need to bolster their data due diligence processes or risk losing years of competitive advantage, says ZE Power’s Aiman El-Ramly

Volatility and uncertainty characterise today’s commodities markets. As participants grapple with the impact of events arising from the Russia/Ukraine conflict, spiralling inflation and the energy transition, demand is soaring for accurate data and analytics. These are needed to provide insight into risk exposures and market opportunities. Changes in commodities market dynamics are being felt across the energy complex. Areas that were once reasonably predictable – such as the direction of liquefied natural gas (LNG) flows around the world, shipping rates, electricity supply and demand – now have many more variable considerations. Firms are now seeing data not just as something that will give them an advantage, but as something that is key to their survival.

With so much at stake, firms need to choose their data streams carefully. But it goes beyond just selecting high-quality data. Firms need a coherent data strategy that envisions the entirety of their business lifecycle. Key considerations are whether they want to maintain their own data warehouses or have data delivered as a service, whether to go on a public or private cloud, and whether to orchestrate their own analytics and integration or outsource it.

“There is a lot of due diligence needed around data,” says Aiman El-Ramly, chief business officer at commodities data management firm ZE PowerGroup. Since its 1999 launch as a provider of North American power market data, ZE PowerGroup has expanded its ZEMA product data coverage across the entirety of traded commodities, and now has the largest number of commodity data streams of any provider, says El-Ramly. Its reach and depth across commodities markets is reflected in this year’s Energy Risk Software Rankings, an industry survey in which end-users vote for their preferred commodities technology firms in a variety of categories.

ZE PowerGroup was voted Preferred data management system and also took first place for Prices and curves and Integrating with other systems and platforms.

El-Ramly talks with Energy Risk about the trends he’s seeing in data and technology for energy and commodities, and also the risks he believes firms will be grappling with in the commodities complex in the coming months.

Energy Risk: What are the biggest risks you see in the energy and commodities markets at the moment?

Aiman El-Ramly, ZE Power

Aiman El-Ramly: I think the next year or so is going to be very difficult for the commodities complex. The fallout from the Russia/Ukraine conflict – for example, sanctions – is going to be dramatic, and this creates enormous uncertainty. We are already seeing massive disruption in agricultural and energy markets, and I think there are going to be significant supply constraints to come. Geopolitical tensions, supply and transportation constraints, and high interest rates are going to make the next few months extremely tricky to navigate.
 

Energy Risk: How are current events impacting firms’ demand for data?

Aiman El-Ramly: There has been an enormous uptick in demand for data in recent months as people navigate the uncertainty of the energy markets. The Covid-19 pandemic obviously created a renewed urgency for better insight into what was happening on the demand side and also along the supply chain. And then today’s heightened geopolitical risk means commodities markets remain volatile and uncertain; delivery risk and selecting viable trade partners must be at the forefront of trade and risk operations. Anyone with market exposure needs to actively manage their positions, which requires accurate, good-quality data upon which to base critical risk-adjusted decisions.

On top of the political and physical challenges, firms are trying to navigate the energy transition. The transition layers a different set of data requirements, particularly looking at clean energy, credits and transition fuels. For example, the transition is making the role of natural gas and LNG more important. Companies are divesting certain assets and buying others, moving in and out of different markets as they plan out decarbonisation across an uncertain time horizon.

Everyone in the market now is trying to determine what their next move is. People who wouldn’t have been as interested in data before, because they felt there was reasonable stability in, for example, their shipping rates and their supply, did not believe they needed to actively manage or mitigate that risk. Now they want to manage those risks and they need data to enhance their understanding. We see a significant broadening of data demand as companies struggle to visualise the future in commodities trading.

Energy Risk: What impact is the energy transition having on demand for data?

Aiman El-Ramly: The energy transition is introducing more data sources. It doesn’t make the underlying job of getting data more difficult, it just makes it a bigger, more burdensome task. What is challenging with the energy transition is trying to understand your risk profile, which depends on the risk profile of your counterparties. Everyone is super exposed. There is opportunity too, but the underlying risk is certainly more prevalent.

The transition affects every step of the supply chain. Oil majors with, for example, production, exploration, extraction, production and refining, are asking themselves how long they can maintain their positions in these verticals. They may also be asking if the transition is another regulatory bubble that may eventually burst if the electrification objectives cannot be realistically or economically achieved in the next 25 years.

The integrity of the entire energy system is in question, as the transition will affect transmission, pipeline companies, shipping firms, road transport fleets and so many more. These industries are also trying to address their own carbon risk, looking at how they can move away from fossil fuels and perhaps embrace new technologies such as hydrogen.We also see this playing out in other industries as, for example, financial firms try to decide which firms to lend (or not lend) to, and how their investment and lending portfolios are being impacted, taking into account reputational risk as well.

Meanwhile, consumers of energy are trying to ensure they meet regulatory requirements around emissions as well as trying to understand and align themselves to longer-term regional renewable portfolio standards and emissions requirements, as well as global initiatives such as the Paris Agreement on climate change. As more sectors, especially transport, become increasingly reliant on electricity over fossil fuels, price exposure to the electric grid is going to increase.

Overall, if people are moving towards an alternative fuel mix, they need data to understand what is possible. Renewable energy sources are reliant on fundamental data, such as weather data, wind speeds and reservoir levels. Firms are trying to layer that fundamental data – which is really uncovering the availability of their renewable resource – onto the workings of the physical grid and the vagaries of the market. This analysis happens under enormous time pressure and constraints, and the engineering complexity is of the highest order.

Energy Risk: What other technology trends are you seeing?

Aiman El-Ramly: The movement to the cloud continues, bringing with it the question of whether to use the public cloud or a private cloud. A lot of firms don’t have the capability to build a private cloud. That said, if your software provider doesn’t have a private cloud, you then have exposure to the public cloud, which isn’t always ideal for high-volume data transactions. You tend also to be exposed to unforeseen costs in the public cloud. For example, costs around storage and recovery can be extremely high. People typically do not perform the cost analysis effectively between private and public cloud.

Firms also need to ask themselves whether the data store they put in the cloud will be their main, or only, data store. How will it be integrated into other mission-critical systems that may or may not be in the cloud?

But, if you decide to build your own private cloud then you take on the responsibility to ensure that it’s fit for purpose. Determining the proper architecture is something we see most companies struggle with. You need to consider how you will address such issues as permissioning, entitlement, security, firewalls and timely integration to multiple systems. Nobody wants to be the next Amazon Web Services or Azure data leak story.

Energy Risk: Is that the service ZE provides in the cloud?

Aiman El-Ramly: Yes, the ZEMA platform is in our own purpose-built cloud. Our software-as-a-service offering was introduced 20 years ago and our dedicated cloud offering introduced 15 years ago, so we are totally tried and tested.

Energy Risk: ZE ranks highly in the 2022 Software Rankings. To what do you attribute this success?

Aiman El-Ramly: People come to us because of quality – the quality of the data and service we offer and our ability to meet their full enterprise data management needs.

Our clients cover all parts of the energy and commodities complex across the entire supply chain and worldwide: power, fossil fuels, agriculture, metals, finance, and more.

I would say that ZE understands trade and risk very well. We can support any modelling requirements in the front, middle or back office. We understand operational analytics requirements, how to increase automation and ensure those analytics are available to downstream systems.

We’re a super good partner for anybody that has market exposure.

Additionally, the market has been very kind to us this year. Market participants are moving to an enterprise view of managing risk and this requires high-quality, timely data. We are seeing firms offload suppliers that can’t meet quality and timeliness criteria.

Then these firms are doing one of two things. Either they are seeking a capable provider such as ZE or they are building their own data links. But this still requires a firm to perform the aggregation, and there is not enough capability on the data aggregation side. Companies need to decide whether to outsource data warehousing and integration or to do it in-house. This is a critical decision that requires tremendous due diligence because trouble with data leads to trouble for the business. A company can lose a few years of competitive advantage on the data side by taking a misstep.

Another stand-out for us is our work around a firm’s end of day. We ensure firms have all the systems and data necessary for a timely end-of-day process, or intraday for that matter. That means extracting the right market data as soon as it is available and building the curves with very little latency. The process is super data-dependent. That is why it is important to collect the data as available, build the curves dynamically and use build-in redundancy in the curve building methodology so you are not exposed to any one data source, missing data or errant data.

Last year we refreshed all of our hardware, so our private cloud is now as state of the art as it is possible to be. It means we’re very comfortable with onboarding a huge amount of new customers, even doubling our client base, always ensuring very timely data and curve delivery.

Energy Risk: What are the latest trends in advanced analytics and data management? What results are coming from the use of artificial intelligence (AI)?

Aiman El-Ramly: AI and machine learning continue to be a little overrated. Useful application is limited in the absence of large volumes of data. As a data management firm, we have access to the large volumes of data required to apply machine learning and AI, and we might do this, for example, to optimise collection schedules, to understand whether there’s an error in a data stream or to identify gaps or likely gaps.

But that real AI/machine learning to support a trade or risk management decision is difficult. You can do it, but you need a huge data store that is continuously being added to and kept constantly up to date. AI/machine learning is not self-generating or self-sustaining; there must be an associated team that is always interacting with the algorithms and updating models or, very quickly, companies will be making poor and costly decisions. There are lots of areas for automation around data and I think it is more important to get them all in place before moving on to AI/machine learning. I am always sceptical of providers that offer the magic bullet of an autonomous AI/machine learning solution; that is probably why we don’t see many of our clients investing heavily in it.

Energy Risk: What enhancements and features are in the ZEMA road map?

Aiman El-Ramly: We believe we’ve built ZEMA to a very high standard, so now it’s a matter of ‘refining around the edges’. We want to move to shorter and shorter time granularities, advancing always towards real-time delivery. We are always improving on ways to visualise and share data through unique views and delivery mechanisms.

Everything we do is to ensure our clients get the best out of the data we deliver, either by using our tools to visualise, analyse and share, or by supplying data in native format into firms’ analytic, trade and risk tools.

Energy Risk: What are the biggest risks you see in the energy and commodities markets at the moment?

Aiman El-Ramly: We are super aggressive in building out our data catalogue. First, we go for geographic expansion. For example, if we secure a client in a new region, we always onboard some data to support them. Second, our partners in the marketplace are constantly introducing new products, and we work with them to ensure any new offering is available as quickly as possible.

We probably have around twice as much available data as our next three competitors combined, given our intensive data strategy and parser development approach.

We build out new data sources very, very quickly, adding dozens of new sources and reports a month. Our current data catalogue collects from 1,400 sources accounting for 14,000 data reports. We like to believe there is no deficit between what the client needs and what we will offer them in a timely manner. Our data implementation timelines are always measured in weeks.

See the full Energy Risk Software Rankings 2022 results

Sponsored content