ETRM systems edge into cloud, but caution urged

Cloud could revolutionise ETRM systems but hurdles persist, according to survey

  • Cloud-based ETRM systems are attracting greater interest from market participants.
  • The benefits – from potential cost savings to flexible computing power – are becoming increasingly important in today’s market.
  • Many firms are still reluctant, with security and implementation concerns cited in a recent Energy Risk survey.
  • Analysts suggest some firms are waiting for cloud ETRM providers to develop the technology and create wider system functionality before migrating.
  • Energy firms urged “not to rush in until they understand all the risks” associated with cloud systems and the migration process.

Cloud-based computing offers considerable benefits to trading and risk functions, and interest from the energy sector is growing. But migrating an entire energy trading and risk management (ETRM) system to the cloud is fraught with difficulties, and is not an exercise that many energy firms have seriously considered – until now, say consultants.

“Around five years ago, cloud was almost not in the picture,” says Sunilkumar Ramakrishnan, a London-based associate partner in IBM’s energy risk management division. “Now almost every client wants to understand the benefits and how they would manage the security risk of ETRM applications in the cloud.”

In a recent Energy Risk survey – in which more than half of the respondents were made up of oil, gas and electricity firms, with consultants, IT specialists, banks, brokers, traders and investors accounting for the rest – almost 43% said they used some cloud-based applications (see figure 1). For the risk function, cloud computing provides some potentially revolutionary benefits, say market participants and IT specialists. One of the most important is the access it gives to exponentially larger amounts of computing power. And by effectively ‘paying as you go’ for that power, the cloud makes it possible to process terabytes of data for intermittent or one-off projects that would otherwise require warehouses of in-house servers far beyond the budget of an energy firm’s IT department.


In the current environment of relatively low oil prices and squeezed margins for oil, gas and electricity producers, gaining a competitive edge through proprietary analytics has become crucial. The cloud is enabling energy firms to take part in a fierce analytics arms race, as many market participants harness the expertise of data management companies in areas such as logistics, supply-and-demand data and forward curves.

“It’s one thing to manage 15 curves, another to manage 50 or 500 curves,” says Sidhartha Dash, London-based research director at risk technology research firm Chartis Research, owned by’s parent company Incisive Media. “A lot of energy companies are simply not equipped to manage the explosion in data and do not have the human resources to manage this. Many would be happy to manage it in the cloud rather than building their own infrastructure.”

In addition to special projects, the cloud’s increased computing power speeds up regular tasks such as end-of-day or large quarterly reports. Being able to achieve near real-time position reporting has become more important recently due to regulation requiring timely transaction reporting, as well as proposed regulation under both the US Dodd-Frank Act and the European Union’s re-worked Markets in Financial Instruments Directive (Mifid II) to introduce position limit regimes.

Being able to do more in less time frees up traders and risk managers to work on more proprietary projects and value-add transactions, say proponents of the cloud. Importantly, cloud-based systems are often more intuitive, allowing traders and risk managers to generate new reports without significant input from the IT department. This, combined with not having as many on-premise servers to maintain, frees up the IT department to work on more strategic operations or one-off pilot-type projects with individual departments or traders.

Cloud-based computing can also have lower upfront costs and quicker implementation times than on-premise systems, according to the survey. In a public cloud environment – where several companies share the same version of an application – upgrades can be sent out simultaneously to all users. While these environments are as yet uncommon, this is where the largest cost savings can occur. With traditional ETRM systems, upgrades can cost up to $2 million and need to take place every three years or so, but with a cloud-based ETRM provider, upgrades are often free.

Partial adoption widely embraced

But obstacles still stand in the way of full cloud-based ETRM usage. In the Energy Risk survey, only 13% of respondents said they use mainly cloud-based ETRM applications, with 44% saying they don’t use any at all (see figure 1).

David Campbell-Montgomery, Swindon, UK-based chief information officer at RWE Supply & Trading, says his firm has “embraced” the cloud recently, moving about 30 out of 150 applications to the cloud since mid-2015. He stresses, though, that these are non-critical applications. “We don’t have our main ETRM system in the cloud or any of our data warehouses or reporting, but we are looking into this,” he says.

This is reflective of the wider industry, consultants say. “For certain standalone solutions such as regulatory reporting and for some analytics, firms are happy to go in the cloud, but they don’t want to use it where there is core trading or proprietary data,” says IBM’s Ramakrishnan.

He gives a recent example of an oil and gas major that wanted to get up and running quickly with a commodity trade management system in a remote country. “They didn’t need a lot of functionality, they didn’t want to spend months on implementation and they didn’t have a large budget,” he says. “In that scenario it was easy for them to go with the cloud-based ETRM solution.”

But moving a firm’s central ETRM system into the cloud is a very different proposal. “Risk systems are at the heart of a trading business and typically have a number of connected systems integrating market data, trade data, logistics, back office and business intelligence systems,” says Baris Ertan, Houston-based managing director and global trading and risk lead at consultancy Accenture. “Our clients are going through assessments to identify the optimal approach for migrating all these systems together, factoring initial investment, cost to operate and IT security for the entire system landscape.”

These assessments often do not reveal the cost savings firms need to make the move, and especially so if not all departments are on board with making the move, say consultants. “When clients go through each element of benefit and cost, many don’t see a huge amount of cost savings compared to on-premise systems,” says IBM’s Ramakrishnan. “If the difference is only a 10% saving, that probably wouldn’t be a big enough incentive to justify the move and the potential loss of functionality,” he adds.

RWE’s Campbell-Montgomery agrees. “I think everyone’s struggling a bit at the moment with the cost model,” he says. “While there isn’t such a big upfront investment with the cloud compared to an on-premise and owned data centre, operating costs can still be significant. So far, our forecasted costs have been lower than what we’ve actually incurred.

“Cloud initially is not about cost-saving, in my view. You only really get actual cost savings if you move everything to realise scale advantages and also decommission legacy hardware.”

He sees it as more about moving onto standard platforms where you can deploy quickly and be able to flex capacity as required. “For example, if you have a huge quarterly credit risk report to run on the last day of each quarter, you can sanction new or additional hardware to run that quicker,” he says. “But the network cost of that additional capacity hasn’t really been worked out yet. For example, if you need to send the results – which could be an enormous file – back over the network to be processed somewhere else, it will increase your costs.”

Having to send huge files from cloud-based applications to on-premise systems is an inevitable result of the fact that energy companies are not going to leap wholesale into the cloud. Because ETRM systems are at the junction of so many different functions, they are bound to need to link to on-premise systems for years to come, consultants say. And there is not yet a comprehensive range of ETRM functionality in the cloud. In the survey, 40% of respondents said they hadn’t adopted cloud-based applications because those currently available didn’t cover all their needs (see figure 2).


“One of the main reasons for not adopting cloud more broadly is that many of the ETRM vendor solutions are still in early ‘proof of concept’ stages rather than operationally ready to be deployed into production,” says Accenture’s Ertan. The ETRM systems built specifically for cloud deployment, such as those developed by London-based Aspect Enterprise Solutions or Minneapolis-based Open Access Technology International, are expanding their range of cloud-based ETRM services fairly rapidly, say consultants. So far, though, web-based coverage is confined to a much smaller sphere than the on-premise offerings of the leading ETRM vendors such as New York-based OpenLink, Dallas-based Allegro and Florida-based FIS.

According to IBM’s Ramakrishnan, some energy companies are waiting for cloud providers to develop their technology. “Firms don’t want to take the risk of having to shut down a cloud application because the functionality hasn’t developed in the way they need it to,” he says.

In the survey, 37% of respondents quoted security concerns as a main obstacle in adopting the cloud (see figure 2). IT specialists maintain it is not the cloud platforms themselves – provided by firms such as Microsoft Azure and Amazon Web Services – that pose security concerns, but rather that security risks lie with moving a system not built for the cloud into the public domain. “Microsoft Azure and Amazon have very high security controls,” says Chartis’s Dash. “It’s not that people will be able to hack the public cloud. Rather, it’s that when a firm moves its internal systems to the cloud, it exposes end points that weren’t designed to go into the public domain,” he says. “Most [current] internal systems are not geared to talk to insecure end points and some internal end points may be insecure or may make assumptions of trustworthiness that become invalid when that system moves outside the firm’s security perimeter [and onto the cloud]. Virtually all the problems we’ve seen are rooted in this issue.”

Dash urges caution, emphasising that security breaches at an energy firm can be particularly costly, and that understanding the process of cloud migration is key. “At this stage, firms may not know enough to understand where the weak points might be,” he says. “I think it’s wise for energy firms not to rush in until they understand all the risks.” He adds that hackers infiltrating a power company’s network could inflict real-world damage. “The risks facing energy firms are very different to those facing social media providers or even financial services firms,” he says, given the nature of their physical assets. 

Security around cloud integration is improving, and linking different systems within the cloud is getting easier, says Bill Bucy, a Houston-based director at MRE Consulting. “The first generation of cloud was a point solution, where it solved one particular problem and was not really connected to anything,” he says. “The next generation has the same integration layer that companies would have if they were building on their own infrastructure. That proof of concept is out there now and people are building on it.”

Integration costs will come down once entire systems eventually migrate into the cloud, consultants say. That might not be in the distant future, says IBM’s Ramakrishnan. “There has been a move in recent years to move the back office to the cloud. In 2016, most back-office enterprise resource planning platforms were still on premise. Starting last year, many of the new implementations are looking at cloud.”

As more systems move into the cloud, the process will become simpler, he says. “As the ecosystem moves more and more to the cloud, ETRM systems will more easily be able to move to the cloud with better cost benefits. This trend could accelerate very fast.”

Big ETRM vendors will not be able to operate like the smaller specialised ETRM software firms and issue upgrades to all users at the same time in the foreseeable future, says MRE’s Bucy. “I don’t envision all customers on the same version of a product from OpenLink or Allegro any time soon, as there is so much customised logic.”

Still, things are starting to become more standardised, he says. “ETRM systems today have broader and deeper sets of capabilities and are much richer out-of-the-box than ever before, so it’s less attractive to customise and quicker to implement. But the idea that one customer has the same [ETRM system] as another is far from the truth. I think the trend is pushing towards that ideal state, but it is years away.”


A gradual migration is expected to continue. In the survey, a significant 66% of respondents said they would like to use more cloud-based CTRM applications (see figure 3). “We expect to see a significant adoption of cloud over the next 18 months due to the cost savings and security capabilities of the leading cloud providers,” says Accenture’s Ertan. “The ETRM vendors that have traditionally only offered a cloud-based platform are ahead of the curve and putting more focus on expanding their end-to-end functional and commodity footprint, while the traditional ETRM vendors are heavily focused on enabling their full suite to operate in the cloud.”

Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.

To access these options, along with all other subscription benefits, please contact or view our subscription options here:

You are currently unable to copy this content. Please contact to find out more.

You need to sign in to use this feature. If you don’t have a account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an individual account here