Op Risk Benchmarking, round II: helping lenders borrow

From KRIs to four-eye checks, how do op risk frameworks at regional and domestic banks stack up?

Credit: Judy Stevens, nbillustration.co.uk

Welcome to round two of Op Risk Benchmarking, a new service from Risk.net that scrutinises operational risk practices at a range of financial institutions.

In each round, a group of participants provides data on how they manage five risks, selected by their peers. For each risk, participants give the same set of information, covering nine aspects of the risk framework, from staffing and key risk indicators (KRIs) to modelling and scenario generation.

Eleven global systemically important banks (G-Sibs) participated in the first round of the exercise. This time, we have data from 12 regional and domestic banks that are headquartered on five continents. The majority are large commercial and consumer lenders that are also active in a mix of other businesses – markets, asset management and insurance. A few have narrower business sets, and one is a development bank. The asset base for most is measured in the hundreds of billions.

The aim of the exercise remains the same, as do the questions, allowing for comparisons to be drawn between the two cohorts. One thing that jumps out: these mid-sized banks appear to be more similar to each other than the G-Sibs were to their own peers. On a range of points, there is more of a consensus. For instance, they are more willing to review and rotate their governance, risk and compliance software vendors, and also to report on how they're doing to internal and external stakeholders.

Risk reporting tends to be more uniform across risks, both in content and delivery: almost all banks report on their top risks to business units, senior management, and the board. Those board packs are largely made up of qualitative discussions of risks, supplemented by KRIs; only a handful explicitly include loss and scenario data.

With lower capital burdens overall, risk modelling is also less prevalent: only two banks report using the advanced approach, and both are in the process of switching to the new standardised approach ahead of the looming phase-in of Basel III.

Perhaps this hints at one of the reasons for the greater similarity between the non-G-Sibs: simpler organisations can run simpler frameworks, implementing them more consistently, changing them more rapidly.

Highlights from each of the five benchmarked risks can be found below. The full article for each risk can be found here.

Cyber risk: information security

Click here for full article and analysis

Banks demonstrate a reasonably tight range when it comes to the size of second-line teams that have responsibility for infosec risk, although the extremes are some way apart, with the largest team cited as 20 and the smallest a one-man band. The mean average team is nine. and a plurality have teams of 10.

Some banks still feel underweight: four out of 10 have added infosec specialists in the second line in the past 12 months, with three planning to continue. Compare that to teams charged with managing the threat of cyber-led IT disruption (see box: Cyber risk: IT disruption), where, despite some direct overlap in responsibilities, team sizes are more static, or even declining somewhat – perhaps a consequence of greater reasonability being pushed onto the first line, some suggest.

Two banks out of 11 do not maintain a dedicated second-line infosec team – something one of the two acknowledges makes them “a bit of an outlier”.

“We don't have a defined 1.5 line of defence in our bank. Looking at peers in the survey, it seems like most do. That's just not something we've had – [but] it is something we've discussed,” says a senior op risk manager at the bank.

Op Risk Benchmarking builds on Risk.net’s annual Top 10 Op Risks, which for the first time this year was broken into four cohorts – G-Sibs; other banks; financial market infrastructures; and asset managers and insurers – creating a separate top five list for each. These lists are the basis for the benchmarking work with each of the cohorts.

Cyber risk: IT disruption

Click here for full article and analysis

Regulators took a keener interest in the risk of IT disruption during the past 12 months than any other risk type surveyed – and they were more interventionist in their approach, banks reported. Given rapid changes in the cyber threat environment – and regulators’ low tolerance for any disruption impacting retail customers – that is perhaps unsurprising.

Two-thirds of banks said their regulator had shown an increased focus on this risk type over the past 12 months. Almost all of these banks had been subject to enhanced reporting, the most common supervisory tool deployed alongside closer monitoring.

One bank had been asked to hold more capital against this risk type, as well as making additional Pillar 3 disclosures. This bank did not provide further details, although it did mention recent actions to combat unspecified “third-party non-compliance”.

Combined with the reputational damage that often results from platform outages, it’s not hard to see why banks set such a low tolerance for this risk type. None had any plans to change their stated risk appetite for IT disruption, but half reported being outside tolerance during the preceding 12 months.

Highlights from our second round of benchmarking can be found in the boxes contained in this article. Deep-dive reports will be published here over the next couple of weeks, focusing on each of the five risks in turn: cyber risk in the form of information security (infosec) was the top concern by some margin for non-G-Sibs, with cyber-triggered It disruption coming second. The top two risks are followed by change management, execution and process errors, and regulatory compliance risk.

Firms were free to participate in as many or as few sections as they wished. The respondent count will therefore be shown for each figure in each of the five deep dives.

Change management

Click here for full article and analysis

Is this a standalone risk type, or a driver of other forms of risk? In the interviews that followed the benchmarking survey, banks offered a mix of views on this fundamental question, and this clearly translates into their frameworks – some have dedicated KRIs, some do not, and only three model the risk.

As the head of operational risk at one participating bank put it: “While it clearly is a risk to manage, it’s not its own level-one risk. It’s a risk within other risks.”

The ambiguity was also evident in data on team sizes and resourcing plans. Four of the eight participating banks have a second-line team with specific responsibility for change management; the other four do not. Two banks plan to significantly add to their second-line teams in the coming 12 months.

Still, almost all non-G-Sibs report on this risk to the board and senior management, with the latter group frequently acting on those reports, and taking the lead in demanding enhanced controls and more concrete steps toward monitoring.

Responses were gathered during the second and third quarters of 2023. Answers were then cleaned and aggregated, and used to build a picture of current practice for each of the cohort’s top five risks. Follow-up interviews offered insight and supplementary analysis.

The full datasets are available to participants in the exercise. Subscribers have access to selected highlights and commentary, comprising a fairly detailed snapshot. In the months ahead, we will produce reports on the data collected from each of the other cohorts about how they manage the five risks selected by their own peer group.

Execution and process errors

Click here for full article and analysis

Given the theoretically infinite array of things that can go wrong in the ordinary course of business, scenario analysis is a vital tool in the armoury of risk managers looking to get a handle on their exposure to execution and process errors. Seven out of 10 banks say it is their primary method of measuring tail risk for this category, and that it features in group-wide scenarios that are used for risk capital setting. Two banks reported losses during the prior 12 months that exceeded worst-case scenario estimates.

That makes the question of what goes into those scenarios – and who gets to decide – a vital one. More than half of banks reported having refreshed their scenario libraries for this risk type in the past year as part of a periodic review. Four said refreshes had been driven by changes in the business environment, while the same number (though not the same banks) said changes were risk-driven.

Almost all banks include their head of op risk in review meetings, along with representatives from the business. Five also invite first-line risk owners to participate – the same number (although for the most part not the same banks) who have senior management in attendance. Three banks invite their internal auditors along, two of which – along with one other bank – also include other members of their risk committees in meetings.

Only two banks have their chief risk officers present at meetings, and just one formally includes model validators. Among G-Sibs, there was also an outlier bank that subjected scenarios to review by model validation teams.

Regulatory and compliance risk

Click here for full article and analysis

Of the eight banks that benchmarked themselves for this risk type, five attempt to model their exposure. Some participants see that as a missed opportunity for the non-modellers, citing the wealth of data at the industry’s disposal.

There were also missed opportunities for the modellers, however: all five apply their quantification skills to the setting of regulatory capital – but often to little else. Three banks use their modelled numbers as the basis for management decisions. One also uses it for the setting of risk appetite and in stress-testing.

That’s not enough for some, who urge the industry to use models – simple ones, where possible – to help run the business and manage risk. “We're looking at a new model currently, and it's just ludicrously overcomplicated. Actually, we should all just really try and use it as a tool to help drive things like management engagement, risk management and things like stress-testing that are far more tangible and understandable,” says one op risk head participating in the exercise.

Risk.net hopes the information will be helpful to a discipline that has grown up rapidly in the past two decades, but often lacks clear standards and best practices.

If you want to see the full datasets, you’ll need to take part in the next benchmarking round. Message us for details: ORMBenchmarking@risk.net

Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.

To access these options, along with all other subscription benefits, please contact info@risk.net or view our subscription options here: http://subscriptions.risk.net/subscribe

You are currently unable to copy this content. Please contact info@risk.net to find out more.

You need to sign in to use this feature. If you don’t have a Risk.net account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an individual account here