Range of practice in operational risk measurement

The lack of a commonly accepted benchmark model is a handicap for the operational risk industry. Marcelo Cruz discusses the areas where industry practice still diverges, and ways to bring it together

It is common in risk management industry gatherings to hear that operational risk measurement as a discipline is still in its early stages. This is to a certain point to be expected since Basel II was signed only eight years ago and has still not even been officially implemented in the US. However, compared with market and credit risks it is taking considerably longer for operational risk measurement to take off as a mature discipline.

As a comparison, in market risk the first value-at-risk model (RiskMetrics) was published in the early 1990s, and just a couple of years after its release 99 out of the 100 largest global banks had implemented such models, to the point where regulators were able to issue the Market Risk Amendment to the Basel Accord in 1995, allowing banks to use VAR models for regulatory capital purposes.

Credit risk also had similar advantages with JP Morgan (CreditMetrics) and Credit Suisse (CreditRisk+) releasing details on their own internal models. Operational risk has had no such luck. A few banks published some outlines of their OR methodology, for example Deutsche Bank ("LDA at work: Deutsche Bank's approach to quantifying operational risk") but these articles never had much impact.

Without a clear benchmark, banks developed their models in isolation and, for this reason, the difference in operational risk measurement methods among banks can be significant. Given this state of affairs, one of the top interests in the operational risk community is to try and benchmark their in-house developed models with ones from other industry participants. For this reason, banks and regulators often hold closed meetings to discuss their approaches more generically and try to benchmark their measurement frameworks. Regulators have the advantage of being the only entity that actually sees the models used in each bank in detail, and from time to time try to provide some view of the range of models and techniques used. There are also a number of industry surveys that provide a good picture on where firms are with regards to operational risk measurement.
There is now some degree of convergence amongst the industry as most firms use the Loss Distribution Approach (LDA) as a base for their models. An absolute majority of the large firms calculates frequency and severity distributions and then aggregates them to get to a final operational VAR. The difference lies in the use of the inputs into the model. Some of the key differences worth mentioning are:

● Use of scenarios: The use of scenarios presents one of the biggest divides. This divide is largely geographical: in Europe scenarios play a bigger role in the measurement framework than in the US. Inputs coming from expert opinion and transformed into severity distributions are the key drivers of operational risk capital in most European banks. In the US, many banks use scenario information only as a benchmark or for post-model capital adjustments, although some may still use it in their measurement framework. There are some other creative uses of scenarios, such as, for example, a tool to select the best severity distribution. Many firms are also making the transition to using scenarios more of a management than a measurement tool.

● Use of external data: Most firms use external data as a direct input to their capital model, particularly in the US, and filter the data by only using losses from business lines that match their own product suite as inputs to their own databases. There is, however, some variation on how this data is used. Some firms use internal data for the body of distribution and only external data for the tail while others use both internal and external for the tail. Quite a few firms pool together internal and external data only if they pass homogeneous tests (like two-sample Kolmogorov-Smirnoff, for example) or other statistical analysis. Some use external data only in units of measure where internal data is insufficient or just for correlation modelling. Almost no firms scale external data (as information to do so is not available).

● Use of business environment and internal control factors (BEICFs): There is quite a bit of interest in this input, particularly in the US, due to the Comprehensive Capital Analysis and Review. A few banks are using BEICFs like key risk indicators to adjust capital up or down after the calculation. Another few are starting to estimate frequency parameters using Poisson or negative binomial regressions. Expect progress on this front in the near future.

● Number and definition of Units of Measure (UoMs): The variance here is tremendous and very idiosyncratic. The ideal situation is that the UoMs are homogenous but making this happen and even testing for this homogeneity is not an easy task.

● Combining inputs into a single measure: There is significant variation here. Some banks have separate VAR models for each input and then combine capital; others combine inputs and then calculate the operational VAR.

Although most banks are using LDA as the base for measurement, the lack of generally accepted practice makes the job of supervisors very difficult as they have to understand the idiosyncrasies and creativity of each individual bank model in the validation process. The industry and regulators need to work closer to try and make operational risk measurement more standardised across banks.

Marcelo Cruz is the editor-in-chief of the Journal of Operational Risk and an adjunct professor at New York University.