The convenient untruth

Most operational risk professionals are aware that there is far more so-called qualitative data available through risk and control assessments (RCSA) than reliable quantitative data.

'So-called' because much of the data collected through RCSA is not truly qualitative at all. It is most often an estimation of quantitative statistics such as expected frequency of an event, expected severity, and information about the design and performance of controls.

The analysis of this qualitative data by quantitative methods represents one of the biggest untapped opportunities for the industry, and particularly for op risk professionals.

Modelling techniques are not only relevant for capital charge calculation purposes, but are valuable for all institutions in understanding their business and generating business benefits.

A structured and rigorous quantitative analysis of RCSA data will generate a number of benefits for the industry. For example, it could greatly facilitate the development of risk-based pricing and could help to promote a 'system' approach to op risks, as opposed to the 'person' or 'legal' approach that seems all too prevalent in the UK and the US.

As James Reason pointed out in Human Error: Models and Management, the 'person' approach seeks to control errors (risk events) by reducing unwanted variability in human behaviour, by creating procedures and applying sanctions to those that fail to carry them out. This is augmented by the 'legal' approach, which seeks by regulation to make individuals responsible for systemic breakdowns.

Sarbanes-Oxley requirements are the embodiment of the 'person' and 'legal' approaches - the idea that bad things only happen to bad people, what psychologists call the 'just world hypothesis'. This might be emotionally appealing, but does it enhance the safety of investors' money?

Now that the chairman of Northern Rock has resigned, perhaps we don't have to concern ourselves with the systemic issues that created the problem, or the legislation on market abuse and takeovers that "hemmed in" the Bank of England and prevented it taking more timely action . . . at least not until a similar problem arises.

The 'system' approach accepts humans are fallible, and will make errors, even in the best companies employing the best people. They are consequences of systemic factors, not causes. As Reason points out: "We can't change the human condition, but we can change the conditions under which humans work."

This approach encourages reporting - errors and near misses are reported and analysed within the context of a blame-free culture.

Most risk professionals would agree this is a crucial part of effective risk management, but are we totally confident that the legal, regulatory and management framework in which we operate encourages this in all cases? Perhaps the people queuing round the block at Northern Rock were telling us something we would be well advised as a profession to act upon.

Studies of 'high-reliability organisations' such as US nuclear aircraft carriers, nuclear power plants, and air traffic control centres have thrown up some interesting paradoxes that are relevant to our industry. For example, one of the most important safeguards to errors was found not to be a strict adherence to procedures but human variability, the ability to make timely adjustments to processes and to adapt to changing requirements. These organisations were able to make rapid changes locally at the centre of a potential problem, to allow experts to take temporary operational control. The success of this process was due to the high degree of shared agreement on objectives and goals.

These organisations also had a 'collective preoccupation' with the possibility of failures, and continually considered scenarios for errors that had not occurred previously.

The modelling of RCSA data can help facilitate this process, by mining the intellectual data prevalent in every organisation through the years of collective experience and knowledge of its people. Given the apparent difficulties many institutions face in uncovering reliable risk event data, it seems entirely logical to focus more attention on the abundant intellectual data that is available.

The quantitative modelling and stress testing of RCSA data empowers op risk managers. The RCSA data collection is the beginning of the process, rather than something to be reported.

The analytical power is immense. Op risk managers can apply multiple 'what if' scenarios to RCSA data, to consider the sensitivity of their organisation and of individual business lines to changes in risk and control profiles.

The results can also be stress-tested to analyse the loss sensitivity of changes to individual data points, such as the effect of the degradation of a control on multiple risks across multiple business lines. The risk profile can be simulated to exclude certain controls, or to include controls that are not yet operational, or to investigate the time sensitivity of the organisation to either an increase in risk or a fall in control effectiveness. For example, an op risk manager could consider the impact on a business if risk frequency or severities increased by 10% while the effectiveness of certain controls fell by 10%.

New business lines can be modelled from an op risk perspective to investigate their likely impact, controls can be analysed for value and whether it is beneficial to re-allocate resources to other areas of the control infrastructure.

The modelling of RCSA data can give the operational manager an analytical 'playbook' to uncover information about an organisation's risk profile that is not readily apparent, to consider and analyse multiple 'what if' scenarios to improve the decision-making process, driving significant business benefits to the organisation, its customers, and to the financial services industry. These types of analytical tools are taken for granted in other risk disciplines; they should also be available as a matter of course to op risk professionals.