Stress-testing to improve strategic decision‑making

Banking regulators remain focused on expanding and developing the range of stress-testing regimes across the globe to maintain stability, monitor emerging risks and avoid another financial crisis. Here, a forum of industry leaders discusses the evolution of stress-testing since the financial crisis and how it has been aided or hindered by the introduction of loss accounting rules, how it can engender improved decision-making and the impact of technology on its future

The panel

• Martim Rocha, Global Business Director, Risk Consulting, SAS
• Jo Paisley, Co-President, Garp Risk Institute
• Antoine Bezat, Head of Stress-Testing, Methodologies and Models, BNP Paribas
• Senior Risk Professional, European banking conglomerate 

How have stress-testing processes evolved since the 2007–08 financial crisis? Where have they improved most and where do challenges remain?

Martim Rocha, SAS

Martim Rocha, SAS: The European Banking Authority (EBA) has changed the stress-testing exercise a fair amount since its inception. There is more than one regime in Europe nowadays – a formal two-year exercise, the resolution plan, the Supervisory Review and Evaluation Process (SREP), and so on. And they keep adding to the list – this year they have begun a liquidity stress test. 

Directly after the financial crisis, the European Central Bank (ECB) and the EBA were anxious to ensure similar events wouldn’t happen again, introducing capital restrictions and stress tests. Now things are a bit more stable.

Banks realise that, if they have to pull together data and models for the regulators, they can use the same infrastructure for their internal stress tests and planning activities. Banks initially viewed it as a burden, but it is increasingly seen as a management tool. The regulators approve because it gives them an assurance of consistency.

In the US, where firms have a more mature infrastructure for stress-testing with the Comprehensive Capital Analysis and Review (CCAR), the regulatory impact has been less as the framework has been more stable. For other regions, stress-testing was new so the upheaval was greater. 

Cost and time remain issues. Stress-testing including balance sheet forecasts means a lot of things must be brought together – volumes, capital, margins and costs – to be able to project profit and loss (P&L) and capital ratios. Many people are involved and much time is spent between various handovers and iterations.

Even within mature markets, the same problems still arise, such as inconsistent data – particularly for tier one and tier three banks. Tier two banks have the data story better figured out – their size is more manageable and they have sufficient resources to control their systems. 

In Europe, each EBA regulatory exercise changes from the previous one, so banks have to rethink their logic, methodologies and models – this is a constant burden. 

Jo Paisley, Global Association of Risk Professionals (Garp): Bigger firms have improved hugely, particularly in areas where there’s been stability in regulatory requirements – such as with CCAR. Banks have invested because it’s been worth doing so, and it’s become an important aspect of their capital planning, risk management and strategic thinking. That applies front to back: from having the right data to workflow processes and documentation. A lot has been automated, but it’s not merely a case of ‘pushing the button’. The tests are still difficult because of the need to make sense of it all and the role of judgement. But there’s an element of practice making perfect – the more tests banks perform, the better they become. 

In the UK, I’m sure the seven firms involved with the Bank of England’s (BoE’s) annual cyclical scenario exercise find it a lot easier now than they did first time. But the additional biennial exploratory scenario (BES) may be proving more difficult. 

The qualitative reviews the regulators conduct are useful in helping banks identify problem areas. Model risk management is one area the Prudential Regulation Authority (PRA) is keen for them to invest in, not just in terms of processes but also in ensuring board engagement and understanding.

Antoine Bezat, BNP Paribas: The importance of stress-testing in bank management has strongly increased since the financial crisis. The regulators played an important role in this process. The creation of the ECB’s Single Supervisory Mechanism has been an accelerating factor for European banks, with the EBA stress test becoming a key exercise to assess banks’ strengths and vulnerabilities every two years, and the central role of stress tests in the SREP.

Stress-testing has become a key strategic tool for risk management and capital planning. While stress-testing for market risk was already well established, credit stress-testing frameworks have been industrialised since then, leveraging large data infrastructures and state-of-the-art technology. The treatment of operational risk has also improved. 

On the other hand, many challenges remain. Business risk and pre-provision net revenue is a developing and demanding area. Likewise, climate risk stress-testing is a trending topic with lots of questions to be answered. On a global level, stress-testing involves significant model uncertainty. For that reason, BNP Paribas has built a bridge between the industry and academic research with the creation of an academic chair in partnership with École Polytechnique.

Senior risk professional, European banking conglomerate: We have a more structured framework now. Regulatory stress tests such as the EBA tests involve many participants – finance, risk, front office, and so on – and hypotheses and assumptions that need to be converted into shock in the system. There’s a more rigorous governance and process involved. Prior to the financial crisis, stress-testing was restricted mainly to the risk department. Stress-testing is also used more as a matrix of risk-taking and liquidity impacts, and there is a limit in front of it. Something we learned during the financial crisis is that simple Greek and value-at-risk (VAR) measurement were missing some out-of-the-money risk, whereas stress-testing done in a smart way can capture the non-linearity of your portfolio. 

The main challenge is that it’s not just plug-and-play. You can’t just switch the stress assumption across the bank and see what it looks like. Big investment banks tend to run on a mix of proprietary and vendor libraries, which aren’t always compatible for the purposes of the tests – I’m yet to see a top-down approach where you can see how a particular shock feeds through every system across the bank. Most work bottom-up, collecting and amalgamating the data.

 

Are today’s regulatory stress tests fit for purpose? To what extent have they become a box-ticking exercise for banks?

Jo Paisley, Garp

Jo Paisley: The banks that find this difficult are the big global banks that must perform multiple tests concurrently. A global bank might have to deal with CCAR, the BoE, the EBA and the Hong Kong Monetary Authority requirements – as well as many more – which are all on different bases, are unco-ordinated and often need to be run simultaneously. The level of complexity is so great that it’s hard to understand how they relate to each other. 

When faced with so many differing requirements, it’s inevitable some of them might feel a bit like a box-ticking exercise. You tend to prioritise what your home regulator requires – or the tests you can fail. That said, firms always treat regulatory tests seriously as they won’t want something inadequate going out the door to a regulator. 

Then there is the question of how much value the banks and regulators get from the tests. For a bank, it’s harder to get value from a test where the methodologies are not closely aligned to a risk perspective, such as a static balance sheet or one with caps and floors imposed. But, in defence of the regulators, they’re trying to run an exercise where they’re constraining banks’ discretion to ensure a level playing field – and that’s very difficult, especially across a large sample of banks. 

Senior risk professional: Regulators have a view of what the result of the test will be for a particular bank established through their internal challenger model. If the stress numbers are smaller than they expect, they are challenged. This is fine, as long as they take into account that the post-financial crisis wave of regulation has been effective. More precisely, most investment banks don’t do proprietary trading anymore and are properly hedged, so you don’t have massive positions open. In practice, European banks tend to have conservative VAR and limited market risk stress test.

It isn’t just box-ticking, as many people place a lot of value on the published results – including analysts and the media – so it has to be taken seriously.

Martim Rocha: In certain markets outside the US and Europe, regulations are still quite thin and have room to be more robust and detailed. In Europe there is SREP, the EBA stress test and now the liquidity stress test. So there’s a clear need to bring them together in an integrated exercise and avoid this scattered approach. In the US there is a more straightforward exercise but you might ask whether it goes far enough.

Antoine Bezat: Stress tests play a valuable role in the regulatory dialogue and in the setting of regulatory ratios, so the final outcome and the quality assessment are important for the bank. They also involve a financial communication component through market disclosure. They are, therefore, clearly not a box-ticking exercise.

Regulatory stress tests are constrained exercises where some simplifying assumptions are imposed to make the results more comparable between banks. It affects their informative content so the final outcome is not the most relevant in terms of risk management for the banks and for market disclosure.

For this reason, banks do not use the outcome of regulatory stress tests directly for their own needs. They develop their own stress tests where approaches are more risk-sensitive and avoid those simplifying assumptions.

 

Has the introduction of new loss accounting rules such as International Financial Reporting Standard (IFRS) 9 and Current Expected Credit Loss helped or hindered the evolution of stress-testing?

Martim Rocha: The first impact was to make it more complex, but in the long term it will make life easier and enable firms to monitor and manage provisions more effectively, strengthening the system. These regulations weren’t meant to be overly detailed – there was a lot of discussion around appropriate methodologies. Everyone was rushing to meet the deadline, which created certain inconsistencies in terms of approach. Now they’ve been in place for a while there’s a chance to look at what should be the best practice and make adjustments. 

Senior risk professional: The new accounting rules have helped reveal exposures that may have previously been hidden. The introduction of ‘other comprehensive income’ disclosures means that, if you’re in the banking book and you have a mark-to-market loss, you have to pass the value of that loss to the balance sheet. You can therefore stress the banking book efficiently.

Antoine Bezat, BNP Paribas

Antoine Bezat: It has mainly helped the development of stress-testing, for several reasons. With the introduction of the forward-looking component of IFRS 9 – where stress-test models are leveraged for accounting – banks have invested more in the development of models, encouraging more effort from those that were lagging. These models are also more established, audited and better understood within the banks. Also, IFRS 9 provisions are so volatile and procyclical that stress-testing is very useful to anticipate their behaviour.

Jo Paisley: IFRS 9 has already been incorporated into the BoE and EBA tests, but it makes you take a step back and think ‘What are we trying to achieve here?’ IFRS 9 is a forward-looking view of provisioning, and one of the reasons we introduced stress-testing is because accounting measures weren’t forward-looking. Are we trying to forecast the accounting balance sheet over the next five years or are we looking at the risk? And how are these views related to each other? IFRS 9 has certainly increased the complexity of stress-testing, but I’m not sure it adds to the usefulness.

 

How can banks drive greater efficiency in model development, benchmarking and execution?

Jo Paisley: The main trend now is using big data and machine learning, which can make it far more efficient to build models, but there’s a danger of overfitting. You have to get the right balance and keep your eye on the underlying theory. 

The modelling landscape is pretty complex, which can lead to inefficiencies in model build. I can understand how this complexity arises: when a new requirement comes in – such as IFRS 9 – there’s often a rush to meet the deadline, and banks have to throw resources at it to get it done. They don’t always have time to review and work out how this new requirement connects with other pre-existing ones before going ahead and developing models. It’s only after the event they then have to step back and work out these interrelationships.  

Some banks have looked at creating ‘uber-models’ – such as for a parameter such as probability of default that could be adjusted for different uses across different areas – such as IFRS 9, stress-testing and economic capital. It’s more efficient, but there’s a debate over whether it’s sensible and achievable. I certainly think it’s very confusing for members of any board to try to understand all the models and their interrelations, so any ways to show how the concepts are related are helpful and might also lead to greater efficiency. 

From a practical point of view, having a really good model inventory is essential. I’ve even seen presentations on how to embed some inventory code within a model to make it far more efficient to track how that model is being used and by whom. So there’s scope for technology to improve the efficiency of inventory management, which I think is currently often manual. 

Senior risk professional: There’s certain information that would be beneficial to be shared between banks. For example, the calibration and propagation of shocks is an industry-wide issue, and you don’t give away anything by revealing your shock in stress as a particular figure. 

Likewise, there could be value from shared insight around model development delivered via a third-party vendor. 

Antoine Bezat: Banks can drive greater efficiency through industrialisation and automation. With the growing recourse to machine-learning algorithms, it is now possible to automate a large part of the stress-testing process, from model development to documentation, backtesting, benchmarking and execution using Python or R libraries, for example, and big data platforms.

Martim Rocha: It’s an area in which banks are investing. Banks now manage a large number of models and are keen to make the whole model lifecycle more efficient – from the development process to benchmarking and deployment. Some banks have made good progress towards a more flexible infrastructure, which helps create synergies. 

Others are using machine learning to benchmark models, which helps them get to benchmarks quicker and more thoroughly. Model governance is also a big topic, with model risk gaining increasing attention. 

 

What influence is stress-testing having on banks’ strategic decision-making? How can banks maximise value from the process?

Senior risk professional: If you have a regulatory exercise, the main focus will be on compliance and on time delivery. In a big bank the process is very complex, so there is little space for a strategic view. But where it is valuable is in setting risk limits on particular business lines and deciding how much stress you are willing to tolerate for any given year – a month of budgeted P&L, a quarter of P&L and so on. 

Antoine Bezat: Stress-testing offers a holistic approach to risk monitoring, covering coherently all components of profit and loss, capital and liquidity positions. The link to one or more scenarios makes it simpler to understand for risk managers and board members than other risk metrics based on VAR, for example.

One way to maximise its value is through building integrated platforms incorporating IFRS 9 accounting, performance management, prediction, stress-testing, internal capital, reverse stress-testing, and the recovery and resolution plan. It then allows consistent exploration of different degrees of severity in the loss distribution from business-as-usual risk events to extreme scenarios, and to assess the benefits of portfolio diversification at these various severities of stress.

Martim Rocha: Banks recognise the benefits. The main one is in having an infrastructure that can be used to project the balance sheet under stress scenarios but can also be applied to planning processes. There’s also the business benefit. As you go through the stress-testing process you look at how well scenarios play out and gain insight on the business by testing different approaches and assumptions. 

Banks can choose to go through the process by themselves or get help from outside. Those who involve key stakeholders from the start embed best practice more effectively and derive more value from the process. Some of these exercises, such as credit, go to a very detailed level so banks can gain a great deal of insight into how their portfolios behave, the profile of their customers, and so on.

Jo Paisley: The best banks are trying to do this. The slicker their processes and the fewer different stress tests they need to do, the better. In my view, nirvana comes from driving closer alignment between stress-testing and strategic and financial planning. Indeed, a board shouldn’t sign off on any of these plans unless they’ve seen them under stress. But it has proven difficult to embed this in firms, often because the systems used for regulatory stress-testing are not necessarily the same as those used for all these other activities. Once firms get to the point where they can run stress tests on these various plans, it’s easier to start getting more value from the tools and processes. 

 

Many banks struggle to maintain adequate resources for stress-testing programmes – what can the industry do to address this?

Jo Paisley: Regulators are mindful of the burden. For example, some US regulators have recently streamlined their requirements and lowered the burden on smaller banks. The BoE and EBA are reviewing their approaches as well. But will it materially reduce the level of resource banks need? I don’t know. 

We now also have climate risk on the agenda. The PRA was the first to announce a climate risk stress test in its BES in 2021, and other regulators are likely to follow suit. I worry that climate risk stress-testing will go the same way as economic stress-testing – regulators worldwide having significantly different requirements. Climate change is the ultimate global issue, so it’s essential the industry works together to avoid a fragmented approach. 

Martim Rocha: A hybrid approach mixing external and internal resources is a good option. As well as capacity, external consultants bring the knowledge and experience from previous exercises that can prove to be very useful; these mixed teams will help develop the internal resources and the insights gradually embedded in the business. 

There are also opportunities in technology. We now have advanced systems capable of crunching numbers much faster and with greater insights. Technology is also helpful in providing a cohesive structure to manage the data, enabling more efficient handovers within the iterative process and providing an integrated detailed view of results.

Senior risk professional: In the same way that consensus emerged around the International Swaps and Derivatives Association’s standard initial margin model, a lot of time and effort could be saved if we had an industry-wide agreement on interpretation of the EBA’s or the Federal Reserve’s stress-testing requirements. 

Antoine Bezat: To maintain adequate resources, banks must improve in two dimensions. They must ensure they take the greatest advantage of their stress-testing infrastructure for decision-making, to justify the investment, while aiming to industrialise and automate processes to drive efficiency and control costs. A third way is potentially through mutualisation to gain economies of scale: mutualisation of various processes – internal and regulatory stress tests, as well as internal capital, reverse stress-testing, recovery and resolution plan, for example. And mutualisation between various entities of large groups, potentially sharing some parts of the process between different banks.

 

How will the stress-testing landscape evolve in the future and what role will technology play?

Martim Rocha: Stress-testing will move beyond a compliance-only environment to become an essential management tool. The emphasis will move away from just stress scenarios towards having an infrastructure capable of simulating the business in any type of scenario, benign and stress, to help with planning. It is comparable to Tom Cruise in the 2002 film Minority Report – banks will be able to instantly call upon relevant scenarios, portfolios and models to project their business and guide their next move.

Antoine Bezat: Technology will continue to play a role in this evolution. We are likely to see more technology and data-driven platforms for holistic stress-testing. These platforms, while very industrialised, will need to offer the flexibility to deal in a consistent and efficient manner with various types of exercises and address new challenges. Those include the assessment of banking profitability, which is not new per se, but is more of a challenge in the low interest rate environment we face; and climate risk, clearly a new challenge for the industry. 

Senior risk professional: The increasing availability of data, and developments in data mining such as deep learning, offer an opportunity to improve reverse stress-testing. With a smart statistical engine you could look at portfolio exposure to a particular stress event and explore more extreme market discrepancies. But banks today are extremely cost-constrained, which will hinder progress. This isn’t something you can do on the back of an envelope. 

Jo Paisley: It will be interesting to see how climate risk testing develops and what questions regulators will be asking – whether it’s impact on portfolios, physical vulnerability or something else. It’s great that the BoE intends to consult on this soon to establish the right framework and methodologies. Traditionally, a stress test looks at a big macro shock, but here we’re talking about physical and transition risks over decades. Banks rarely think beyond one to three years, so it will be a challenge. 

The other area I think will see an increase in focus is cyber risk testing – particularly in terms of the operational resilience agenda.

 

Stress-testing – Special report 2019
Read more

Sponsored content