Journal of Risk Model Validation
ISSN:
1753-9587 (online)
Editor-in-chief: Steve Satchell
A dual backtesting framework for quantifying nested model error and unlocking capital efficiency
Need to know
- Nested model errors from macroeconomic scenarios are a major source of capital misstatement.
- A dual backtesting framework (single and double blind) is essential for robust validation.
- Isolating these errors enables a strategic shift from costly conservatism to precise capital efficiency.
- Mastering this analysis turns model risk management into a competitive advantage.
Abstract
Macroeconomic scenario forecasting serves as a cornerstone for regulatory capital exercises such as Comprehensive Capital Analysis and Review (CCAR), Current Expected Credit Loss (CECL), International Financial Reporting Standard 9 (IFRS 9) and the Internal Capital Adequacy Assessment Process (ICAAP), and it introduces a critical and frequently overlooked problem: nested errors. These errors originate from third-party scenarios and the internal modeling process, and they compound nonlinearly as they move through pre-provision net revenue and credit loss forecasting models. This compound result yields distorted capital projections that can lead to either wasteful overcapitalization or dangerous capital shortfalls, highlighting the need for a robust solution. To address these issues, this paper proposes a dual backtesting framework: single-blind backtesting evaluates core models, while double-blind backtesting assesses the entire system. The difference between the two tests can be used to quantify the error from the macro scenario forecast in pre-provision net revenue and credit loss forecasting models. This methodology provides a comprehensive solution adhering to the “use test” principle in US Federal Reserve Board Supervision and Regulation Letter 11-7, facilitating a more profound comprehension of its application. Through the precise quantification of these nested errors, financial institutions can make efforts to minimize errors driven by upstream macro components, enabling them to transition from a strategic approach of blunt conservatism to a more nuanced and accurate calibration, thereby underscoring the strategic significance of this methodological shift. This transition should unlock significant capital efficiency, enhance profitability and build stronger regulatory trust. Our novel framework establishes a new standard in model risk management by proving robust validation and established guidelines for improving accuracy, providing a decisive strategic advantage.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@risk.net
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@risk.net