Podcast: Montoro on FRTB thresholds and non-modellable risks
Senior risk manager also argues Kolmogorov-Smirnov test is better than Chi-squared
Adolfo Montoro, a director in the market risk management and risk methodology team at Deutsche Bank, visited our offices in London to discuss his new paper, The revised P&L attribution test and the suitability of new proposed thresholds, co-written by two of his colleagues, Marco Spinaci and Marc Georgi.
The Basel Committee on Banking Supervision published the latest consultation on its revised market risk capital requirements – the Fundamental Review of the Trading Book – in March this year, more than two years after the regulator published the finalised rules.
The FRTB requires each trading desk at a bank to individually pass a so-called profit and loss (P&L) attribution test in order to use the internal models approach, which is much less punitive than the fall-back standardised approach.
Following industry criticism that the test is extremely difficult to pass, even for well-hedged portfolios, the regulator introduced an 'amber zone' to allow for a smooth transition between pass and failure of the test, marked by the green and red zones, respectively.
In his paper, Montoro shows the thresholds for the amber zone are too narrow, by reverse-engineering the process used by the committee to calibrate the thresholds.
“When the latest consultation paper came out, we were interested to understand how those thresholds were calibrated in the first place…it took us a while to reverse-engineer those thresholds…but we clearly noticed the amber zone was very narrow and the red zone was quite big,” says Montoro.
The issue stemmed from what seems to be an incorrect assumption within the test, Montoro argues.
The P&L attribution test requires two different measures of P&L to be compared – the hypothetical P&L (HPL) generated by front-office pricing models, and the risk-theoretical P&L (RTPL), generated by the bank’s risk models. In the original proposals, the measures were compared in two different ways – one looks at the gap between the two using a mean ratio, and the other looks at the variance of that gap using a variance ratio. Both ratios should fall within certain established thresholds – if not, a breach is counted. Four breaches within any 12-month period will force a desk on to the standardised approach.
Our view is we are in favour of the Kolmogorov-Smirnov test, as the test has a slightly better ability to discriminate between different models used by different banks
Adolfo Montoro, Deutsche Bank
Following criticism that the tests are too sensitive, in its latest consultation, the regulator accepted industry requests to replace the mean ratio with the better statistical test of the Spearman correlation method. The regulator provided an option between the Kolmogorov-Smirnov and Chi-squared tests as a potential replacement for the variance ratio test.
The issue with the calibration of the two sets of tests are that they assume the underlying cumulative distributions of the HPL and the RTPL are independent – but they aren’t in practice.
“On reverse-engineering, which we justify in the paper, when you look at the usage of this Kolmogorov-Smirnov test and the Chi-squared test, they are parametric tests, but we shouldn’t forget the fact that they are used in the context which is not a natural context in which this test is supposed to be [used] – [that is,] when you test independent distributions,” argues Montoro.
The analysis by Montoro and his co-authors finds that the regulatory thresholds may have been calibrated using the p-values generated by the tests, which, given the invalid assumption, would not be the right way to go.
“In the paper…we move away from calibrating those thresholds using a p-value approach, but we’d rather look at an approach where we assess what a better model is…and then from there, saying let’s calibrate [the thresholds],” says Montoro.
Montoro and his co-authors instead calibrate the new thresholds by assuming what a good model is and what a good portfolio would look like as a benchmark, then build the thresholds from there.
“If I assume a good model should be characterised by this level of correlation, this level of bias and this level of under- or over-estimation [of risk], then from there, reusing the reverse-engineering process of how the thresholds should be calibrated in terms of intervals of confidence, the mathematical ground is very clear. And clearly, it has been shown there is a miscalibration, particularly for the amber and red zones,” Montoro added.
In the paper, the quants also argue why the Kolmogorov-Smirnov test is a better choice over the Chi-squared test.
“Our view is we are in favour of the Kolmogorov-Smirnov test, as the test has a slightly better ability to discriminate between different models used by different banks,” he said.
Index
0:00 Introduction
0:54 The P&L attribution test – outstanding issues
7:14 The amber zone
8:10 Paper in Risk
10:23 Kolmogorov-Smirnov test versus Chi-squared test
14:30 Are current thresholds set too narrow?
27:03 Calibration of thresholds
32:15 Assumption of independent distributions
36:45 Outstanding issues in the NMRF rules
41:16 Data pooling: regional fragmentation concerns
To hear the full interview, listen in the player above, or download. Future podcasts in our Quantcast series will be uploaded to Risk.net. You can also visit the main page here to access all tracks, or go to the iTunes store or Google Podcasts to listen and subscribe.
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@risk.net or view our subscription options here: http://subscriptions.risk.net/subscribe
You are currently unable to print this content. Please contact info@risk.net to find out more.
You are currently unable to copy this content. Please contact info@risk.net to find out more.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@risk.net
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@risk.net
More on Risk management
ECB official open to offering liquidity aid to non-banks
Risk Live: Deputy director doesn’t rule out copying UK plan to extend repo facility to pension funds and life insurers
Banks must loosen up on ChatGPT use – risk chiefs
Risk Live: ’Shadow use’ and inability to attract new hires mean restricting access to GPTs is untenable
Simm casts off Covid pain for $40 billion IM reprieve
Recalibration cuts risk weights in equity and commodities, but some credit exposures double on ABX halt
Rate risk modellers relieved as EU deposits stay sticky
Banks feared retail deposits would be flightier than during previous periods of rate hikes
Rough patch: CrowdStrike sparks an auto-update debate
Automating software updates helps keep hackers at bay but can introduce op risk; banks balance the two
Banks urged to keep regulators in the loop on AI plans for AML
Risk managers advocate five-year strategies and compliance teams’ ownership of the tech they use
Banks urged to boost third-party scrutiny amid AML crackdown
Three US regulators highlight deficiencies in banks’ due diligence on fintech partners
Clearing members welcome JSCC initial margin reforms
Stress loss add-ons touted as path to ensure defaulter pays and default fund contributions shrink