
A tale of two (or three, or four) models
Performance measure based on quality of replicating portfolios outperforms ‘P&L explain’, new paper claims
Working out how to compare the performance of derivatives pricing models seems like a simple problem. Yet a satisfactory solution is hard to come by. The financial literature is limited to a handful of studies, and the standard approach – so-called P&L explain or P&L attribution – is considered too simplistic.
The search for a better way has taken on added urgency with the introduction of new market risk capital rules. Under the Fundamental Review of the Trading Book, banks must pass a series of tests before they are allowed to use their own internal risk models.
The P&L attribution test, a key plank of the regime, uses a mean and variance ratio to compare two P&L figures, one generated by a bank’s front-office pricing models and one generated by its risk models.
Banks employ similar techniques to compare pricing models and select the best ones from a risk management perspective. But some feel the practice falls short of industry needs – particularly for hedging – and question its reliability.
“It’s useful to have but it is quite a simplistic test,” says Alexandre Antonov, chief analyst at Danske Bank.
Antonov, together with Rajiv Sesodia, head of traded risk models at Standard Chartered, and Jan Baldeaux, a quantitative analyst at ANZ Bank, proposes an alternative approach.
Their technique consists of a backward-looking test that compares the models’ replicating portfolios, which are combinations of assets or derivatives designed to mimic the payoff of the original position. In the Black-Scholes world, replicating portfolios are the building blocks of arbitrage-free pricing and hedging.
In practice, the authors extract the hedging components within the payoff, recalculate the hedging portfolio using an alternative model, and observe its movements. If these movements can be described by a deterministic function of the market factors on the current position, then the model perfectly fits the market and the instrument. A deterministic function will give the same results given the same inputs.
Any deviation between hedge and original position will give the hedging error, the difference between perfect hedging and imperfect hedging.
We cannot say [the test] will always give the same answer to the comparison between two models… but we estimate it is about 95% accurate
Alexandre Antonov, Danske Bank
The approach aims to provide front-office desks with an objective way of gauging whether a new model outperforms the existing one. The method could also find a home in the model validation team in a bank.
The research shows that a more sophisticated model like the Heston model performs better than Black-Scholes, even with the adjustments that are normally plugged into it. “This is a quite strong conclusion, because there are publications that reached the opposite one,” explains Antonov.
In particular, one study in 2018 by Frederic Vrins, finance professor and chairman of the Leuven finance research centre, and his then student, Nathan Lassance, found that the Black-Scholes option pricing model was preferable to the Heston model.
The opposing conclusions of the two studies could stem from one crucial difference between techniques. Antonov et al’s method observes the hedging error throughout the whole duration of the contract, whereas Vrins and Lassance’s technique observes the hedging error at maturity.
Vrins explains that, at the time of his study, there was very little research into optimal models over the full lifetime of the hedge.
“When we started our research, we were surprised not to find a study that analysed the hedging error up to the maturity of the derivative. Most of them deal with a very short hedging horizon or focused on fitting the implied volatility surface,” Vrins says.
Antonov admits there are weaknesses in the technique he describes, and that its output cannot be blindly taken on board at all times. “Of course we cannot say it will always give the same answer to the comparison between two models for any set of data and contract, because it’s based on statistics, but we estimate it is about 95% accurate,” he explains.
However, he is confident in the benefits it can bring to derivatives desks and would like to see it implemented soon: “I’m planning to present the method internally to the model validation team at Danske Bank,” he reveals.
Others in the industry say the new test is an improvement on the existing method for comparing models.
“The P&L explain defines performance over a fairly short term and in a narrow way,” says Colin Turfus, senior model validation quant at a major European bank. “I like this new approach insofar as it is more of a global perspective of the performance of the model from the beginning to the end of the trade.”
Editing by Kris Devasabai and Alex Krohn
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@risk.net or view our subscription options here: http://subscriptions.risk.net/subscribe
You are currently unable to print this content. Please contact info@risk.net to find out more.
You are currently unable to copy this content. Please contact info@risk.net to find out more.
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Printing this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@risk.net
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Copying this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@risk.net
More on Our take
Data shines light on Tibor fragility
Lack of actual transactions in D-Tibor should be considered in fallback discussions
Living with SA-CCR, one year on
Collateral agreements and FX futures may be some of the ways to tackle increased capital costs
GFXC to entice buy-side code adoption with ESG tie-ups
Rating agency partnerships would link FX code adoption to ESG scores
Clock ticking on UK plan for regulatory reforms
Changes to SMCR and short-selling rules least likely to be completed before next election
How did EU regulators miss the FTX horror story?
Gruesome accounting practices and a questionable cast: plenty of grounds to reject Mifid licence
ARRC’s trivial fight over term SOFR use
Toyota’s ABS deal should not derail effort to expand use of term rate in derivatives
What happens when a bank drops off the systemic risk radar?
Russia’s Sberbank skipped this year’s G-Sib assessment. But just because a bank is invisible doesn’t mean it no longer poses a risk
Degree of influence 2022: in the grip of volatility
Rough volatility, liquidity and trade execution were quants’ top priorities this year