A tale of two (or three, or four) models
Performance measure based on quality of replicating portfolios outperforms ‘P&L explain’, new paper claims
Working out how to compare the performance of derivatives pricing models seems like a simple problem. Yet a satisfactory solution is hard to come by. The financial literature is limited to a handful of studies, and the standard approach – so-called P&L explain or P&L attribution – is considered too simplistic.
The search for a better way has taken on added urgency with the introduction of new market risk capital rules. Under the Fundamental Review of the Trading Book, banks must pass a series of tests before they are allowed to use their own internal risk models.
The P&L attribution test, a key plank of the regime, uses a mean and variance ratio to compare two P&L figures, one generated by a bank’s front-office pricing models and one generated by its risk models.
Banks employ similar techniques to compare pricing models and select the best ones from a risk management perspective. But some feel the practice falls short of industry needs – particularly for hedging – and question its reliability.
“It’s useful to have but it is quite a simplistic test,” says Alexandre Antonov, chief analyst at Danske Bank.
Antonov, together with Rajiv Sesodia, head of traded risk models at Standard Chartered, and Jan Baldeaux, a quantitative analyst at ANZ Bank, proposes an alternative approach.
Their technique consists of a backward-looking test that compares the models’ replicating portfolios, which are combinations of assets or derivatives designed to mimic the payoff of the original position. In the Black-Scholes world, replicating portfolios are the building blocks of arbitrage-free pricing and hedging.
In practice, the authors extract the hedging components within the payoff, recalculate the hedging portfolio using an alternative model, and observe its movements. If these movements can be described by a deterministic function of the market factors on the current position, then the model perfectly fits the market and the instrument. A deterministic function will give the same results given the same inputs.
Any deviation between hedge and original position will give the hedging error, the difference between perfect hedging and imperfect hedging.
We cannot say [the test] will always give the same answer to the comparison between two models… but we estimate it is about 95% accurate
Alexandre Antonov, Danske Bank
The approach aims to provide front-office desks with an objective way of gauging whether a new model outperforms the existing one. The method could also find a home in the model validation team in a bank.
The research shows that a more sophisticated model like the Heston model performs better than Black-Scholes, even with the adjustments that are normally plugged into it. “This is a quite strong conclusion, because there are publications that reached the opposite one,” explains Antonov.
In particular, one study in 2018 by Frederic Vrins, finance professor and chairman of the Leuven finance research centre, and his then student, Nathan Lassance, found that the Black-Scholes option pricing model was preferable to the Heston model.
The opposing conclusions of the two studies could stem from one crucial difference between techniques. Antonov et al’s method observes the hedging error throughout the whole duration of the contract, whereas Vrins and Lassance’s technique observes the hedging error at maturity.
Vrins explains that, at the time of his study, there was very little research into optimal models over the full lifetime of the hedge.
“When we started our research, we were surprised not to find a study that analysed the hedging error up to the maturity of the derivative. Most of them deal with a very short hedging horizon or focused on fitting the implied volatility surface,” Vrins says.
Antonov admits there are weaknesses in the technique he describes, and that its output cannot be blindly taken on board at all times. “Of course we cannot say it will always give the same answer to the comparison between two models for any set of data and contract, because it’s based on statistics, but we estimate it is about 95% accurate,” he explains.
However, he is confident in the benefits it can bring to derivatives desks and would like to see it implemented soon: “I’m planning to present the method internally to the model validation team at Danske Bank,” he reveals.
Others in the industry say the new test is an improvement on the existing method for comparing models.
“The P&L explain defines performance over a fairly short term and in a narrow way,” says Colin Turfus, senior model validation quant at a major European bank. “I like this new approach insofar as it is more of a global perspective of the performance of the model from the beginning to the end of the trade.”
Editing by Kris Devasabai and Alex Krohn
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@risk.net or view our subscription options here: http://subscriptions.risk.net/subscribe
You are currently unable to print this content. Please contact info@risk.net to find out more.
You are currently unable to copy this content. Please contact info@risk.net to find out more.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@risk.net
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@risk.net
More on Our take
Thin-skinned: are CCPs skimping on capital cover?
Growth of default funds calls into question clearers’ skin in the game
Quants dive into FX fixing windows debate
Longer fixing windows may benefit clients, but predicting how dealers will respond is tough
Talking Heads 2024: All eyes on US equities
How the tech-driven S&P 500 surge has impacted thinking at five market participants
Beware the macro elephant that could stomp on stocks
Macro risks have the potential to shake equities more than investors might be anticipating
Podcast: Piterbarg and Nowaczyk on running better backtests
Quants discuss new way to extract independent samples from correlated datasets
Should trend followers lower their horizons?
August’s volatility blip benefited hedge funds that use short-term trend signals
Low FX vol regime fuels exotics expansion
Interest is growing in the products as a way to squeeze juice out of a flat market
Can pod shops channel ‘organisational alpha’?
The tension between a firm and its managers can drag on returns. So far, there’s no perfect fix