Journal of Risk Model Validation

This issue of The Journal of Risk Model Validation is split between the more formal notion of model validation and the more general one of backtesting. If I was capable of time travel, I would, at inception, have named the journal The Journal of Risk Model Validation and Backtesting. I am very happy for the journal to publish research on both topics and I hope readers do not feel misled by this policy. Purists may argue that backtesting is more model implementation than model validation. I welcome correspondence on this subject and would be delighted to publish it.

The first paper, "Dynamic value-at-risk models and the peaks-over-threshold method for market risk measurement: an empirical investigation during a financial crisis" by Marco Bee, is concerned with backtesting various value- at-risk models using a range of stochastic processes based on either generalized autoregressive conditional heteroskedasticity volatility or exponentially weighted moving average processes. The period covered, 2004-9, spans times of both fear and greed and one therefore feels that if a model works during this period, it should work in a wide range of circumstances, whether those circumstances represent "black swans" or not. Readers will need to read the paper to find out which model does best, but the aspect of the paper that struck me most was the claim that dynamic value-at-risk modeling reacted quickly to periods of turmoil. The authors are careful to restrict this claim to these models and this period, but this model feature is currently of tremendous research interest.

The second paper, "Probability of default validation: a single-year and a multiyear methodology for the Basel framework" by Oliver Blümke, is based on hypothesis testing and, if I may quote from the abstract, studies "two methodologies that are designed to test whether observed default rates are in line with default probabilities applied within the Basel framework". Hypothesis testing is a particularly pure form of model validation and represents an ideal procedure. Practical realities often make it difficult to use, however. For example, in the construction of equity models with macroeconomic factors, it is invariably the case that betas (exposures) are statistically insignificant. It is often necessary to augment such testing with practical Bayesian considerations. These can be as simple as lowering the acceptance probabilities or some sort of sequential testing. The authors focus their tests on ratings information, and it may well be that this is a practical and useful tool.

The third paper, "Further recipes for quantitative reverse stress testing" by Peter Grundke, extends earlier work by the same author on reverse stress testing. This new regulatory requirement has proved easy to describe but harder to implement, and when it was first announced I wondered whether it could ever be made operational, particularly if the identification of reverse stress tests was based on people arguing over lunch! The mathematical approach advocated by the author seems, to me at least, to be a valuable contribution.

The final paper, "A realistic approach for estimating and modeling loss given default" by Rakesh Malkani, aims to provide "an integrated conceptual structure for estimating and modeling commercial loan loss given default". As in one of the previous papers, the author's focus is on hypothesis testing, which allows us to use a formal decision-theoretic approach to model validation and backtesting. One of the author's contributions is to provide a number of testable hypotheses that are consistent with the model structure.

Without detracting in any way from the quality of our excellent contributors, the focus of the journal remains very much in default space. This may well reflect the economic realities that practitioners are facing at the moment, but, with my equity background, I would welcome contributions on the validation of stock selection models and traded instruments generally.

*Footnote:

It has been brought to our attention that the paper 'A realistic approach for estimating and modeling loss given default' by Rakesh Malkani, which was published in The Journal of Risk Model Validation in 2012 (Volume 6, Number 2, pp. 103-116), consists almost exclusively of work written by Mr Christopher Karr, without proper attribution.

We recognise that the paper should never have been published under Rakesh Malkani's name and apologise to Mr Karr unreservedly. We are reliant on our authors for giving accurate attribution information and it is sadly the case that errors of this kind will occasionally happen. We would like to assure our readers that we regard correct attribution as being of the upmost importance.