Journal of Risk Model Validation

Risk.net

Validation of the backtesting process under the targeted review of internal models: practical recommendations for probability of default models

Lukasz Prorokowski

  • This paper complements the validation function. The purpose of this study is to provide practical guidance on validating the backtesting process.
  • For the introductory validation of the backtesting process, the validation team should assess the data quality standards, data directory and databases used for creating the backtesting dataset.
  • For the assessment of the backtesting frequency, the validation team should confirm the appropriateness of the size of the observation window in order to ensure that the poor performance of the model is not masked.
  • For the validation of the calibration backtest, the validation team should assess the homogeneity of the rating groups, differences between the PD estimates and realised observations and the appropriateness of the hypothesis underpinning the analysed statistical tests.
  • For the validation of the discriminatory power, the validation team should at least interpret the curves with the associated statistics and assess the number of defaults.
  • For the validation of the stability backtest, the validation team should assess the power of the statistical test based on its assumptions and justify any PD’s deviations using macroeconomic factors.

This paper provides practical recommendations for the validation of the backtesting process under the targeted review of internal models (TRIM). It advises on the introductory  steps for validating  the backtesting process and reviews the available statistical  tests for calibration, discrimination  and stability backtesting. The TRIM regulatory exercise is an international  supervisory initiative that inspects the internal models and related internal risk and governance policies of eurozone banks that are permitted  to use the advanced internal  risk-based (AIRB) approach. Under the TRIM guidelines, the designated banks should have specific policies  and internal guidelines for the validation of the backtesting process. Further, the affected banks are required to validate the entire backtesting process. Addressing these needs, this paper serves as a basis for producing such policies and utilizing appropriate statistical tools for validating  the backtesting process. The paper focusses on probability of default models. To date, no academic study has discussed the validation of the backtesting process with reference to the TRIM rules.

To continue reading...

You need to sign in to use this feature. If you don’t have a Risk.net account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an individual account here: