Journal of Risk Model Validation

Risk.net

Underperforming performance measures? A review of measures for loss given default models

Katarzyna (Kasia) Bijak and Lyn C. Thomas

  • For a fair assessment, it is recommended to include performance measures of all categories.
  • As AOC quickly converges to MSE/MAE, we recommend applying either AOC or MSE/MAE.
  • We only recommend applying R-squared and interpreting its value in OLS models.
  • We suggest using the proposed measures, MAUROC and MAR, instead of AUC and AR for LGD.
  • For models that estimate quantiles, we suggest using the proposed measure, MELLE.

As far as the prediction of probability of default (PD) is concerned, the model performance is typically measured with the Gini coefficient and/or the Kolmogorov– Smirnov (KS) statistic. For loss given default (LGD) models, however, there are no standard performance measures. In fact, more than fifteen different measures may be used, including mean square error (MSE), mean absolute error (MAE), the coefficient of determination (R-squared) and various correlation coefficients between the observed and predicted LGD. However, some measures cannot be readily recommended for LGD models, even though they have been used for this purpose. It is argued that some measures should only be employed for specific types of model. It is also pointed out that other measures may be applied interchangeably to avoid information redundancy. Moreover, the area under the receiver operating characteristic curve (AUC) is critically discussed in the LGD context. Four new measures are then proposed: mean area under the receiver operating characteristic curve (MAUROC), mean accuracy ratio (MAR), mean enhanced Lin–Lin error (MELLE) and a generalized lift. The review is illustrated using an empirical example.

 

To continue reading...

You need to sign in to use this feature. If you don’t have a Risk.net account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an indvidual account here: