Underperforming performance measures? A review of measures for loss given default models

Katarzyna (Kasia) Bijak and Lyn C. Thomas

Abstract

As far as the prediction of probability of default (PD) is concerned, the model performance is typically measured with the Gini coefficient and/or the Kolmogorov– Smirnov (KS) statistic. For loss given default (LGD) models, however, there are no standard performance measures. In fact, more than fifteen different measures may be used, including mean square error (MSE), mean absolute error (MAE), the coefficient of determination (R-squared) and various correlation coefficients between the observed and predicted LGD. However, some measures cannot be readily recommended for LGD models, even though they have been used for this purpose. It is argued that some measures should only be employed for specific types of model. It is also pointed out that other measures may be applied interchangeably to avoid information redundancy. Moreover, the area under the receiver operating characteristic curve (AUC) is critically discussed in the LGD context. Four new measures are then proposed: mean area under the receiver operating characteristic curve (MAUROC), mean accuracy ratio (MAR), mean enhanced Lin–Lin error (MELLE) and a generalized lift. The review is illustrated using an empirical example.