The accuracy of credit scoring receiver operating characteristic in the presence of macroeconomic shocks

George A. Christodoulakis, Stephen E. Satchell

Abstract

ABSTRACT

The receiver operating characteristic (ROC) curve is often used by creditors to assess credit scoring accuracy and as part of their Basel II model validation. The purpose of this paper is to provide a mathematical procedure to assess the accuracy of ROC curve estimates for credit defaults in the presence of macroeconomic shocks. Our approach supplements the nonparametric method recommended by Engelmann et al (2003) based on the Mann–Whitney test, which is used as a summary statistic of ROC curves. Our method assumes initially that both sick and healthy loan credit rating scores are generated by normal distributions, and shows how ROC estimate depend on the location and scale parameters. We then use these results to construct ROC confidence intervals in closed form and examine the influence of exogenous macroeconomic shocks. We further generalize our method by allowing credit rating scores to be generated by skewnormal distributions, thus allowing skewness to affect the moments of the distribution. We show how the presence of skewness would result in further inaccuracy in model validation.