Journal of Risk Model Validation
ISSN:
1753-9579 (print)
1753-9587 (online)
Editor-in-chief: Steve Satchell

The effect of variant sample sizes and default rates on validation metrics for probability of default models
David Li, Ruchi Bhariok, Radu Neagu
Abstract
ABSTRACT
In this paper a survey of common model-validation metrics for scoring models where the response is a binary variable is presented. These metrics include the Hosmer-Lemeshow statistic, the accuracy ratio, the standardized residual sum of squares and the conditional information entropy ratio. More specifically, we restrict ourselves to probability of obligor credit default models, and investigate the effects of varying sample sizes and default rates in the population. We show that no single validation metric gives accurate evaluations for a set of varying conditions, and we document the weaknesses and the strengths of these metrics using simulation and empirical data. We recommend that decision makers use information from multiple sources to drive their decisions, and that they understand the weight they need to put on each source given the specifics of the situation at hand.
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Printing this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@risk.net
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Copying this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@risk.net