Ratings can still sharpen credit risk picture

Study shows even the most modern default models benefit from adding credit rating information

The shortcomings of credit ratings are well known. Like the fuzzy screen of an old TV set, they give an impression of what’s going on, but lack reliability. Now, new research suggests that combining ratings with public financial information can get you a much clearer picture.

Credit ratings are clearly imperfect. The 2008 crisis notoriously revealed their vulnerability to commercial influence, and recent research highlighted their tendency to be ‘sticky’ just above the point of a downgrade to high yield. What’s more, a 2016 paper demonstrated that even aggregating ratings from different agencies did little to improve results.

But in a paper publishing later this year in the Journal of Credit Risk, Rainer Hirk, of the Vienna University of Economics and Business, finds that standard credit risk model reliability improves significantly when combined with credit rating information.

The study doesn’t investigate what this improvement in accuracy would mean for debt investors in financial terms, says Hirk. As research continues, however, he says “the model class could be used to gain further insights into the behaviour of the [credit rating agencies]”.

Hirk tested the accuracy and discrimination of three standard credit risk models against Compustat data on US companies from 1985 to 2014. He then calculated ‘accuracy ratings’ for each model, both with and without the addition of credit rating information, to measure their ability to discriminate between high and low credit risks.

For all three models, and for almost every year, adding credit rating information improved the accuracy rating. The models used were: Altman’s Z-score, which uses publicly available accounting information to produce an estimate of default probability; CHS (Campbell-Hilscher-Szilagyi), which uses information from both financial statements and equity market moves; and TYG (Tian-Yu-Guo), which builds on CHS by using statistical methods to select the most important inputs.

The TYG model gave the best overall performance with the added rating information – the model’s authors noted in 2015 that it outperformed earlier models, including CHS – while the Altman model was boosted the most by the combination.

[The standard credit risk] model class could be used to gain further insights into the behaviour of the [credit rating agencies]

Rainer Hirk, Vienna University of Economics and Business

The rating was calculated by ranking each category of firm by predicted default risk in a bar chart, and comparing the area between the curve and a straight line with the area under a curve produced by a ‘perfect model’ that predicted defaults with 100% accuracy.

Not all were rated by the three major rating agencies through the period – Fitch, in particular, had limited coverage – but use of a multivariate framework allowed Hirk to compensate for missing data points.

Imperfect science

Even with the added credit rating information, the models were far from perfect. A perfect model would produce an accuracy rating of one. Without ratings information, out-of-sample tests showed ratings of between 0.8112 and 0.8968. Adding the ratings information increased accuracy by roughly 0.04 in each case. For Altman, it rose from 0.8112 to 0.8587; for CHS, from 0.8828 to 0.9232; and for TYG, from 0.8968 to 0.9257.

Another indicator of prediction quality, the weighted Brier score, shows the deviation between predicted probability of default (PD) and actual default – a binary one/zero – for each firm. A good model would have a high predicted PD – of one – for all the companies that defaulted, and low PD for the rest – and a low-weighted Brier score. Again, adding rating information improved the model, lowering its weighting. Altman’s score went from 0.3505 to 0.3065, CHS from 0.3027 to 0.2758, and TYG from 0.2852 to 0.2602.

There’s no way to indicate an individual rating agency’s overall optimism or pessimism in the analysis, but Hirk points out that his earlier research with colleagues Kurt Hornik and Laura Vana suggests that they track each other fairly closely. They did find, however, that Moody’s tends to be conservative for high-yield debt, and Fitch tends to be optimistic, particularly the border between high-yield and investment-grade ratings.

But Hirk warns against using the latest model too freely to compare agencies: “In principle, one can use the polychoric correlations between the failure indicator and the rating agencies as a measure of association. As we have many missing observations, in particular for Fitch, one has to be very careful when interpreting the correlations.”

As for further research, there’s reason to believe that time could be an interesting added dimension. The previous study already found signs that rating standards tightened significantly in 1999, loosened during the 2008 financial crisis, then rapidly returned to pre-crisis levels of strictness. So, it’s possible that the value of the default information they provide may also have changed over that period.

“At the moment, we are trying to extend the model by including a longitudinal dimension, which makes the estimation even harder,” says Hirk.

Editing by Louise Marshall

• Tweet  
• Facebook  
• LinkedIn  
• Save this article
• Send to  
• Print this page  

• Topics
• Buy side
• Correlation
• Credit ratings
• Credit risk
• Credit risk modelling
• Data
• High yield
• Investment
• Modelling
• Probability of default (PD)
• Yield
• Risk management
• Multivariate asset modelling
• Probability
• Research
• Investing