CECL models may leave banks ill-prepared for next downturn

Mortgage backtest study shows some loan-loss models miss the mark

Recession

Some models used by US lenders to set loan-loss provisions under new accounting rules fail to produce accurate estimates of credit risk during economic downturns, research reveals. The study backtested six modelling methods with mortgage data from the financial crisis of 2008 and showed that most models were unable to equip banks with appropriate levels of reserves.

The results raise questions for banks as they adapt to the Current Expected Credit Loss (CECL) regime, a new set of standards that requires reporting of lifetime expected losses for all loans – including those which have so far incurred no actual losses.

The study suggests that two models would have started calling for higher reserve levels in 2005 – when macroeconomic conditions still looked good but credit quality was starting to shift. Other models fared less well. Moving to CECL from the previous incurred-loss standard was hoped to reduce procyclicality, the US Federal Reserve has argued.

The US Financial Accounting Standards Board has not specified a particular approach for estimating losses under CECL, instead saying that “acceptable methods include loss rate, roll rate, vintage analysis, discounted cashflow, and probability of default/loss given default methods”.

Banks were told they could use risk models to make “reasonable and supportable” forecasts, or rely on historical loss data. The ruling left risk managers and accountants at loggerheads over the choice between model-based or historical data-based calculation.

In practice, the results of moving to CECL rules have varied substantially from bank to bank.

Truist – the lender created by the merger of SunTrust and BB&T – reported a 153% increase in loan-loss allowances as of January 1, 2020, compared with the previous year; Ally Financial and Goldman Sachs reported 103% and 83% increases. The biggest driver was their holdings of long-term or indefinite-term exposures such as credit card debt, which attract high allowances under CECL. Wells Fargo pinned its 12% drop in loan-loss allowances to the relatively short-term maturities of its commercial loan book.

The broad spread of the changes attributed to CECL echoes the effect of the IFRS 9 accounting change last year – the analogous framework for non-US banks – when different banks’ models were found to produce wildly differing loan-loss reserve figures for the same portfolio, leading to calls for more prescriptive rules on model use.

Research author Joseph Breeden, associate editor of the Journal of Credit Risk and chief executive of vendor Prescient Models, used economic data from the period around the 2008 financial crisis to backtest the most widely used CECL modelling approaches for 30-year fixed rate mortgages, and found that the choice of model would have made a big difference in preparing banks for the losses that the crisis brought.

Breeden used weighted average remaining maturity (Warm), time series, roll rate, age-period-cohort, state transition and multihorizon discrete time survival models to produce CECL reserve figures at quarterly intervals for all US states and territories, with data provided by Fannie Mae and Freddie Mac on 30-year fixed rate mortgage performance from 2001 to 2017. The models were fed consensus macroeconomic forecasts published during the period in question – which were often very different from the subsequent course of events, either underestimating the speed and severity of the subsequent recession, or underestimating the speed of recovery from its peak.

Ideally, Breeden says, the models would have been deprived of any information not available at the time. However, not enough data was available for the pre-crisis period to calibrate the models adequately, so full-history data was used to estimate economic sensitivity and product lifecycle inputs. Breeden has argued in earlier research that several cycles worth of data would be the optimum for adequate risk modelling.

The results of the models were almost unaffected by different economic forecasts. The author used three sets of scenarios: ones that reverted to the mean immediately, reverted after two years of flat extrapolation, or reverted after a two-year period of macroeconomic forecasting. Tests using all three scenarios produced very similar loss estimates over the course of the crisis.

“The difference between the best and the worst realistic macroeconomic scenarios is slight, but the difference between the best and worst models is dramatic,” Breeden says. He argues that a good CECL model should not be expected to predict macroeconomic shocks, but should still have picked up rising exposure to credit risk in the mortgage portfolio before the economic situation worsened, and informed loan-loss provisioning among banks accordingly.

Warm warning

The Warm model performed worst out of all models tested, “out of phase with actual reserve needs”, with reserve estimates only beginning to rise in 2008, when the crisis was already underway, and peaking in 2012 – well after the worst of the crisis had passed.

“In short, Warm should not be used for CECL,” Breeden writes. “Lenders would be better off using a completely flat through-the-cycle average loss rate.”

Warm has come under question before – but guidance from the Financial Accounting Standards Board in January 2019 was that it was “an acceptable method to estimate expected credit losses”, at least for less complex asset pools.

Breeden remarks: “The most important question is not when reserves will peak … When large volumes of new loans are booked, the hazard function or age-period-cohort lifecycle will predict when those losses should occur in the future. Similarly, a strong credit cycle exists in mortgage [markets] which can also be incorporated in the forecast. Therefore, the question is whether any of the models give a warning or simply jump to peak levels at the last moment.”

Other models also showed flaws. The time series model displayed procyclical traits and did not call for more reserves until late 2008. When mortgage credit risks were rising in 2005–08, Breeden notes, the time series model held reserves flat.

However, two other models, the age-period-cohort (vintage) model and the multihorizon discrete time survival, performed significantly better. If in use at the time, both models would have started calling for higher reserve levels in 2005, when macroeconomic conditions still looked good but credit quality was starting to shift.

The age-period-cohort model uses origination date, age of loan and current economic environment as inputs to produce a figure for probability of default, exposure at default, and attrition, producing a monthly forecast for each vintage’s performance in the future until the maturity date or estimated repayment date.

Muiltihorizon discrete time survival models take a similar approach at the level of single loans rather than cohorts of loans of the same vintage, using the vintage model to estimate the effects of lifecycle and economic environment for each loan, and then constructing separate origination and behavioural models and calculating the outputs (probability of default, exposure and attrition) at various time horizons.

Breeden points out that the choice of model will not initially make much difference in terms of the acceptability of the calculation to regulators – as CECL is bedding in, the priority will be on collecting data, creating systems and managing loss reserves.

“However,” he continues, “the purpose of CECL is to help lenders survive crises. Simple models appear to offer little help in surviving crises and may actually be harmful. To be useful in the period leading up to a crisis, having an effective model will be important.”

The paper is due to be published in the March edition of the Journal of Credit Risk.

Editing by Alex Krohn

Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.

To access these options, along with all other subscription benefits, please contact info@risk.net or view our subscription options here: http://subscriptions.risk.net/subscribe

You are currently unable to copy this content. Please contact info@risk.net to find out more.

Most read articles loading...

You need to sign in to use this feature. If you don’t have a Risk.net account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an individual account here