Journal of Risk Model Validation

Steve Satchell
Trinity College, University of Cambridge

This issue of The Journal of Risk Model Validation contains three papers, all of which are heavily influenced by the recent rise in data-intensive methods. The old world of linear regressions seems to be disappearing. However, these new techniques are not without their own challenges, a number of which our contributors seek to address.

Our first paper is “Research on the dynamic early warning effect on the manufacturing industry from the perspective of systemic financial risks: evidence from the Chinese market” by Guanghui Han, Zijin Cao and Hui Xie. They argue that accurate measurement and effective prevention of systemic financial risks are crucial factors in promoting financial stability and high-quality development. They therefore construct a variant of the China systemic financial risk composite index (CSFRI) by combining the entropy weighting method and the criteria importance through use of an intercriteria correlation (CRITIC) weighting method. After further calculations, this study explores whether the CSFRI can contribute to the development of the manufacturing industry. By establishing a time-varying parameter–stochastic volatility– vector autoregressive (TVP-SV-VAR) model, the authors explore the dynamic and time-varying impact of changes in the CSFRI on the manufacturing industry. They claim that the CSFRI developed in this study accurately reflects the risk status of China’s financial market while also serving as a leading indicator with clear dynamic and time-varying characteristics for the manufacturing industry. If this is the case, it could clearly be used as an early warning indicator of financial risk.

The issue’s second paper is “A three-stage fusion model for predicting financial distress considering semantic and sentiment information” by Jiaming Liu and Bo Yuan. The authors confirm the importance of analyzing the management discussion and analysis (MD&A) text of listed companies in financial distress prediction models. Their contribution is to integrate text analysis and machine learning techniques; they claim that doing so reveals the financial information hidden in MD&A text and accurately captures the emotional tendency of the authors of the text through a sentiment analysis lexicon, providing a more comprehensive and detailed method for predicting a company’s financial condition. The precise steps they carry out to do this involve multiple stages of analysis. According to the authors, the introduction of semantic and sentiment features significantly enhances the model’s predictive performance, as one would hope.

The final paper in the issue, by Lukasz Prorokowski, is titled “Model risk quantification for machine learning models in credit risk”. This focuses on core issues in risk validation and addresses the interplay between regulatory developments and the current vogue for machine learning. The validation function is used to assess model risk. This is detailed in the European Banking Authority’s “Supervisory handbook on the validation of rating systems under the internal ratings based approach” (EBA/REP/2023/29). Further, Capital Requirements Regulation III – which is applicable from January 1, 2025 – introduces a detailed definition of model risk, thereby providing the principles to which the model risk assessment should adhere. Recently, the rise in various applicable data-driven procedures has meant that one could use machine learning models (MLMs) instead of common internal ratings-based (IRB) models. The consultation conducted by the European Banking Authority in 2023 (EBA/REP/2023/28) suggests that banks intend to use MLMs in selected areas of the IRB approach. The author reviews bank-specific model risk measurement techniques with a focus on implemented model risk rating solutions for MLMs, and he reports the key challenges faced by the validation functions at participating banks in relation to quantifying MLMs’ model risk. Recognizing these challenges, Prorokowski provides a model risk rating concept for MLMs. I find the interplay between model development and regulatory change rather fascinating.