Best modelling innovation: CompatibL

Collecting the Best modelling innovation award for the third consecutive year is CompatibL. The US-headquartered vendor’s win on this occasion was thanks to the launch of the CompatibL AI Co-Pilot for model governance, which was put into production as part of the CompatibL Platform release 9.1. 

Model governance is a critical bank function that controls the model development process and lifecycle. This includes ensuring all model changes are authorised and properly implemented, that models are documented for management and regulators, and that comprehensive model release notes are published. This requires significant time and effort, with each regulatory submission or internal model approval requiring several documents that often reach 300–500 pages in length. 

Judges said:

• “The solution is compelling.”
• “Brings a high degree of automation to the model governance process.”
• “Good progress in the past 12 months.”

Traditionally, this work has been done manually, with documents drafted in Microsoft Word. Technology solutions – including trading and risk management systems, and specialist compliance and governance products – have offered limited scope for automation because of rigid templates and an inability to comprehend free-form documents or source code. 

CompatibL’s AI Co-Pilot solves this problem by empowering users with an artificial intelligence (AI)-based model governance process that helps create the required documentation. Clients can choose between two generative AI models: OpenAI’s GPT-4 for use in Azure accounts or Meta’s Llama2 for use in data centres or private clouds. Data security is paramount, and the solution’s architecture guarantees no client data is ever sent to the core large language model vendor or any other outside party. 

Each model has been customised by CompatibL’s engineers using curated input documents and human feedback. Prompt engineering techniques are also used to obtain the desired results. Using this combination of fine-tuning and prompts, models can be adapted for a specific purpose. 

The result is that CompatibL AI Co-Pilot performs labour-intensive work that would be impossible for a human team to complete without AI technology. It can look at every line of source code and every version control log message, perform in-depth analysis of prior documentation and release notes, and integrate the resulting data in a meaningful way. It can also cross-reference the output with specific lines of source code, which is helpful for control purposes but rarely executed because of the time and effort involved.

CompatibL AI Co-Pilot can also flag areas of concern inside the source code, including potential bugs, discrepancies between the source code of the model and how the model is described in documentation, or use of numerical methods and modelling techniques that were not intended or approved. 

Production clients using CompatibL AI Co-Pilot report a significant improvement in the quality of model governance processes, with the work undertaken to collate the information rendered less repetitive and more fulfilling. Analysts no longer need to work through lines of code or revision log messages, but instead use the model to adjust the prompts and settings to generate documentation in line with the firm’s policies and guidelines.

Alexander Sokol, executive chairman at CompatibL, says:

"Model governance deals with highly specialised subject matter outside the scope of conventional AI Co-Pilot tools. As a company with more than 20 years of experience in model development and model validation, CompatibL was uniquely positioned to develop the first AI Co-Pilot solution for model governance. We would like to thank the judges for recognising the tremendous potential of AI to enhance model governance that was harnessed by our Co-Pilot solution. We hope this prestigious award will lead to more AI-based innovation in this area."

Sponsored content