US banks’ VAR shortfalls are wrapped in a black box

Public disclosures only allow crude approximations of loss size and timing

Poring through US banks’ market risk disclosures can feel a bit like examining the cutout board from an archery contest. One can see how many arrows hit the target and how close they landed to the bullseye – but not how fast they flew, or in what order they were loosed.

Like arrows knocked off course by a gust of wind, value-at-risk predictions – the main input to the Basel 2.5 internal models approach (IMA) to market risk capital requirements – have repeatedly been swayed by volatility in recent years. Between Q4 2021 and Q1 2024, daily VAR estimates failed backtesting on 161 occasions.

Some of the most dramatic VAR overshoots have taken place at the US’s top systemic dealers. Goldman Sachs logged two in Q3 2024 alone, when losses reached 258% and 177% of VAR. JP Morgan recorded four between Q4 2024 and Q1 2025, ranging from 108% to 262%.

 

We know these figures thanks to the rigid taxonomy of the FFIEC 102 forms, which US banks must file quarterly. Unlike the more flexible Pillar 3 disclosures, these filings require firms to report their top three days of hypothetical losses, expressed as a percentage of that day’s VAR.

But beyond that, the filings provide little detail on how trading desks fare each quarter. Crucially, they don’t disclose the actual dollar amounts of these losses, or the dates on which they occurred. One can see the holes left in the target, but not the speed or trajectory with which the arrows struck.

There are enough fragments scattered across these disclosures to at least narrow the range of a given loss. FFIEC 102 filings contain both average and end-period VAR, while Pillar 3 reports add intra-quarter highs and lows. These figures are usually calibrated to a 10-day holding period, as required under IMA, but can be scaled to a one-day equivalent – within a reasonable margin – by dividing by the square root of 10. With that, one can bracket the plausible range for a given breach.

Take JP Morgan’s Q1 backtesting failures, for instance: 154% and 115% of VAR. If the larger breach occurred on the day VAR peaked for the quarter, the overshoot might equate to a loss of $128 million. Conversely, if it came on the day of VAR’s nadir, it could be as little as $8 million.

 

Market participants are sceptical about this approach. A risk head at a US bank concedes that, if pressed to guess the size of a peer’s VAR breach, they would base it on average VAR. For JP Morgan’s larger Q1 overshoot, that method yields about $41 million. But they were quick to stress this is a back-of-the-envelope calculation – one that says little, if anything, about trading profit and loss (P&L). The head of risk modelling at a European bank concurs that this methodology only provides a crude approximation, while the market risk head at a second US bank found it to be misleading.

That’s partly because the potential outcomes are as numerous as the trading days in the quarter. And because backtesting, for US banks, is based on hypothetical P&L alone – which assumes positions held flat from the previous close – it ignores intraday changes, fees or hedges that may have softened, or even erased the loss in reality.

Related

The IMA map: charting market risk capital under Basel 2.5 

US dealers tally most VAR breaches since mid-2023 

In theory, Pillar 3 disclosures should fill in these blanks. Under Basel rules, the MR4 template is meant to show the daily evolution of VAR, hypothetical and actual P&L, and include an accompanying narrative explicitly stating the date and size of any excess.

In practice, however, implementation varies. It is up to national regulators to verify the disclosures are sufficiently granular, and the template – required only semi-annually, as opposed to the FFIEC 102’s quarterly cadence – often boils down to a crowded bar-and-line chart, with little, if any, in the way of explanatory notes. Many banks have often proved inconsistent in detailing when breaches occurred, and by how much they eclipsed VAR.

Even the US bank’s risk head admits market risk disclosures are written with regulators in mind – not external analysts trying to reverse-engineer trading performance. And supervisors, of course, have access to far more granular and timely data than anyone accessing publicly available disclosures. Trading desk performance that could move a share price is arguably the domain of 10-Q and 10-K Securities and Exchange Commission filings, not prudential risk templates.

Nevertheless, to an outside observer, market risk modelling remains a black box. Overshoots hint at when something has gone awry, but they reveal little else. As volatility keeps roiling markets, bank executives are wont to tout their trading prowess – but their models, and how they really perform, remain largely off-limits to public scrutiny.