Whither stress testing?

In light of the Basel Committee's renewed emphasis on stress testing, David Rowe says an industry dialogue on how this oft-cited technique should be applied is overdue, and offers some preliminary thoughts on the issue

In the September issue of Risk, I discussed the portion of the Basel Committee's July statement that describes intended revisions to the market risk component of Basel II. An important aspect of these revisions is making sure stress testing is incorporated systematically into the regular process for generating market risk information.

Commenting on my discussion, Barry Schachter notes that "it appears to have been accepted as a self-evident matter of fact" that "the results from portfolio stress testing generate additional knowledge about potential portfolio losses not obtainable from value-at-risk".1 Nevertheless, he continues, "it is not easy to find any concrete discussion of how stress testing generates additional knowledge, what that knowledge is, how great is that increment over VAR-obtainable knowledge and whether stress testing can provide sufficient additional knowledge to fill the gap between VAR-obtainable knowledge and 'enough' knowledge".

Schachter faults the Basel Committee because it has "not addressed the open issues" about the information obtainable from stress tests. I tend to be less critical on this score. Until the mid-1990s, banking supervision had always been conducted in a thoroughly prescriptive fashion. I believe the trend away from this approach and towards 'best practice' as the operative regulatory benchmark has been most beneficial. That said, I fully agree with Schachter that, "what is needed is some general agreement on standards for conducting stress tests... and for incorporating those results into a comprehensive risk measurement framework". Herewith, I offer some preliminary thoughts on this topic.

The focus of attention

At the risk of stating the obvious, the intended focus of stress testing is what may happen beyond the threshold at which we measure VAR. The rationale for this from a supervisory perspective is clear. Losses in the range of 99% VAR estimates will not threaten the existence of an institution. Were this not so, the life expectancy of trading firms would be a few years at most.

Potentially lethal events are those hidden in the far tail of the loss distribution, well beyond the VAR cut-off. The goal of stress testing is twofold: to provide a rough sense of how extreme such losses could be and of what configuration of market events could produce such losses given existing trading positions.

Perhaps the most difficult mental adjustment required for stress testing is to abandon the attempt to assign probabilities to results. We are focusing on an area where observations are so scarce and causal effects so varied and uncertain that statistical analysis is unlikely to provide useful insight. Rather, we are looking at implications of hypothetical scenarios constrained only by some multiple of historical worst-case observations.

Even here, however, it is reasonable to demand some degree of logical consistency. Therefore, it is appropriate to constrain the configuration of the yield curve so that it does not fall too far outside historically observed shapes. Fuel substitution means we need not consider scenarios where the price of one energy source soars while that of a competing source collapses. The exact nature of 'logical constraints' on stress scenarios will hopefully be part of an unfolding debate. That some such constraints are appropriate, however, seems clear.

Standard versus bespoke scenarios

There is a role in stress testing for a set of fixed scenarios to be analysed repeatedly. Among the most important of these are what an old colleague of mine used to call 'the market's greatest hits'. Every institution has staff who lived through traumatic historical events. These include the two oil embargoes of the 1970s, the stock market crash in the 1980s, the sterling and Asian currency crises in the 1990s, and the collapse of the internet bubble early in the current decade. Applying stress scenarios built around these events to current positions can provide a valuable historical perspective. This is basically a scattershot approach, however, and must be supplemented by bespoke scenarios tailored to existing positions.

One systematic way to develop bespoke scenarios is to evaluate all the results of standard Monte Carlo or historical simulations that produce losses beyond the value-at-risk threshold. One would expect these to cluster into groups that exhibit similar market variable movements. Having isolated a limited set of such market change ensembles, these could then be stressed to some agreed excess of historical worst-case movements. One might call this 'pessimisation' of the scenarios, since they are designed to exploit the biggest vulnerabilities in the existing trading positions. The resulting loss simulations would provide a useful sense of both the magnitude of a 'thinkable worst-case' loss, as well as the types of market events that could trigger such a loss.

Another interesting insight could be derived by comparing the nature of the pessimised scenarios over time to see if they remain similar or shift regularly from day-to-day and week-to-week. Supervisors would also find it useful to examine whether these scenarios are similar across multiple institutions, possibly presaging compound liquidity problems if everyone tried to unwind at once.