Basel II: What If?

Investment firms are relying on scenario analysis to gauge their operational risk.

The deadline for Basel II is looming and firms are working hard to meet the global banking initiative. Acknowledging that bad things happen to good divisions, business managers are holding scenario analysis sessions to highlight specific risks. In October, Waters gathered a panel of experts for a breakfast briefing on how to implement scenario analysis for Basel II op risk. Our panel included Daniel Mayo, an analyst for DataMonitor; Paul Mokdessi, senior manager; Sandeep Vishnu, senior manager; and Bjorn Pettersen, managing director of BearingPoint. The panel also featured Todd Loudenslager, director of enterprise risk management of US Bancorp and David Wong, vice president of operational risk management of ABN Amro. They shared their insights into how their firms tackle scenario analysis.

Waters:

David Wong, can you give us a sense of scenario analysis inside ABN Amro. Is it vital?

David Wong, ABN Amro:

It's certainly not new from a business perspective. It's something that we do for a number of purposes. Business continuity planning is a very pertinent example of that. As far as using it for operational risk, it is new. It's the next phase of what we're doing. We've done a pretty good job on loss event collection and establishing a risk self-assessment approach, and we're starting to move into the next levels. We are starting to move into key risk indicators and scenario analysis.

We have received encouragement from the US regulators. We spent two weeks with them earlier this summer during a benchmarking review, and their guidance and direction around scenario analysis pointed us closer to what we're talking about today versus what we initially had been considering.

Waters:

Todd, have you seen any butting of heads with senior managers when it comes to implementing imagination in these scenarios?

Todd Loudenslager, US Bancorp:

Definitely within the businesses, and with respect to the regulators, our local regulators have been very cooperative and very supportive in helping us understand what the landscape looks like. I've been fortunate to be a part of some larger groups that have been able to talk to the policymakers, and they seem to be acutely attuned to making sure that whatever infrastructure we come up with to manage this, from a regulatory and supervisory perspective, makes sense.

Operational risk management is a very young discipline at this point and it requires a lot of maturing. Therefore, a certain aspect of flexibility is needed.

Waters:

Do you find that some business lines can't conceive of scenarios in which they could lose business?

Loudenslager:

In The Right Stuff, Tom Wolfe describes the test pilot who comments on a pilot who crashes. He says, "He did something wrong because no good pilot would ever crash." Well, that's the same type of attitude as business folks who say, "Well, that might happen to another bank, but it couldn't ever happen to us." So they do have a certain amount of limitation to their imagination.

I'm not against financial folks; they're great. But they like to take existing data and apply it in a predictable fashion to the future. And scenario analysis is not necessarily built to fit automatically into a quantitative model. Just because you haven't yet gone through an event that cost $10 million doesn't mean that three of them couldn't happen tomorrow. That's a bit hard for people to fathom and you need to work through that with those folks. They need to understand that quantitatively, we're talking about alternative realities and things that potentially could happen. You can't necessarily say that the probability is low just because it hasn't happened yet.

Paul Mokdessi, BearingPoint:

It's clearly human nature that when you ask somebody an open-ended question like, "What could go wrong?" people look at you with blank stares. They'd probably not come back with something immediately. But when you walk them through a facilitated, structured process, item-by-item, and you say, "What could happen in your organization if you discover employees working together to defraud a client?" Then you've got something to work with. If you add to that some external examples that have happened at another organization, people can start to envision it. I think that's one of the best uses of the external database.

Waters:

How do you overcome the understandable reluctance to hypothesizing these potential losses? How do you wake the firms up?

Loudenslager:

There are two approaches for external data. One is to generate creative thought within the scenario analysis with publicly available data, such as Wall Street Journal and New York Times articles that can shock someone into thinking outside of their normal realm. This is a useful way to facilitate that process. The other approach is using a type of data that could be directly added into your model, something that is much more quantitative, much more scaled, and which adds a particular financial institution's loss history. You can fill in the difference between your own internal loss history and those scenarios that are going to be a little too far out there.

Waters:

David, can you tell us the timeline for scenario analysis inside ABN Amro?

Wong:

We're just getting started. What we'd ideally like to do is update the scenarios any time a significant change occurs in the business or in the business environment. Practically speaking, I think we're talking more on an annual basis. We'd certainly like to do them as frequently as we can. We're also considering rolling it in with the risk self-assessment (RSA) process because the exercises themselves seem very similar. When we approach the business to say, "Would you like to participate in a scenario analysis pilot?" some of the comments that we've received are that they just did an RSA three months ago. They ask how this is different and we've had to go through that dialogue.

I see that there's value in combining the efforts. The challenge is going to be making sure that you get enough out of both efforts when you combine them and not leave too much by the wayside.

Waters: Do people in the firm roll their eyes and say, "Oh great, here comes the risk guy?"

Wong:

I would like to say no, but being realistic, I think that does happen. We've received a much better reception than we expected after approaching some of the units about scenario analysis. We went in with what I'd call cautious optimism but we have certainly received a very good response. I think calling it "scenario analysis" and not calling it "risk self-assessment" has a different kind of tone and they've been more open in discussing it. It's more of a brainstorming session versus actually admitting what is wrong. It's about what could happen because of the business that we're in.

Waters:

Sandeep, why should investment firms conduct scenario analyses when all of these external databases already exist?

Sandeep Vishnu, BearingPoint:

Very simply stated: relevance and scaling. The way to incorporate external data into your own environment is to build scenarios around it and say, "This happened there, how could it happen to me?" The real issue that people struggle with is a degree of precision, and this alludes to both the risk assessment and the scenario analysis. What tends to make people comfortable with scenario analysis is when the degree of precision has been ratcheted down. The goal is not to get a precise answer. The goal is to get insight. External data offers a data point that allows you to generate the insight that ultimately leads to some quantification. But once you take off that onus on precision, it really opens people up to saying, "Here's how it can happen in our world." That's how you use external data.

Waters:

Dan, even with this wealth of knowledge out there, is it still necessary to conduct scenario analysis?

Daniel Mayo, Analyst:

Certainly. Scenario analysis seems to be what institutions are pursuing. A number of institutions have rolled that out and it is now something that people are familiar with. The next stage seems to be focused on the scenario analysis partly because it is regulatory driven but also, I think people have started to see the value in capturing data on extreme events.

Waters:

Are there any pitfalls in implementing scenario analysis?

Bjorn Pettersen, BearingPoint:

Like any kind of change program that you take into an organization, there can be tons of pitfalls. The most important thing is to make sure your approach is very pragmatic. If you're going to gain acceptance inside the firm, you have to make this low-tech and get the sponsorship. So, communication is key. It's as simple as that. If somebody comes out of the work session and they didn't really get much business value out of it, that's definitely a blow to the program.

Waters:

Were there any pitfalls in the scenario analysis implementation at your firm, Todd?

Loudenslager:

Well, once our most active and interested business line got the scenarios, they started undertaking a very systematic, quantitative analysis of the results. They concluded that these scenarios were unrealistic because we haven't seen them in the past. This is not necessarily a strict quantitative exercise. This is more of a creative exercise and one that can lead to a lot of creative thinking and a lot of creative solutions for the business line.

Waters:

Was it frustrating as a risk manager? Are these people in denial?

Loudenslager:

Sandeep was there and I think both of our perceptions were that they were trying to game it. They felt that they were smart enough to realize that their scenarios have potential capital implications and so their objective became to minimize the scenario so that they would minimize their capital. That was our concern. Now, upon investigation, I think it was just the quantitative nature of the person who was trying to make sense of this in his own mind, and not necessarily anybody trying to minimize their exposure.

There's got to be some other type of quantitative process that complements the scenario process to be able to keep people aligned with respect to what their exposure is. This particular business line has absolutely the largest single variation of events going on in their actual internal loss history, which shows me that they probably have the greatest potential for a major scenario. So we used that to be able to counter their argument that we haven't seen a particular event. "We've only seen one $10 million event in the last 10 years," they say. We can then counter with "Yes, but it happened last year and you're likely to see another one here pretty soon if you don't get your processes under control."

Waters:

Sandeep, you were at that meeting. Can you give us some perspective?

Vishnu:

There is an inherent natural tendency for people to do some level of gaming. Now, Todd is accurate when he said in that situation there was a person trying to get their arms around it. But we found incredible clustering around the threshold. When the threshold changed, the clustering changed back to the new threshold. And that's part of the gaming but it's also part of natural tendency. They say, "I think it's $10 million, it might be $15 million, but why do I want to put in $15 million when I don't have a good, sound basis for it?"

This is where some sort of iterative process of scenario analysis will come into play. It's almost na�ve to believe that you can get it right the first time. Can you put a stake in the ground and start to calibrate with others and then improve it as you go along? Once we got into that follow-up discussion on validation with some of the business units, we found that they were starting to separate away. Otherwise we had situations where there were six scenarios and five of them were at the threshold.

Waters:

David, how does ABN Amro determine risk scenario thresholds?

Wong:

We've taken a very practical approach. We've just said $1 million. It ties to something that we use in our capital model as a threshold to distinguish between the low-frequency/high-impact events and the high-frequency/low-impact events. We felt that that was at least an initial starting point.

We certainly haven't had the lessons learned that Todd has had but fortunately, in talking to Todd before we kicked off this event, he shared some of his thinking with us. We initially wanted to start with a lower threshold than a higher threshold, understanding that could always go higher but if we started higher we could miss some of the lower points that certainly may come to play during the discussion.

We talked about pitfalls, and to use Dan's example, one of the potholes that we're seeing for the industry is that a number of organizations take scenario analysis and think it's going to solve their capital model. They believe that you perform this scenario analysis and suddenly you're going to be able to get an accurate capital number. We would like to be optimistic about that but practically speaking, I think we're talking about a very subjective process that's difficult to validate. It could be dependent on who is in the room that day, who is facilitating the meeting, and we're optimistic about it but we'd much prefer to see this leading to good risk management and better awareness in the business about what could potentially go wrong. If we're able to use it for capital, that's phenomenal, but it's certainly not our primary focus. I don't think it's a silver bullet.

Waters:

How does a firm turn scenario analysis into action?

Loudenslager:

When we went through the initial iteration of scenario analysis we did a comparison of all the scenarios with potential risks that were already documented in a particular business unit's risk profile questionnaire. Is this a new one? Is this something that they've thought about before? Is this something that's been on their radar? And I think about half hadn't been on their radar before.

One of the follow-up actions to the meeting was to say, "If you think there's a scenario there, you should put that in clear, plain language in your risk profile questionnaire so that we document it and it becomes part of your quarterly exercise to look at it and ask whether or not it merits action." We asked, "What type of action does it merit?" So, out of the initial iteration, we were able to close the loop on things that hadn't come up before. As I mentioned, our future plans include talking about the range of these things all in the same exercise.

One of the other issues that we've noted as a general examination of our risk framework is that the risk profile questionnaires came into being because of regulatory influence. It wasn't really thought through, possibly because of the experience of the folks who were putting it together. We're trying to figure out how to make the tool itself more useful for the business line.

Audience Member:

I have a question about the potential liability of these scenarios. Say that you came up with a few scenarios and you decided to defer one of them for a year due to technology or resource constraints. What if that scenario happens the next day? What is the firm's legal liability?

Pettersen:

That question always comes up in the scenario work sessions. We always say it's nice to have somebody from legal involved in the work session. I think that's part of the exercise. But the fact that it was a scenario that should have been identified, and you didn't identify it, also points to a lack of prudent business practices. So there's exposure both ways in terms of what you identify and what you don't identify. A lot of this comes down to rationalizing, and which scenarios you keep versus which ones you do not keep.

Waters:

How much have firms spent this year compared to what they'll spend next year?

Wong:

We're just entering into this. As far as developing a program, you could say we've dedicated one full-time employee for the past two months, and then add three months to that. We chose to jumpstart ourselves with consulting help. We thought that was a good way to get some lessons learned, in essence.

Loudenslager:

For 2005, it was pretty easy to explicitly state what the cost was. I think we spent about $650,000 between outside resources and inside resources to conduct the workshops. Going forward, it's going to be much harder to parcel out which resources are being applied to scenarios because I see it as more inherent in our scenario process. We would be doing the activity of a facilitated risk assessment with the business lines even if we didn't incorporate scenarios into that. There's going to be an operating cost in conducting what is probably going to be 50 to 60 facilitated risk assessment sessions through 2006, but only a portion of that is going to be dedicated to it. As David Wong said, we found ourselves in a situation where we had one person at the beginning of the year who was specifically allocated for scenario analysis, and we had a deadline of June 30 to have all the scenarios done and doing the math, we figured out that we needed outside help so we went out and got it.Waters