Operational Risk 101: Management by Fact

So, here we are at last, the final article in this introductory series on operational risk management. In the previous five articles – Operational Risk 101: The Basic Definitions; Operational Risk and the Basel II Accords; Operational Risk in Terms of Operational Performance; Operational Risk 101: Tackling Basel II; Operational Risk 101: Roles and Responsibilities – we made enormous strides in developing a practical and implementable approach to managing operational risk. In fact, I think it is fair to say that we accomplished nearly everything that we set out to do and, maybe, even a little more. As a crowning achievement, we were ultimately able to transform operational risk management from a passive, backward-looking function into a key leg of a pro-active, forward-looking, and effective corporate governance practice.

Over the course of our journey, we also discovered a number of surprising and unexpected aspects of operational risk, some of which constituted rather important results in their own right – I must admit that some were a surprise to me as well.

As this is the last article in this introductory series, it seems appropriate to recap some of these more important findings. However, to make a little more sense out of it all, I have taken the liberty to group these by their underlying motivations as opposed to strictly following the flow of the earlier articles – oh yes, there was some method to the madness.

Basel II AMA Compliance and Capital Calculations

The basic motivation behind all this was to find a practical way to compute the Basel II AMA capital requirement for operation risk.¹ Having failed following the two most popular approaches, COSO and OpVaR, it seemed pretty obvious that we were going to have to try something radically different.

So, seemingly at a dead-end, we tried a fairly common mathematics trick of transforming an apparently unsolvable problem into something that we knew how to solve.² Though the route was a bit circuitous and the outcome not always certain, in the end, we were ultimately successful.

Recapping some of the highlights, we found:

• Operation risk can be expressed in terms of operational performance rather than in terms of financial losses due to operational failures and still be consistent with the Basel II definition.
• As a function of operational performance, operational risk can be expressed as the possible financial impact of missing a specific performance target.
• By applying error tolerances to the performance targets, we defined an expected loss as the financial loss suffered by missing a performance target within a given error tolerance. Conversely, we defined an unexpected loss as the financial loss resulting from falling outside the tolerance level.
• With this in mind, we were able to calculate the Basel II AMA capital charge in terms of estimating the amount of unexpected losses (i.e. the expected unexpected losses) over all performance targets within the enterprise, over a given time horizon, within a given confidence level.

I must admit that is pretty impressive. And yet, that is only one pillar of our work – I know, a really tasteless pun, but this is the final article after all.

Risk Factor Identification – The KPI and KRI

Above, we transformed the problem of computing operational risk from the difficult task of estimating future losses due to operational failures into one of measuring operational performance and its variance from desired targets. However, that left us with the problem of how to systematically define and measure operational performance. Moreover, this had to be in terms of quantitative metrics, something not widely used in the financial services industry – no easy task.

So, yet again, we were forced to establish a formal theoretical foundation so that we could identify such factors in an intuitive and practical way. Fortunately, we were successful and ultimately established the following results:

• First, we extended the traditional balanced scorecard in such a way that we could easily define operational metrics and performance targets (a.k.a. key performance indicators or KPI) in terms of the goals and objectives of a given organization within the enterprise
• By using this modified balanced scorecard, we formally identified the critical success factors necessary to meet these performance targets. Through this, we were able to define leading indicators of performance that we dubbed key risk indicators or KRI.
• Through the efficient operations hypothesis, we were able to explicitly link the KPI and KRI for a given organization to the overall corporate goals and objectives. This allowed us to express corporate performance in terms of the performance of each individual organization within the enterprise. Mathematically, this took the form of an iterative relationship between the KRI of a given organization and the KPI of its subordinates.
• Next, we applied the fundamental operational objective in order to explicitly tie the performance of the overall enterprise to the performance of every element of the operation – each process, procedure, information, system, and person – to the overall corporate goals and objectives. In other words, we developed a formulaic method to construct KPI and KRI for every aspect of the operation that subsequently could be used to mathematically express operational performance – hence, operational risk – at any arbitrary level of the organization.
• Lastly, we realized that it was not necessary to measure performance at every element of the operation in order to estimate operational risk. While this gave a gauge of overall operational efficiency, for operational risk, we only had to measure the performance of each internal control.

These five points allow us to establish the means to systematically define and gather operational performance data at each component of the operation across the entire enterprise. Moreover, this information can be aggregated in such a way that it can provide us with an overall macrocosmic measure of operational performance. While this has extremely important applications outside of operational risk management, in terms of estimating operational risk exposure, we had all the tools we needed.

Aligning Corporate Governance and Operational Risk Management within an Integrated System of Internal Controls

Indeed, expressing operational risk in terms of operational performance and then measuring the performance at each internal control gave us a practical means of computing Basel II capital requirements.³ More importantly, however, our approach also provided a number of the surprising by-products. The biggest of these was undoubtedly the fact that our approach actually integrated a financial institution’s corporate governance, operational risk management, and internal controls in a meaningful way, one that we thought important enough to give it a name, the Integrated System of Internal Controls.

While consistent with other proposed control frameworks, such as COSO,⁴ the Integrated System of Internal Controls has several important advantages, including:

• Our framework provides a full operating model for corporate governance and operational risk management that includes organizational duties, roles, and functions that are both intuitive and regulatorily compliant. Through this relationship, the staff can effectively interpret and implement executive management’s rules and directives. Moreover, it provides management with a secondary and independent source of information regarding impact of rule and directives on the operations as well as the degree of compliance with their rules and directives.
• Our framework not only gives management the means to quantitatively measure operational risk arising from the performance of the operations and internal controls, it also highlights sources of systemic operational risk resulting from the very design and implementation of the operations and controls. Further, our framework ties operational failures, whether structural or execution, back to the specific causes and sources within the operations.
• Additionally, our framework gives the means to measure both potential and actually operational efficiency, something well beyond the scope of the other frameworks. This allows management to not only reduce operational risk by implementing better controls, they can also reengineer the legacy operations to remove the need for unnecessary or redundant processes and controls.
• Most importantly, our framework allows management to accurately judge the effectiveness of the overall operations using timely and independently generated information. This allows them to quickly react to franchise threatening failures before they get out of hand, something executive management in many institutions of late wished they had.

So, although we started out trying to find a reasonable way to compute Basel II operational risk capital requirements, we ended up establishing the basis of a full-blown theory on corporate governance and operational risk.

Oh Yes, Every Rose Does Have Its Thorns

Now, before you go crazy building operational performance and risk dashboards and run off firing all of your supervisors and line-managers, a few words of warning.

You won’t be able run a financial institution with metrics alone. No one can. Performance metrics are not intended to replace the human element of your management. They were designed simply to provide additional important information that you can use in conjunction with your normal management structure in order to improve your operations and maintain its level of performance.

We have spoken with a number of senior managers who envisage a single digital control room populated with hundreds of flat-panel screens each displaying the status of a host of metrics placed at every critical point in their operations. Further, this data is not gathered monthly or even daily, but in real time, automatically adjusting the internal controls to meet changing internal and market conditions. While this may sound like science fiction, there are a number of financial institutions that are actually attempting to create such capabilities, even if it is only on a limited scale.⁵

While this might be a reasonable goal for a manufacturer – and even here I have problems – this model clearly doesn’t fit the financial services industry for a variety of reasons.

First, the industry is extremely dynamic. While it’s true that manufacturers, such as Sony and HP, introduce products on a nearly continuous basis, the production models for these products are very similar and, once developed, they are extremely static. Unfortunately, a financial institution’s processing is anything but static since it must comply with terms negotiated with external counterparties that can be unique for every transaction.

Secondly, it will take significant time and effort to get to the point where you will have the necessary technical infrastructure to replace your middle management. At some point during this effort, your staff will become aware of your ultimate plans and find subtle and not so subtle ways to sabotage the entire project. Anyone who has tried to outsource a major part of the operations knows what I am talking about.

Lastly, and most importantly, once your controls are fully automated, fraud actually becomes easier. It is impossible to predict all possible actions; hence, every automated control is limited. Once someone has found the vulnerabilities of a given set of automated controls, they can easily exploit these to reap substantial ill-gotten gains – just ask Joe Jett.⁶

People, on the other hand, generally adapt to change much more quickly than systems and people are much less predictable than systems. Contrary to what you might have experienced going through airport security, as long as your staff pays attention and work diligently, manual controls can greatly enhance the performance of automated ones.

Therefore, before you begin building Norad II,⁷ you should design a metrics system that augments your management. Specifically, design a performance monitoring system that can validate your staff’s status reports to help make better strategic decisions as well as react more quickly to potential franchise threatening events.

If you remember our driving example back in the second article of the series (Operational Risk and the Basel II Accords), most of us only look at the speedometer and the gas gauge while driving. In fact, we really don’t have time to do much else. If we had to watch a wall of dials and gauges giving us a real time status of every component of the car, we wouldn’t have time to watch the road – in New York, you wouldn’t even get a block before hitting something. As for warnings of possible failures, a simple yellow light warning directing us to the nearest mechanic seems to suit everyone I know.

Similarly, a performance measurement system should be an important source of information and a crucial management tool, but it should be the only tool. Remember, the goal of a business is to make money. Good controls help, but controls by themselves aren’t very good revenue producers.

¹ See AMA quantitative standards, page 144, “International Convergence of Capital Measurement and Capital Standards: a Revised Framework”, Basel Committee On Banking Supervision, Bank of International Settlement, June 2004 (Basel II).

² A good example of this is Einstein’s Theory of Special Relativity in which all motion, no matter how apparently complex is greatly simplified when expressed in terms of its inertial frame of reference. (A short-note from the author… I had a professor, Isadore Singer, that stressed that a well placed reference to Einstein always got a paper published – a lesson I never forgot).

³ Remember that most financial institutions already have a number of reports available at each internal control that we can leverage.

⁴ See “Internal Control – Integrated Framework”, Committee of Sponsoring Organizations of the Treadway Commission, September 1992 (a.k.a. COSO) .

⁵ For example, a number of large broker/dealers, including JPMorganChaseBankOne (man, that is one hell of a name), have been experimenting with replacing all of the equity market makers with algorithmic trading, primarily to reduce cost. While this has only met with limited success to date, the success of commercial vendors offering such software clearly shows the dealers commitment to this strategy.

⁶ Vinella, Peter. “Joseph Jett’s Phantom Bets”, Derivative Strategies, May 1999.

⁷ For those of you who do not know “Norad” (clearly, you also have an extreme dislike for science fiction), “Norad” stands for the “North American Aerospace Defense Command”, a joint U.S./Canadian military organization which uses a wide variety of technology to monitor possible military threats to North America from the air or outer space.