Operational Risk and the Basel II Accords
In the first article of the series, we made several important steps towards understanding operational risk in practical terms. First, we linked together three important concepts: operational performance, cost, and risk. Quite simply, in order to achieve a desired level of operational performance, there is a natural trade off between cost and operational risk – once you have reached your minimum cost for a given level of performance, operational risk will increase with further cost-cutting.i
Second, we were able to directly state operational risk in terms of operational performance and performance targets as opposed to the more common Basel II definition which stresses operational losses.
Now, we didn’t actually show that these two definitions are equivalent, we just implied that they are equivalent and postponed the proof until a future article.ii We also carefully avoided the sticky problem of formally defining the terms, “operational performance” and “performance targets”. We simply stated that operational risk is the probability that our operations will fail to meet one or more performance targets and left it at that.iii In this article and the next, we are going to take them both head on.
Of course, we will be very careful to link our work to Basel II to make sure that in the end, we are still compliant with the Accords. But as you will see, our approach has many practical advantages, not the least of which is a theory of operational risk that is intuitive and easy to understand.
Unless you have been under a rock for the past couple of years, I’m sure that you have heard the terms, “Key Performance Indicators” (KPI), “Key Risk Indicators” (KRI), and “Operational Risk Factors”. You have probably used them yourself. But do you really know what they are?
Sure, they are used to describe possible sources and measures of operational risk – quite interchangeably I might add. And everybody seems to talk about them like old friends who need no introduction, but just what are they? How do you identify them? How are they used to describe operational risk? How do they fit into Basel II?
It was thinking about these very questions three years ago that got me and my co-author started on the road to developing a comprehensive theory of operational risk in the first place.iv Even though Basel II requires the use of operational risk factors, nobody, including the regulators could tell us if KPI were KRI or if they were different. Or what they should measure or how? Or how often they should be measured? Did they span the enterprise or were they business specific? Did they span multiple companies? Ad infinitum…v
Other than the fact that you had to use them somehow, nobody knew too much about them and so our journey began. Over the next three of years, we did succeed in answering most of these questions and while it wasn’t necessarily easy, along the way, we did develop a simple analogy to help describe KPI and KRI and their very critical role in the successful management of operational risk.
Modern automobiles are quite complicated machines. Dozens of microprocessors adjust nearly every aspect of a car’s operation, from the fuel/air mixture to the tightness of the suspension in a turn – all in realtime no less. Fortunately, while driving, most of us only have to worry about two gauges on the dashboard: the speedometer and the petrol gauge. With just these two indicators, we can safely drive to our destinations as fast as possible without getting a ticket or waiting for hours on the side of the road for petrol – provided, of course, that we actually pay attention while driving to avoid accidents, maintain our course, avoid traffic, etc.
If you think about it a bit, that’s pretty impressive. Ignoring driving skills, a driver only has to worry about his speed and the amount of gas while driving and nothing else regarding the performance of his car. But what does this have to do with KPI and KRI?
To answer this, let’s assume that on a specific trip we would like to get to our destination as quickly as possible – not an unreasonable goal. In order to do that, we need to drive at the highest speed permitted, which is, of course, measured by our speedometer. In other words, for us to achieve our goal, we need to attain a certain level of operational performance: in this case, speed. Hence, it is not too hard to infer that speed is a key performance indicator while the speedometer is how speed is measured.vi
This begs the question that if speed is a KPI, is the amount of gas also a KPI? Well yes and no. As we drive along, the car consumes gas. Although the rate of gas consumption may be an indicator of engine performance, in terms of our specific goal – getting to our destination as quickly as possible – the amount of gas is only a factor when it’s gone, i.e. when our speed drops to zero because we have run out of gas or have to pull over for a fill up. In other words, the amount of gas is not a direct indicator of our ability to achieve our goal, rather, it simply indicates that our speed may drop in the future, i.e. it is a risk that we may not achieve our required level of performance and, hence, miss our goal.
Although equally important, speed and the amount of gas are fundamentally different indicators and we really can’t call the amount of gas a KPI since it is a measure of performance risk, not performance. As clever academics, we will coin the term “key risk indicator” for the amount of gas as well as all those factors that describe the risk of a future drop in performance.
The car analogy also brings up another interesting aspect KPI and KRI: they are not absolute, but are based on our choice of operational goals. The importance of speed and the amount of gas in our example was due to the specific goal of our trip – to get to our destination as quickly as possible. If we were to choose a different goal, say, riding in style and comfort, speed and amount of gas wouldn’t be too helpful and we would need performance and risk measures that better reflect style and comfort such as the smoothness of the ride for example.
Therefore, in order to identify meaningful KPI and KRI for any operation, we must first define our operational goals (i.e. performance target) – we need to know what we want to achieve in order to measure how well we are achieving it. This may sound pretty trivial, but it has big ramifications.
Since KPI and KRI are specific to our operational goals, it is unlikely that they all will span multiple business lines or companies unless all of us happen to share the same goals. While some goals are fairly universal – maximize profits, reduce error rates, improve customer satisfaction, etc. – many will be specific to a particular line of business: lying on the efficient frontier may be a great goal for an investment portfolio, but it doesn’t mean a whole lot to AP/AR department.
Secondly, goals, as well as their priority, change over time; hence KPI and KRI must also change if we are to accurately reflect the performance. On top of this, while goals may change quickly, the operation itself doesn’t. As a result of both of these drift factors, it is quite possible that our operations, as measured by outdated or misaligned metrics, may appear to be working great when we are actually losing money left and right due to unseen operational failures and inefficiencies. Additionally, reaching our performance targets may be misleading. For instance, if my daily Repo daylight overdraft target is $100K, just because I only lose $90K doesn’t mean I have great operations. It could just mean that I have very low expectations.
The car analogy also points to another problem with KPI and KRI. In the example, as drivers, we only had to be concerned with speed and the amount of gas. However, we were really relying on the car’s numerous microprocessors to measure a large number of hidden KPI and KRI and optimize the overall performance of the car. If fact, there may actually have been hundreds of important KPI and KRI that we are simply unaware of due to a lack of operational transparency. As long as the microprocessors worked correctly, we were fine. However, while this lack of operational transparency may be a benefit in driving a car, in a financial operation, you could end up in jail.vii
While the Accords leave much to be desired in terms of concrete, practical definitions, the definitions of loss event categories and business lines do help make the problem of defining industry-wide KPI and KRI much easier.viii To be consistent with the Accords, we label the most important KPI and KRI – defined in terms of our specific goals – as “Operational Risk Factors” (ORF), which can be used for scenario analysis and regulatory reporting, both Basel II requirements.
Ideally, we use the 80/20 rule to select the 20% of the KPI and KRI that explain 80% of our operational risk – for instance, in the car example, we only needed one KPI and one KRI to explain most of the risk. At worse, we can simply use those indicators we have on hand until we have better operational transparency and a bigger, more accurate collection of KPI and KRI.ix
Once we have selected the most predictive KPI and KRI, we can map them to the Basel II loss event categories and business lines to construct ORF which are consistent with general industry definitions. These can be used for regulatory reporting purposes as well as benchmarking against external data, both Basel II requirements.x
While the Accords do very little in helping us measure operational risk in terms of our specific operational goals and do very little in terms of achieving operational transparency, they do give us a way to at least standardize the definition and reporting of KPI and KRI across lines of business and companies. However, in practice, good operational risk management requires KPI and KRI that are tied directly to specific and idiosyncratic goals of our unique operations.
In this article, we established a general definition of KPI and KRI in terms of performance and particular operational goals of a business line or the enterprise. We also saw that the KPI and KRI are very dependent on the specific choice of operational goals and except for the mom and apple pie indicators, they don’t easily span businesses nor are they constant in time. We also saw that due to a lack of operational transparency, we may only find a small fraction of the important indicators without some significant detective work.
We also saw that we could use the Accords to help define a smaller, more intuitive set of KPI and KRI – the ORF – that can be used for regulatory reporting and external benchmarking. In the next article, we will take on the somewhat Herculean task of establishing a practical way of identifying all the critical KPI and KRI in any operation, one that will ultimately provide full operational transparency.
Fortunately, we won’t have to this on our own. In fact, we will be able to borrow quite a bit from other disciplines that will make this a whole lot easier. And we will step briefly into academia with a look at the “Efficient Operations Hypothesis” which will cleverly tie all the KPI and KRI to the overall corporate objectives. Now that’s something to look forward to.
i For those of you who can’t wait, we will see in a future article that this is completely analogous to Modern Portfolio Theory’s Efficient Frontier, but let’s not get too far ahead of ourselves just yet
ii While we won’t get to this in this article, don’t worry, we’ll get to it in the very next article
iii In the next article, you might be surprised to see that our primarily performance target will be increasing shareholder value
iv This a reference to Jeanette Jin as well as a shameless plug of our upcoming book, “Corporate Governance and Operational Risk Management: A Practical Guide, currently slated for release through John Wiley & Sons in Q2 2005.
v Unfortunately, this lack of formal definitions and rigor, especially on the part of the regulators, is one of the main reasons that it is so damn hard to implement an operational risk management practice. Definitions seem intuitive and obviously on the surface, but when you try to implement the concepts, there is far too ambiguity and room for interpretation – just one of my pet peeves
vi What metric is measured and how it is measured are often confused with regards to KPI and KRI. We deal with this more in a future article
vii This balance between too much information and too little with be discussed at length in a future article
viii See the Accords for the complete list of loss event types and business lines
ix There are a variety of statistical techniques that are outside the scope of these articles, such as “Spectral Analysis”, that can be used to identify the number factors as well as help identify the factors themselves . However, we will touch upon this issue on a less theoretical level in future articles
x This can be done via a simple linear regression. This will covered in more detail in the next article