In the first article in the series, we basically gave up on the Basel II definition of operational risk as a practical means of implementing an effective operational risk practice. It wasn’t the definition itself that was the problem, but the overwhelming number of nasty definitional problems that arose from losses and loss events – what are they, how do they begin, how do they end, how are they measured, what is the relationship between a loss and loss event, and so on.
In the place of the Basel II definition, we proposed an alternative definition of operational risk, one that is based on operational performance. Essentially, we generalized the Basel II definition to include all operational failures, not just those directly resulting in financial losses.i
Now on the surface, it might not look like we made a lot of progress. In fact, it may even appear that we made the problem even bigger. Sure, we no longer have to worry about the myriad questions about losses or loss events. And our new definition does have the added advantage that we can actually pinpoint the material sources of operational risk, something that the Basel II definition sadly lacks. But alas, there aren’t any free lunches in life and our new definition led to another series of rather troubling, philosophical questions, including what is operational performance and how it is measured?
In the second article in the series, we took these head on and did our best to begin to answer some of these primordial questions using a simple analogy – driving a car. Nothing too philosophical here, just push on the gas and go. And yet, even through this modest illustration, we were able to show the intrinsic relationship between operational performance and operational goals, one in which operational performance can be measured simply as the degree to which operational goals are met.
In fact, we subsequently used this link between operational performance and operational goals to informally define key performance indicators (KPI) as the measures operational performance stated specifically in terms of operational goals and objectives. We also used this link to informally define key risk indicators (KRI) as indicators of a potential drop in future operational performance when performance is measured in terms of operational goals and objectives.
Of course, this relationship between operational performance and operational goals and objectives also extends to our definition of operational risk. In other words, we can express operational risk either in terms of the degree to which an institution meets its strategic goals and objectives or a failure to meet certain performance targets. They are equivalent.
However, while we can see the relationship between operational performance and strategic goals and objectives, we are still faced with several daunting questions when it comes to operational risk, specifically:
- What are the critical goals and objectives that lead to meaningful KPI and KRI?
- How do we know that we have the right number of KPI and KRI?
- How do we connect the KPI and KRI in a way that we make sense it out of it all?
In this article, we will attempt to answer all three of these questions and tie up any other loose ends lying around to finally arrive at a practical definition of operational risk and, most importantly, one that can actually be implemented. How will we perform this slight of hand? By using a widely used performance management technique, the balanced scorecard, and something quite new, the efficient operations hypothesis.
What is a Balanced Scorecard and What Makes it Balanced?
The balanced scorecard is a performance measurement system that broadens the class of metrics traditionally used by management to measure corporate performance. It was first proposed in 1992 by two Harvard professors, Robert Kaplan and David Norton, as a means of giving management a fast, yet comprehensive, view of all aspects of their business, including those that cannot be measured directly by financials alone.ii
As presented by Kaplan and Norton, the Balance Scorecard expands the usual financial-based performance metrics to include three forward-looking performance categories: customer-based performance, business process performance, and learning and growth performance (i.e. HR measures), all of which, not surprisingly, are also operational performance metrics. The scorecard also includes specific corporate and operational objectives for each category along with specific quantitative targets for those objectives and the strategic plan to achieve those objectives.iii
The observant reader may have already seen that the balanced scorecard directly associates performance with strategic goals and objectives. And, indeed, we will shortly use this relationship to formally define our KPI in terms of the Balanced Scorecard’s strategic goals and objectives. Additionally, the balanced scorecard offers us extra benefits as well, including better overall compliance with Basel II since the Accords mentions that the scorecard’s forward-looking performance categories should also be included in any calculation of operational risk.iv
So, Where Does this Leave Us?
There is an obvious link between the balanced scorecard and our definition of operational risk since both equate operational risk with strategic goals and objectives. Moreover, the scorecard gives us a proven systematic way of defining those strategic goals and objectives.
Importantly, there is also a natural connection between the balanced scorecard and the Basel II Accords based on the scorecard’s additional forward-looking performance categories. This not only gives a way to comply fully with the Accords, it also helps reinforce our assertion that Basel II is a special case of our generalized definition of operational risk.
Unfortunately, for all these wonderful properties, the Balance Scorecard, as presented by Kaplan and Norton, still does not answer our three questions listed above. It merely helps justify our approach and make it easier in practice. More work needs to done.
From the Very Big to the Very Small
In many ways, our definition of operational risk, while described in terms of operational performance, still suffers from many of the same problems that afflict the definition proposed in Basel II. They both view operational risk at the enterprise/business-line level or the macrocosmic view – in short, the very big view. This is generally true of the Balance Scorecard, which is also typically applied at enterprise and business-line levels.
However, in order to actually identify sources of operational risk, we need true operational transparency and that means a microcosmic view of the operations. In other words, in order to identify a source of operational risk, we must be able measure operational performance at any arbitrary level of the operation. Since we won’t know what is the right level beforehand, we must be prepared to measure operational performance at every level of the operation – in short, the very small view.
But how do we transform the very big to the very small? We perform this feat of magic by simply applying the Balanced Scorecard, albeit somewhat unorthodoxically, to every component of the operation, no matter how small. That’s right, we get operational transparency by applying the Balance Scorecard to every process, person, system, bit of information, and piece of infrastructure used to support all business and related activity.
While there is nothing in the definition of the balanced scorecard to prevent us from doing this, it sure sounds like a lot work. Furthermore, while we were transforming the very big to the very small, we also went from a problem with too little data to a problem with apparently too much data and more practical problems. For now, let’s just take this path on faith and see where it leads.
Extending the Balanced Scorecard
Although applying the Balance Scorecard to every level of the operation may help us achieve true operational transparency, we still don’t know what to measure. Now, it is pretty easy to see that if the strategic goals and objectives can be expressed quantitatively,v they will lead us to some sort of KPI. However, this is a lot to rest our hopes on. For instance, we can’t be sure if the scorecard will lead to all of the KPI or even the most important ones. What’s more, the strategic goals and objectives can’t really help identify the KRI.
We can overcome these setbacks, if we simply add some of the other traditional strategic factors to the balanced scorecard analysis to get the missing information we need, specifically:
- The Mission Statement – the fundamental purpose of the operational component in question
- Constraints – practical limitations on the performance of the operational component
- Critical Success Factors – the requisite conditions necessary to meet the desired performance targets for the operational component
It is fairly easy to see how these additional factors help us. First, if we align the goals and objectives against the specific mission of the operational component, we can prioritize them as well as ensure that we have the most important ones. Secondly, by adding the constraints and critical success factors, we have included all the issues that can limit performance, or as we saw in our driving analogy, the KRI.
So, Can We Finally Define the KPI and KRI?
Using our extended Balance Scorecard, we are now in a position to formally define two of the most critical concepts in operational risk management: KPI and KRI.vi
Key Performance Indicators – A key performance indicator is quantitative metric representing one or more goals or objectives for a given operational component
Key Risk Indicators – A key risk indicator is quantitative metric representing one or more critical success factors or constraints associated with a given KPI
With these two definitions, we have answered partially questions 1 and 3 above.vii However, we still have not addressed problem of what is the right number of KPI and KRI and we are still up to our neck in data. Fortunately, we still have one more trick up our sleeve.
And Now for the Efficient Operations Hypothesis
Essentially, the efficient operations hypothesis boils down to the straightforward belief that an operation is efficient when all the goals and objectives of its components are well aligned with the overall corporate goals and objectives.viii Seems logical. To see this, let’s use a simple thought experiment.
Intuitively, we can assume that a strategic goal of any operational component should be helping the company achieve its overall corporate objectives as defined by the company’s KPI. However, the company can only meet its strategic goals, if it first meets its KRI. Therefore, it seems clear that any operational component must have KPI that include the corporate KRI. If not, the company as a whole could never reach it goals and objectives.ix
Now, let’s assume that the operational component has a KPI that is not a KRI of the company. In this case, the operational component would be consuming resources in order to meet that objective. Of course, there would be a cost associated with these resources that would have to be borne by the company. However, from the company’s point of view, this would be an unnecessary cost since it doesn’t help it meet any of its objectives. Hence, the operational component’s KPI should be implying that the KPI of an operational component must not exceed the KRI of the company. We can use this same argument to show the same argument across all the operational components to show that the sum of all the KPI must be equal to the KRI of the company.x
Are We Done? Not Quite
With our extended balanced scorecard and the efficient operations hypothesis, we have finally answered our three questions above. By applying the extended balanced scorecard first at the corporate level and then at each subsequent level of the operation, we can generate all the key KPI and KRI necessary to compute operational risk. Using the efficient operations hypothesis, we can reduce this set to only those indicators which uniquely describe operational performance, hence, operational risk.
Although we will still have a number of practical problems to tackle, we’ve really come a long way. But before we deal with these practical issues once and for all, in the next article in the series, we will go back to Basel II to show that with this approach, we are finally ready to comply with all the AMA requirements.
i Specifically, we defined operational risk as the probability that the operation will fail to meet one or more of its defined operational performance targets. See Vinella, Peter, “Operational Risk 101 – The Basic Definitions”, gtnews, November 2004 (gtnews.afponline.org)
ii See Kaplan, Robert S. and Norton, David P., “The Balanced Scorecard – Measures That Drive Performance”, Harvard Business Review, January-February 1992
iii As a great deal has already been written about the Balance Scorecard and measurement-based management, in this article, we will only go in those aspects which a pertinent to our discussion. Those interested in more information should visit www.balancedscorecard.org
iv We’ll explore the more of the benefits of the balanced scorecard more fully in the next article
v For instance, simply stating the strategic goal, “we want to have better operations” leads naturally to the question, “what does better mean?” As such, one could argue that the only meaningful goals and objectives are quantitative
vi While terms such as KPI, KRI, Loss Events, and Operational Risk Factors, have been used somewhat interchangeably in the subject of operational risk, here, we have tried to formalize these definitions as much as possible. As such, they have distinct meanings in our framework
vii Mathematically, we can show that, in fact, a KPI can be expressed as finite sum of KRI to any degree of accuracy that we desire. See Vinella, Peter, “Describing a Formal Foundation for KPI and KRI”, Operational Risk, November 2004 (www.operationalriskonline.com)
viii Actually, we will see later that this is both a necessary and sufficient condition
ix Of course, this argument assumes that management has set realistic goals and objectives in the first place. Not necessarily a safe assumption
x This is a simplification of the actual proof which consists of layers of operational components. For the full proof, see Vinella, Peter, “Describing a Formal Foundation for KPI and KRI”, Operational Risk, November 2004 (www.operationalriskonline.com)