The Basel Finish Line
Basel II is intended to make banks better align their regulatory capital with risk and strengthen their risk management and disclosure. Regardless of the approach taken to comply with the Basel requirements-the Standardized, Foundation Internal Ratings Based (FIRB), or Advanced Internal Ratings Based (A-IRB)-banks will face significant challenges in documenting their credit policies and processes, quantifying the risk attendant with their portfolios, and demonstrating to both regulators and investors that they have properly provided for anticipated risks.
An IRB system allows banks to implement a systematic approach to credit pricing and assessing risk for capital allocation. Banks will use their IRB systems to associate a probability of default (PD) with each obligor grade, as well as a loss given default (LGD) with each credit facility. In addition, institutions will estimate exposure at default (EAD) and will calculate the effective remaining maturity of credit facilities.
Because of this reliance on the bank’s own information in the estimation of risk and related required capital protection, Basel II touches on validation of the bank’s systems, estimations, calibrations and how these elements can be satisfactorily demonstrated to its regulator. Because of its focus on how banks need to determine and communicate their compliance, the validation framework provides a useful lens to view current bank activities and the hurdles banks are facing.
Virtually no bank in the world has a system that fully meets all of the Basel II requirements. Existing systems may, for example, do an excellent job of determining an obligor’s risk of default, but fail to provide sufficient information regarding losses in the event of a default. Different units within the bank may deem an obligor in default using widely different definitions.
While performance data on the new system will be forthcoming, that information does not currently exist and most banks will have only limited experience under the new system by the official launch date of 2006. A great deal of weight may initially be placed on evidence that the system has been thoughtfully and carefully developed and implemented to provide comfort and confidence that it should perform well. This is called Developmental Evidence.
Developmental Evidence reflects all the various elements in a credit evaluation system that underpins the expectation that the system should perform well. It’s just like it is impossible when buying a new car to know for certain that it will not be a lemon, the proof will only come with time.
However, before committing to the investment in and launch of the system, we can look to a whole host of factors to raise our confidence that the system will perform as intended. A few examples of developmental evidence are segmentation of the credit portfolio into certain sector or buckets which should respond in a fairly homogeneous manner to changes in credit conditions; the comparability to systems utilized by the bank or it peers; and the historical performance of those systems
For many banks the bulk of their investment is spent on developing the IRB system and the review process is treated as a secondary issue. Review is done on an ad hoc basis and is undertaken only after serious shortcomings are revealed or, for example, when a new chief risk officer is appointed. However, it is essential that the process for building and calibrating a system and the parallel process of reviewing that system work together hand-inhand.
Another important aspect of the review process is who within the organization undertakes the review. While those who construct the system may have the best technical knowledge of it, they will probably not be the best to evaluate its performance. There is an inherent conflict of interest built into a review conducted by the creators of the system who inherently carry with them a vested interest in seeing it perform well.
Benchmarking will be especially important in the next several years as banks build up their history with new rating systems. Benchmarking refers to a range of activities that allow a comparison between your bank’s internal rating of a credit and a third party’s rating of the same credit. This examination will both provide a test of the performance of the bank’s internal system as well as supplementing internal data with comparable information from wider sources. Benchmarking applies equally well whether the IRB system is judgmental, based on a statistical model or a hybrid of several different approaches.
Great care is necessary to obtain optimal results from benchmarking and will require considerable judgment and skill to perform it well. The essence of good benchmarking revolves around the proper identification of what is truly comparable across different systems and to make the appropriate adjustments to the factors to interpret results correctly. These judgments start with the definition of default (or loss given default) and extend to the comparability of experience across different credit portfolios, time and geography.
The final piece of ongoing validation is backtesting. Backtesting compares predictions to actual outcomes, looks at the combined outcome of the rating assignment and the parameter estimation processes. It is important to emphasize that while backtesting will be a key part of the validation process, it is still only one part. Backtesting is not synonymous with validation and all other aspects of validation must be met in addition to backtesting.
Two serious issues arise in regards to backtesting. First, it will take time, perhaps years, with new systems being established to obtain meaningful results. Second, backtesting focuses on a single point in a broad interlocking system all of whose components need validation.
Obviously the bottom line”of the rating exercise is the creditworthiness score and the how this relates to the probability of default. From that standpoint, testing the prediction of default probability against the actual outcome for a group of comparably rated entities is the ultimate test.
Are these all the issues and potential hurdles to having a successful, validated IRB credit system? Certainly not. However, by planning now, as IRB systems are being developed and implemented, banks will have a good leg up on the process and will make accommodation to later changes much, much easier.
This article originally appeared in Waters Magazine March, 2004