Developing a Governance Process to Manage and Control Model Risk

Models as tools have undoubtedly become an important means of evaluating, analyzing and reporting key business information at most large organizations. They help calculate or estimate results based on certain data. As organizations have sharpened their focus on operational-risk issues, many have identified model risk as an area requiring a particular framework, as well specialized governance, methods and approaches. This awareness has increased in the current market climate due to the following factors:

• The growth of model use.
• Increased regulatory directives.
• Complexity of decision-making.
• External reporting demands.

Considerable operational risks are associated with the use of models, not the least of which is that organizations may report inaccurate financial results or make poor decisions based on a faulty model or spreadsheet.

In many cases, model risks have begun to outpace the infrastructure that organizations have in place to manage them. In this environment of tight financial reporting and other governance controls, leaders are advised to make sure they have a firm grasp over their models, as well as a process for assessing and managing this risk.

A Sizeable Challenge

Just as the use of models has expanded recently, the complexity of these tools has also increased. The improved quality of third-party modeling software and homegrown spreadsheets has made the use of complicated models a routine facet of decision-making. Models are now used regularly for efforts ranging from calculating fair-market value to estimating enterprise-wide risk.

As models flourish, regulators have been reviewing their use more closely than in the past. Following the lead of control developments supported by regulators – the Sarbanes-Oxley Act of 2002 and Basel II, for example – and industry groups, a number of organizations are now intending to:

• Document each model’s theory and application.
• Define a regular model-validation process.
• Conduct an independent model validation, separating development from implementation efforts.
• Enforce strong model change controls.
• Establish controls around model inputs.
• Conduct model testing to determine accuracy, strengths and weaknesses.

Are Risks Fully Managed?

Although an organization’s use of models affects many aspects of its financial health and reputation, model development and validation is not typically an item on senior leadership’s agenda. Many leaders may not have the time or skill set to evaluate the suitability, limitations or other aspects of the models they rely on.

As part of their oversight responsibilities, board members and senior management need to be sure they are fully aware of their organization’s exposure to model risks and whether those risks are understood and managed.

Senior executives at many organizations may also not be fully aware of how many models are in place, whether they remain valid and if they are used appropriately. For instance, a model might be changed or altered by one user without the knowledge or input of other users. When that happens, leaders need to determine whether the model’s underlying assumptions remain legitimate.

Addressing the Challenges

Lessons learned during the compliance efforts surrounding Sarbanes-Oxley have highlighted challenges and risks associated with financial reporting and controls. Now, organizations should ensure that they apply that experience and knowledge to managing the risks associated with models. Organizations can address these risks by developing a framework for model control that provides for models to be governed from an enterprise-wide perspective, although they can be managed individually.

Such a framework encompasses the models (financial, decision support, risk management), their risks (misapplication, improper implementation, misspecification), and the source of those risks (data, method, process and governance). It is governed by a process of model control and validation as well as a policy for model governance. The framework is built on a foundation of the cataloging and measurement of the models themselves.

Chief risk officers would determine, for example, how many models they have, what they are used for, to what extent the organization relies on them, the trustworthiness of the criteria used to populate them, and the decisions that are based on them. This process can be considered in three phases.

During phase one, risk managers would:

• Define what a model is and is not.
• Produce an inventory of all models used in the business.
• Categorize the models into usage classes.
• Rank the models according to risk using criteria, such as financial impact, existing controls, robustness, cost benefit, consistency and input reliability.

Phase two would focus on the definition and application components for specific models. Definition components include model policy, model descriptions and other processes designed to minimize model risk. Application components include model validation and testing. The goal of Phase II is to achieve better practice and improve the entity’s management of models.

Finally, phase three focuses on evaluating the effectiveness of phase II.

Conclusion

External stakeholders are demanding improved management of model risk, and market leaders have made it a priority. Board members and senior executives need to ensure that they understand the risks their organizations face and that steps are taken to manage them appropriately.