10 Things all Internal Auditors Must Know about Model Risk & Model Risk Management

Model risk has become one of the most critical areas of concern for internal auditors, risk managers, and regulators alike. In today’s data-driven world, organizations across industries—from global banks to manufacturing firms—rely on models to drive decision-making. These models power everything from credit risk assessments and interest rate forecasts to supply chain optimization and AI-driven customer insights. However, when models fail, the consequences can be catastrophic, leading to massive financial losses, regulatory penalties, and reputational damage.

Unlike traditional operational risks, model risk is unique because it stems from uncertainty, incorrect assumptions, and flawed data. A model is only as good as the inputs and logic behind it, and even small errors in design, data, or assumptions can create outsized risks. This is why regulators such as the Federal Reserve (SR 11-7), the OCC (Bulletin 2011-12), and international bodies have established robust guidelines for Model Risk Management (MRM). While financial institutions have long been at the forefront of managing model risk, non-financial firms increasingly rely on complex models and face similar challenges.

For internal auditors, understanding model risk isn’t just a technical necessity—it’s a fundamental requirement to provide assurance over the integrity of business decision-making. Internal audit teams are expected to evaluate governance structures, challenge model assumptions, and ensure ongoing validation efforts are effective. As AI, machine learning, and algorithmic decision-making continue to evolve, auditors must be equipped to assess these new risks with the same rigor as traditional financial models.

1. Model Risk Is Everywhere – Even Outside Financial Services

Model risk is often associated with banks and financial institutions, but its relevance extends far beyond finance. Every organization that relies on data-driven decision-making, predictive analytics, or automated systems is exposed to model risk. In financial services, models drive loan approvals, capital adequacy assessments, trading strategies, and risk forecasting. However, in non-financial firms, model risk can impact everything from supply chain logistics to marketing strategies and product pricing.

For instance, retail and e-commerce companies use demand forecasting models to optimize inventory management. A flawed model can lead to overstocking, tying up working capital, or understocking, leading to lost sales and customer dissatisfaction. Similarly, manufacturing companies leverage predictive maintenance models to anticipate equipment failures. If these models are inaccurate, firms may either overspend on unnecessary maintenance or suffer unexpected downtime.

Healthcare organizations also rely heavily on models. Predictive analytics in patient care, disease outbreak forecasting, and hospital resource allocation all depend on models that, if flawed, can compromise patient safety or lead to financial inefficiencies. Energy companies use models to forecast electricity demand and optimize grid stability, while insurance firms depend on actuarial models for pricing policies and managing claims risks.

One of the biggest takeaways for internal auditors is that model risk isn’t confined to finance—it affects every industry that uses models for critical decision-making. As more businesses embrace artificial intelligence (AI) and machine learning (ML), model risk is becoming an even more pervasive challenge. Internal auditors need to assess whether their organizations have recognized and addressed the risks associated with model reliance, even if they aren’t in the banking sector.

2. SR 11-7 and OCC 2011-12 Set the Standard for Model Risk Management

Model risk management (MRM) has evolved significantly, particularly in response to regulatory scrutiny following the 2008 financial crisis. The Federal Reserve’s SR 11-7 and the OCC Bulletin 2011-12 are the gold standards for MRM and provide guidance that is widely adopted even beyond regulated financial institutions.

Key Takeaways from SR 11-7

Issued in 2011, SR 11-7 (Supervisory Guidance on Model Risk Management) outlines the Federal Reserve’s expectations for how banks should manage model risk. The guidance emphasizes:

• Model Development & Documentation – Models should be built with robust methodology and have clearly documented assumptions, inputs, and intended uses.
• Independent Model Validation – An independent team should validate models before deployment and regularly thereafter.
• Ongoing Monitoring – Model performance should be continually assessed through backtesting, benchmarking, and sensitivity analysis.
• Governance & Oversight – Banks must establish clear policies, define roles and responsibilities, and involve senior management in model risk governance.

OCC Bulletin 2011-12

The OCC (Office of the Comptroller of the Currency) issued Bulletin 2011-12 around the same time as SR 11-7, aligning closely with its principles. OCC guidance highlights the need for formal model risk management frameworks that include:

• A clear definition of what constitutes a model
• A model inventory covering all in-use models
• Policies for model validation and independent review
• Strong internal controls to prevent misuse or overreliance on flawed models

Why These Standards Matter for Internal Auditors

Even companies that aren’t regulated financial institutions can benefit from applying SR 11-7 and OCC 2011-12 principles. If an organization depends on data models for critical business decisions, it should have clear governance, validation, and monitoring mechanisms in place. Internal auditors should assess whether model risk is being properly managed across departments, from finance and risk management to marketing and operations.

3. Strong Model Governance Is the First Line of Defense

A model is only as good as its governance. Model governance refers to the policies, structures, and oversight mechanisms that ensure models are properly developed, validated, and monitored. Without strong governance, models can be manipulated, misused, or left unchecked—leading to significant risks.

What Strong Model Governance Looks Like

1. A Clear Model Risk Policy – Every organization using models should have a well-defined policy outlining:
   • What qualifies as a “model”
   • Roles and responsibilities for model development, validation, and approval
   • How frequently models should be reviewed and tested
2. A Model Risk Committee (MRC) – Many firms establish a Model Risk Committee to oversee model use and risk mitigation strategies. This group typically includes:
   • Senior executives from risk, compliance, and operations
   • Model developers and validators
   • Internal auditors ensuring independent oversight
3. Segregation of Duties – Model development and validation should be performed by separate teams to prevent bias or conflicts of interest. If the same team that builds a model is also responsible for testing it, the risk of blind spots and overlooked flaws increases.
4. Model Inventory & Risk Categorization – Organizations should maintain a centralized inventory of all models, along with their associated risk levels. Higher-risk models (e.g., those affecting regulatory capital calculations or financial forecasting) should undergo more rigorous scrutiny.

What Internal Auditors Should Look For

• Does the organization have a model risk governance framework in place?
• Are roles and responsibilities for model development, validation, and oversight clearly defined?
• Is there independent review and testing of models before they are put into production?
• Does management have adequate visibility into model risk exposure?

Model governance is the foundation of effective model risk management, and internal auditors should ensure their organizations have the right frameworks in place.

4. Model Validation Isn’t One-and-Done – It’s an Ongoing Process

A common misconception is that once a model is validated, it can be relied upon indefinitely. However, models degrade over time due to changes in data, business conditions, and external factors. That’s why regulatory guidance, including SR 11-7 and OCC 2011-12, stresses ongoing validation and performance monitoring.

Key Elements of Ongoing Model Validation

1. Regular Performance Reviews – Organizations should monitor models periodically to ensure they are functioning as intended. This involves:
   • Backtesting – Comparing a model’s past predictions with actual outcomes.
   • Benchmarking – Comparing the model’s results with alternative models or industry standards.
   • Sensitivity Analysis – Assessing how small changes in inputs affect outputs.
2. Challenging Assumptions – Many models are built on historical assumptions that may no longer hold. Internal auditors should ask:
   • Are outdated assumptions affecting model accuracy?
   • Are alternative models being considered?
3. Documentation Updates – Models evolve over time, and so should their documentation. Auditors should check whether documentation is being updated to reflect changes in model logic, data sources, or risk assessments.

Why This Matters for Internal Audit

Internal auditors play a key role in ensuring model validation isn’t just a one-time compliance exercise but a continuous risk management practice. A model that worked perfectly last year may be producing highly inaccurate results today, leading to poor decision-making.

5. AI and Machine Learning Models Bring New Challenges

With the rise of AI and machine learning (ML), model risk has become even more complex. Unlike traditional models with defined equations and assumptions, ML models learn from data and evolve dynamically. This creates significant challenges in explainability, bias detection, and ongoing validation.

Key AI/ML Model Risks

1. Black Box Problem – Many AI models are difficult to interpret, making it hard for auditors to understand how decisions are made.
2. Bias & Fairness – If AI models are trained on biased data, they may produce discriminatory outcomes (e.g., biased loan approvals).
3. Data Drift – AI models rely on real-time data. If data patterns change over time, models can become unreliable without warning.

What Internal Auditors Should Assess

• Are there controls in place to monitor AI model fairness and bias?
• Is there documentation explaining AI model logic and decision-making?
• Are model drift and performance degradation being actively tracked?

AI and machine learning bring new risks that traditional validation methods may not fully address. Internal auditors need to adapt their approach to model risk management in response to these technological advancements.

6. Data Quality Issues Can Destroy Model Accuracy

Even the most sophisticated models are only as good as the data they rely on. Poor data quality can lead to flawed models, inaccurate predictions, and increased model risk. Regulators, including the Federal Reserve and OCC, emphasize the importance of data integrity in model risk management (MRM).

Key Data-Related Risks in Models

1. Incomplete or Inaccurate Data – If a model is trained on missing or incorrect data, its outputs will be unreliable.
2. Data Bias – If historical data reflects biases (e.g., demographic disparities in lending), the model may produce discriminatory results.
3. Overfitting – When models rely too heavily on past data patterns, they may not generalize well to new situations.

What Internal Auditors Should Assess

• Is there a data governance framework ensuring accuracy and completeness?
• Are data sources well-documented and validated before being used in models?
• Is there a process for detecting and correcting data anomalies that could affect model performance?

Organizations often underestimate how much poor data quality contributes to model risk. Internal auditors should ensure that data integrity controls are in place to prevent flawed decision-making.

7. Stress Testing and Scenario Analysis Are Critical Checks

Many models work well under normal conditions but fail during periods of economic or operational stress. This is why stress testing and scenario analysis are essential components of model risk management.

What Is Stress Testing?

Stress testing involves running models under extreme but plausible scenarios to see how they perform. Financial institutions use it to assess how models behave under conditions like:

• Market crashes (e.g., 2008 financial crisis)
• Economic downturns (e.g., COVID-19 recession)
• Operational disruptions (e.g., supply chain breakdowns)

What Is Scenario Analysis?

Scenario analysis goes beyond historical stress tests by examining forward-looking risks. Organizations create hypothetical events (e.g., geopolitical conflicts, regulatory changes) and test their models under these conditions.

What Internal Auditors Should Assess

• Is stress testing part of the model validation process?
• Do scenarios adequately reflect real-world risks?
• Are the results used to improve risk management strategies?

Without rigorous stress testing, organizations may unknowingly rely on models that fail when they’re needed most.

8. Model Risk Management Should Align with the Three Lines of Defense

Many organizations rely on third-party vendors for risk, forecasting, and decision-making models. While outsourcing model development can be efficient, it introduces additional risks.

Key Risks of Vendor Models

1. Lack of Transparency – Many vendors don’t fully disclose their model methodologies, making independent validation difficult.
2. Regulatory Non-Compliance – Vendor models may not align with industry regulations like SR 11-7 or OCC 2011-12.
3. Inadequate Updates & Monitoring – Vendors may not provide frequent updates or transparency into model performance over time.

What Internal Auditors Should Assess

• Does the organization have a due diligence process for vetting third-party models?
• Are vendor models validated internally before use?
• Are there contractual agreements requiring vendors to update and disclose model changes?

Regulators expect firms to own the risk of outsourced models, meaning internal auditors must ensure that vendor-provided models meet the same rigorous standards as internally developed ones.

9. Model Risk Isn’t Just a Regulatory Issue – It’s a Business Risk

Model risk regulations continue to evolve, and firms must stay ahead of new expectations. While SR 11-7 and OCC 2011-12 remain key, recent regulatory developments indicate increased scrutiny of AI models, climate risk modeling, and systemic risk assessments.

Key Regulatory Trends

1. AI & Machine Learning Oversight – Regulators are developing new guidelines for AI-based models to address bias, explainability, and governance challenges.
2. Climate Risk Modeling – As climate change impacts financial stability, regulators expect firms to incorporate climate risk factors into their models (e.g., OCC Bulletin 2021-62).
3. Stricter Vendor Model Requirements – Regulators are increasing pressure on firms to hold third-party models to the same standards as internal models.

What Internal Auditors Should Assess

• Is the firm actively monitoring regulatory updates related to model risk?
• Are new regulations being incorporated into MRM policies?
• Is senior management engaged in understanding and addressing evolving regulatory risks?

Firms that fail to keep up with regulatory changes may find themselves exposed to compliance risks, enforcement actions, and reputational damage.

10. Internal Audit’s Role in Model Risk Is Growing – Are You Ready?

Internal auditors aren’t just checking compliance with model risk policies—they play a crucial role in challenging assumptions, identifying weaknesses, and improving MRM frameworks.

How Internal Audit Adds Value to Model Risk Management

1. Independent Review – Auditors provide an unbiased perspective on model risk governance, validation, and controls.
2. Testing & Validation – Auditors assess whether model performance monitoring is effective and whether assumptions are still valid.
3. Challenging the Business – Internal audit asks tough questions about overreliance on models and gaps in governance.

Key Questions for Internal Auditors

• Does the firm have a strong model risk management framework aligned with regulatory guidance?
• Are models being tested, challenged, and updated regularly?
• Is model risk governance a priority at the board and senior management level?

Internal auditors serve as a critical line of defense in ensuring that models are reliable, transparent, and aligned with risk management best practices.

---