, ,

How Internal Audit Should Audit the ERM Program Itself: Criteria, Maturity and Independence

An audit of the enterprise risk management (ERM) program is an assurance engagement over the second line’s own machinery: whether the organization has a sound framework for identifying, assessing, responding to and reporting risk, whether that framework operates as designed, and whether the risk information it produces is good enough for the board and management to rely on. It is the engagement the Standards point at directly (Domain I and Standard 9.1 ask internal audit to assess the organization’s risk management processes) and the one most functions handle worst, because the second line is a peer, the chief risk officer may have helped design the audit function, and in smaller organizations the auditors themselves facilitated the risk workshops they are now meant to assess.

This guide gives the method. It covers why the engagement needs three lenses rather than one (framework adequacy, process operation and the quality of the risk outputs), the criteria from COSO’s 2017 framework, ISO 31000 and the Standards, how to use a maturity model as a communication device without letting it replace the auditor’s judgment, the independence choreography for functions that also facilitate risk work, a work program with test steps and evidence, sampling the register and the assessments, how to write the report so that it improves the program rather than embarrassing its owner, the findings that recur, and a worked example of a drinks distributor’s audit function auditing an ERM program it helped set up. It is the enterprise-level companion to the operational risk framework audit and builds on the role of internal audit in ERM and ERM misconceptions guides.

In this guide

Why the ERM program gets audited, and what the Standards ask

The Global Internal Audit Standards make risk management one of the three processes internal audit exists to assess: Domain I’s purpose statement says the function enhances the organization’s governance, risk management and control processes, and Standard 9.1 requires the chief audit executive to understand them and to use that understanding in planning, with a documented assessment where the function is positioned to make one. Boards ask for the assurance because they approve a risk appetite and receive a risk report and have no other independent view of whether either means anything; regulators of banks and insurers expect it explicitly, as the banking guide describes; and external assessors ask whether the function has assessed the second line at all. The engagement answers three questions for the board: is the ERM framework adequate for an organization of this kind; does it operate as designed; and is the risk information reaching the board reliable enough to govern by. Those are the three lenses, and an audit that uses only the first produces the comfortable report that says the policy exists and the committee meets.

Three lenses: framework, process, outputs

LensQuestionWhat is examinedWhat it alone would miss
Framework adequacy (design)If everything in the documents happened, would risk be managed?Policy, taxonomy, appetite statement, roles, committee charters, methodology, tools, reporting design, against criteriaWhether any of it happens; whether the outputs are right
Process operationDoes it happen as designed?Assessments performed on schedule by the right people; register maintained; appetite monitored; KRIs produced; committees meeting and deciding; actions tracked; reporting issuedWhether a process that operates perfectly produces useful risk information
Output qualityIs the risk information reliable?The register’s completeness against reality (incidents, audit findings, strategy); rating consistency and calibration; the appetite comparison; the board report’s accuracy against the underlying data; whether decisions used itNothing; this is the lens that answers the board’s question, and it requires the other two to explain what it finds

Most ERM audits stop at the first lens because it is cheap and uncontroversial. The second lens is where a RCSA that was due in March and completed in November appears. The third is where the audit earns its hours: a register of 63 risks that omits the risk which produced the year’s largest loss, a residual rating of Low on a process the function rated Unsatisfactory two months earlier, a board report that says “within appetite” when the appetite measure has no data behind it. The three lenses are assessed together and reported together, because a finding under the third lens needs the first two to explain why it happened.

Criteria: COSO ERM, ISO 31000 and the organization’s own policy

SourceStructureHow to use it as criteria
The organization’s ERM policy, framework and committee chartersWhatever management committed toPrimary criteria: the audit tests the organization against its own promises first
COSO, Enterprise Risk Management: Integrating with Strategy and Performance (2017)Five components (governance and culture; strategy and objective-setting; performance; review and revision; information, communication and reporting) and twenty principlesThe component and principle structure as the framework-adequacy checklist for organizations that adopted COSO or none; principle-level evidence expectations
ISO 31000:2018, Risk management: guidelinesEight principles; a framework (leadership and commitment; integration; design; implementation; evaluation; improvement); a process (scope, context and criteria; assessment; treatment; monitoring and review; recording and reporting; communication and consultation)The framework and process clauses as criteria for ISO-based programs; the principles as the test of whether the program is integrated rather than bolted on
Global Internal Audit StandardsDomain I; Standard 9.1; Standard 11.5 on risk acceptance; Standard 2.3 on safeguardsWhat the audit function must do and must not do in the engagement
Sector expectationsBank and insurance supervisors’ risk governance guidance; the FSB’s risk appetite framework principles; rating agency ERM criteriaAdditional criteria where they apply; the appetite layers guide covers the FSB structure
The Chartered IIA’s risk maturity scale and the RIMS Risk Maturity ModelFive levels from naive to enabled; seven attributesCommunication devices for the overall conclusion, not criteria in themselves (see below)

Where the organization has adopted no framework, the audit uses COSO’s components or ISO’s framework clauses as the adequacy criteria and says so in the planning memo, because an audit against unstated criteria is an opinion. The frameworks comparison explains the differences between COSO and ISO for the choice; the COSO guide covers the internal control framework, which is a different document from the ERM one and is often confused with it.

Using a maturity model without hiding behind it

Level (the Chartered IIA’s long-used scale)What it looks likeTypical evidence
Risk naiveNo formal approach; risk managed by individuals as they see fitNo policy, register or committee
Risk awareScattered silo approaches; some registers; no common languageDepartmental registers in different formats; no taxonomy
Risk definedStrategy and policies in place and communicated; appetite definedPolicy, taxonomy, appetite statement, methodology, committee; operation uneven
Risk managedEnterprise-wide approach developed and communicated; processes operate; information usedRegister complete and current; assessments on schedule; KRIs; board reporting against appetite; decisions reference risk
Risk enabledRisk management and internal control fully embedded in operations and decisionsRisk considered in strategy and investment decisions as a matter of course; appetite drives limits; lessons feed back; outputs reliable

Maturity models are useful for one thing: telling a board where the program stands in a word it can place on a scale, and where the organization has decided it wants to be. They are dangerous for two reasons. Used as the criteria, they let an auditor conclude “risk defined, as expected” without testing whether a single risk rating is right, which is the first-lens trap in a new form. And used as a target, they invite a program to collect the artifacts each level lists (a policy, a register, a committee) rather than to manage risk, which produces programs that score “managed” and miss the loss. The discipline is to assess the three lenses first, with evidence, and then to express the overall conclusion on the maturity scale as a summary the board can use, with the scale’s levels tied to the findings rather than the other way round. The IIA’s practice guidance on assessing risk management, including its guide on using ISO 31000, follows the same order: criteria and evidence first, a summary rating last.

Independence choreography when audit also facilitates risk work

In many organizations without a risk function, and in most small ones, internal audit facilitates the risk workshops, maintains the register tool or drafts the risk policy, because nobody else can. The IIA’s 2009 position paper on internal audit’s role in ERM drew the lines that still apply: core assurance roles the function should perform (assurance on the processes, on the evaluation of risks, on reporting and on management of key risks); legitimate consulting roles it may perform with safeguards (facilitating identification and evaluation, coaching management on responding, coordinating ERM activities, consolidating reporting, maintaining and developing the framework, championing ERM, developing the strategy for board approval); and roles it should not undertake (setting the appetite, imposing risk processes, providing management assurance on risks, deciding on responses, implementing responses on management’s behalf, and holding accountability for risk management). The Standards carry the same discipline in Standard 2.3 on safeguards for impairments and in the requirement that assurance over an activity the function performed be done by someone else.

SituationSafeguardWhat the audit can still conclude
The function facilitated the workshops that produced the registerA different auditor leads the ERM audit; the facilitation is disclosed in the report; the assessment of the register’s quality uses external evidence (incidents, findings, strategy) rather than re-performing the workshopsEverything except an opinion on the facilitation itself, which is management’s to own
The function drafted the policy or methodologyDisclosed; the framework-adequacy lens is assessed against external criteria (COSO, ISO) rather than against the function’s own drafting choices; the audit committee told that design assurance is limitedOperation and output quality in full; design with the limitation stated
The function maintains the register toolAdministrative role only; management owns content; the audit tests content, not the tool’s administration, or co-sources that partRegister completeness and rating quality
The CAE sits on the risk committeeAs an observer or adviser without a vote, recorded in the charter; the audit tests the committee’s decisions, which the CAE did not makeCommittee effectiveness, with disclosure
The function has done all of the above for yearsCo-source the ERM audit, or have an external assessment cover it, and tell the committee whyIndependent conclusion by the co-source; the function regains the ability to audit the program once management owns it

The independence and objectivity guide covers the general rules; the assurance versus advisory guide explains how the facilitation should have been classified in the first place. The choreography works when it is visible: the audit committee should see, in the plan and in the report, which roles the function played and how the assurance was protected.

Scoping and the engagement plan

A full ERM program audit covers all three lenses across the whole framework and runs 300 to 500 hours in a mid-size organization; a cycle approach audits the framework in full every three years and the output quality (register and report accuracy) every year, because the outputs decay fastest. Scoping decisions to record in the planning memo: the criteria chosen and why; the period (usually the last full risk cycle); the sample of risks and business units for the output-quality tests; whether the sub-frameworks (operational risk, compliance risk, model risk) are in scope or audited separately; and the independence disclosures. The engagement team should include someone with risk methodology knowledge and someone with none, because the second will ask whether the outputs make sense to a reader and the first will not.

The work program

StepLensProcedureEvidenceHours
1PlanningCriteria, scope, sample, independence disclosures; document requestPlanning memo25
2FrameworkMap policy, taxonomy, appetite, roles, committees, methodology and reporting design to the criteria; identify design gapsCriteria mapping workpaper50
3FrameworkGovernance: board and committee charters, minutes and decisions; the chief risk officer’s position and resources; three-line roles as practiced versus documentedCharters; minutes for the period; organization charts30
4ProcessAssessment cycle: were risk assessments and RCSAs performed on schedule, by the right people, with the method; trace ten assessmentsAssessment records; schedules40
5ProcessRegister maintenance: changes in the period, approvals, aging of entries, retired risksRegister history25
6ProcessAppetite monitoring and KRIs: measures produced, thresholds applied, breaches escalated (the KRI tests)KRI pack; breach log35
7ProcessResponse and action tracking: treatments assigned, due, closed; risk acceptances approved at the right level (guide)Action log; acceptance records25
8OutputsRegister completeness: compare the register with the year’s incidents and losses, internal audit’s findings, the strategy’s assumptions and external events; list what is missingIncident log; issue log; strategy; register50
9OutputsRating quality: re-rate a sample of 20 risks independently using the organization’s own scales; test inherent-control-residual logic; compare with audit ratings of the same areasRe-rating workpaper45
10OutputsReport accuracy: trace the last four board risk reports to the register, KRI data and appetite measures; test every “within appetite” statementBoard reports; underlying data30
11OutputsUse in decisions: for five major decisions in the period (investments, new products, acquisitions), find the risk input in the decision papersDecision papers20
12AllConclusion by lens; maturity summary; report; closing meetingReport45
Total420

Sampling the register and the assessments

The register population is small enough to read in full and large enough to sample for re-rating. Read all of it for completeness (step 8), because the question is what is absent, and absence cannot be sampled. For re-rating (step 9), select judgmentally: all risks rated in the top band, the five lowest-rated risks in areas internal audit has recently rated poorly, three risks whose ratings have not changed in three years, three added in the period, and two from each major business unit, for about twenty. For the assessment cycle (step 4), take ten assessments or RCSAs across units and dates, including the two completed latest. The judgmental sampling guide covers documentation; the risk matrix guide’s inter-rater test (three raters, twenty risks, agreement within one point) is the method for step 9 and the result is itself the finding when agreement is poor.

Testing the quality of risk outputs

TestMethodFinding pattern
Completeness against incidentsEvery incident and loss above a threshold in the period mapped to a register risk; unmapped incidents listedThe year’s largest loss came from a risk not in the register, or rated Low
Completeness against audit findingsEvery High finding mapped to a register risk and its rating comparedResidual Low on a process internal audit rated Unsatisfactory
Completeness against strategyEach strategic initiative’s key assumptions checked for a corresponding riskThe acquisition strategy carries no integration risk entry
Completeness against the outside worldPeer incidents, regulatory actions and industry events in the period checked against the registerRisks every peer suffered, absent here
Rating calibrationIndependent re-rating of twenty risks; inter-rater agreement; distribution of ratings (a register where 70 percent of risks are Medium has not discriminated)Systematic under-rating of risks owned by senior executives
Control creditFor ten risks, the controls credited in the inherent-to-residual step compared with internal audit’s testing of those controlsCredit taken for controls that failed testing
Appetite comparisonEach “within appetite” statement in the board report traced to a measure with dataQualitative appetite with no measure; statements that cannot be tested
Report accuracyFigures and ratings in the board report agreed to the register and KRI dataRegister changes between the data date and the report not reflected; reds omitted
UseRisk sections in decision papers; committee minutes citing risk analysisDecisions taken with no reference to the risk information the program produced

The report that improves ERM

The chief risk officer or whoever owns ERM is a peer whose cooperation the function needs every year, and the report should be written to be acted on rather than to win. Four practices help. Lead with the output-quality conclusion in plain terms (“the board’s risk report in the period omitted the organization’s largest realized loss and overstated the control position in two areas”), because that is what the committee needs and it cannot be dismissed as methodology quibbling. Attribute design and operation findings to root causes rather than to the owner’s competence; most ERM weaknesses are resourcing and authority problems (a risk function of one, a committee that cannot compel business units to assess), and saying so helps the owner get what the program needs. Give the maturity summary with its path: where the program is, where the organization said it wanted to be, and the three changes that would move it. And recognize what works, specifically, because an ERM owner who is told only what is wrong will treat the next audit as a threat. The recommendations and negotiating findings guides apply; the presenting to the committee guide covers the delivery, where the CRO should be in the room.

Findings that recur

FindingLensRoot cause usuallyRating usually
Register omits risks that materialized or that audit rated HighOutputsAssessment cycle driven by the register rather than by events; no incident-to-register reconciliationHigh
Residual ratings contradict control testingOutputsControl credit taken on the owner’s word; no link between ERM and audit resultsHigh
Appetite not measurable; “within appetite” unsupportedFramework and outputsQualitative-only appetite statement (guide)High
Assessments late or skipped in parts of the organizationProcessRisk function lacks authority; business units treat ERM as optionalMedium
Risk acceptances made below the authorized levelProcessNo acceptance procedure; appetite unknown to the acceptorsMedium to High
Board report unchanged for yearsOutputsReport produced from the register template, not from analysisMedium
Taxonomy and tools inconsistent across unitsFrameworkDecentralized growth; no owner of the taxonomyMedium
Risk function under-resourced for the mandateFrameworkProgram designed for a larger team than fundedReported as the root cause of several findings
Internal audit’s own independence unclearGovernanceFacilitation roles undisclosedDisclosed as a limitation, with safeguards

Worked example: MidState Beverage audits the program it helped build

MidState Beverage is a three-state drinks distributor with twelve depots, about 1,400 employees, $221 million in revenue and a six-person internal audit function, used in examples across this site; it has no compliance function, and its ERM program is owned by the finance director, who holds the role alongside his own. The program was built four years earlier with internal audit’s help: the audit manager facilitated the first risk workshops, the function’s analytics auditor built the register spreadsheet, and the chief audit executive drafted the risk policy the board approved. By FY27 the register carried 63 risks, the finance director ran an annual refresh, and the board received a top-ten risk slide each quarter. The audit committee asked for the first audit of the program, and the CAE had to design an engagement that could reach a conclusion despite the function’s history with it.

The choreography: the engagement was led by one of the two seniors, who had joined after the program was built and had never facilitated a workshop; the audit manager and the analytics auditor were excluded from the team; the framework-adequacy lens was assessed against ISO 31000’s framework clauses rather than against the policy the CAE had drafted, and the report disclosed the function’s historical roles and the limitation that design assurance was independent only to that extent. The output-quality lens, where the function had played no role, was assessed in full. The planning memo recorded all of this and the committee chair approved it before fieldwork. The engagement took 310 hours over nine weeks.

LensWhat the audit foundRating
FrameworkPolicy and register adequate for the size; no taxonomy (the 63 risks mixed causes, events and effects, as the taxonomy guide describes); appetite statement qualitative only; no KRIs; the finance director’s dual role left the program with about 40 hours a year of attentionMedium (design), with resourcing as root cause
ProcessAnnual refresh performed on schedule; depot managers’ assessments collected by email with no challenge; 11 of 63 risks unchanged in rating for four years; treatment actions not tracked after the first yearMedium
Outputs: completenessThe FY26 Dayton driver theft ($18,400) mapped to a register risk rated Low; the FY27 route cash findings (five findings, Unsatisfactory) concerned a risk rated Medium residual on the strength of controls the audit had found not operating; the cyber program’s eleven findings mapped to a risk rated Medium; no risk for the two unintegrated acquisitions’ systems; shrink write-off of $410,000 mapped to a risk whose rating had not movedHigh
Outputs: calibrationRe-rating of 20 risks by two auditors and the finance director: agreement within one point on 9 of 20; 44 of 63 risks rated Medium residualHigh (with the completeness finding)
Outputs: reportingTop-ten slide unchanged in composition for seven consecutive quarters; no appetite comparison possible; none of the four major decisions in the period (the audit management system purchase, the managed security service, two depot lease renewals) had a risk section drawn from the programMedium
Maturity summaryRisk defined, with elements of risk aware in the depots; the board had, two years earlier, stated an ambition of risk managed

The report led with the completeness finding, stated plainly that the board’s top-ten slide had not reflected the year’s realized losses or the audit function’s own results, and attributed the operating weaknesses to the program’s resourcing rather than to the finance director, who had been given a second job and forty hours to do it. The recommendations were three: a taxonomy and a quantitative appetite for the top risks, built with the taxonomy and appetite methods and owned by management without internal audit’s facilitation; a reconciliation of incidents and audit findings to the register each quarter, which costs little and would have caught every completeness gap; and a decision by the board on whether ERM would be resourced for the ambition it had stated. The committee funded a part-time risk coordinator reporting to the finance director, and the following year’s output-quality re-test, run by the same senior, found the register reconciled to incidents and findings and the top-ten slide changed in three of four quarters. The CAE’s note to the committee recorded that the function had stopped facilitating risk workshops, so that the next full audit could assess design without a limitation.

Questions about auditing ERM

Should internal audit audit the risk management function?

Yes. The Standards make assessing risk management processes part of the function’s purpose, boards have no other independent view of whether the risk information they receive is reliable, and regulators of financial institutions expect it. The engagement needs safeguards where the function has also facilitated risk work.

What criteria should an ERM audit use?

The organization’s own policy and framework first, then COSO’s 2017 ERM components and principles or ISO 31000’s framework and process clauses, chosen to match what the organization adopted and stated in the planning memo. Maturity models summarize the conclusion; they are not the criteria.

How do you test whether a risk register is complete?

Map the period’s incidents and losses, internal audit’s findings, the strategy’s assumptions and relevant external events to register entries, and list what has no home or sits under a rating the evidence contradicts. Absence cannot be sampled, so the register is read in full.

Can internal audit facilitate risk workshops and still audit ERM?

With safeguards: a different auditor leads the audit, the roles are disclosed, design is assessed against external criteria, and the output-quality tests use evidence the function did not produce. Where the function has done everything for years, co-source the audit and tell the committee why.

What is a risk maturity model and how should it be used?

A scale (commonly naive, aware, defined, managed, enabled) that summarizes how embedded risk management is. Use it to express the overall conclusion to the board after the evidence-based assessment, never as the checklist that replaces testing the quality of the risk outputs.

How often should the ERM program be audited?

The framework in full every two to three years and the output quality (register completeness, rating calibration, report accuracy) every year, because outputs decay fastest; banks and insurers audit annually under supervisory expectations.

What is the most common finding in ERM audits?

A register that omits or under-rates the risks that actually materialized or that internal audit rated High, usually because the assessment cycle runs from the register rather than from events and nobody reconciles incidents and findings to it.

New guides & tools by email

Useful so far?

There are 400+ more guides where this came from. Get new guides, templates and free audit tools by email when they ship. No schedule, no filler.

Free. One confirmation email from WordPress.com, then you’re in. Unsubscribe anytime.

New guides & tools by email

Don’t lose this library.

400+ practitioner-written guides and free tools. Hear when new ones land.

One confirmation email from WordPress.com, then you’re in. Unsubscribe anytime.

Comments

Leave a Reply

Discover more from internalauditguide.com

Subscribe now to keep reading and get access to the full archive.

Continue reading