Auditing operational risk management is an audit of a framework, not of the operations it covers. The processes where operational losses occur (payments, trading, IT, HR, procurement) are audited in their own engagements; this engagement asks whether the organization’s system for identifying, assessing, monitoring, controlling and reporting operational risk is designed well and works. In a bank that system is a formal operational risk management framework owned by a second line function and judged by supervisors against the Basel Committee’s principles; in a manufacturer or a hospital it may be a risk register, a self-assessment cycle and a quarterly report, judged against COSO or ISO 31000. Either way the audit has the same shape: criteria, a set of framework components, a test of each component’s design and operation, and a conclusion on whether the board can rely on what the framework tells it.
This guide gives the criteria, the scoping method, a sixteen-test work program across nine components, the loss-data completeness test that produces the most findings, the evidence of second line challenge that examiners ask for, a maturity scale for rating the framework, the findings that recur, and a worked example from a $9 billion regional bank. It is distinct from the site’s operational risk guide, which explains what operational risk is, and from the RCSA guide, which covers running a self-assessment; both are inputs here. Readers in banking should also have the OCC risk categories primer to hand, because supervisors use that vocabulary.
In this guide
- The framework, not the events: what this audit covers
- Criteria: Basel’s twelve principles, COSO ERM, ISO 31000 and supervisory standards
- The nine components of a framework
- Scoping the engagement: hours, depth and what to leave out
- The sixteen-test work program
- The loss data completeness test
- Testing the RCSA and the key risk indicators
- Testing scenario analysis
- Evidence of second line challenge
- Testing what the board is told
- The coordination protocol: complement, not duplicate
- Rating the framework: a maturity scale
- The findings that recur
- Template: the framework assessment record
- Worked example: Lakeshore Bancorp’s operational risk framework audit
- Questions about auditing operational risk management
- Related guides
The framework, not the events: what this audit covers
Operational risk is the risk of loss from inadequate or failed internal processes, people and systems, or from external events; the Basel definition includes legal risk and excludes strategic and reputational risk. Every process audit on this site tests operational risk in one process. This engagement steps back and tests the machinery that is supposed to see all of them at once: the taxonomy that names the risks, the self-assessments that rate them, the loss database that records what went wrong, the indicators that warn before it does, the scenarios that imagine the losses that have not happened yet, the issue log that tracks the fixes, the reports that reach the board, and the second line function that is supposed to challenge the first line’s view of all of it.
| Question | Process audit (for example, wire transfers) | Operational risk management audit (this guide) |
|---|---|---|
| Objective | Are the controls over wire payments designed and operating? | Does the framework identify, assess, monitor and report operational risk across the organization reliably? |
| Auditee | Payment operations (first line) | The operational risk function (second line) and the business units’ use of the framework |
| Criteria | Policy, regulation, control design principles | Basel principles, COSO ERM, ISO 31000, supervisory standards, the organization’s own framework policy |
| Evidence | Transactions, approvals, reconciliations | Taxonomy, RCSA records, loss database, KRI reports, scenario workshops, committee papers, challenge records |
| Typical finding | Self-approved wires | Loss events unreported; KRIs without thresholds; appetite breaches not escalated |
| Who cares most | The COO and the payments head | The board risk committee, the CRO and the supervisor |
Criteria: Basel’s twelve principles, COSO ERM, ISO 31000 and supervisory standards
Standard 13.4 requires evaluation criteria for every engagement, and for this one they are unusually well supplied. The Basel Committee’s revised Principles for the Sound Management of Operational Risk, published in March 2021, set twelve principles that supervisors use for banks and that non-banks can borrow; the principles describe the framework as implemented by the first line, reviewed and challenged by the second and independently reviewed by the third. COSO’s Enterprise Risk Management framework (2017) and ISO 31000 (2018) supply the criteria for organizations that do not use the Basel vocabulary. In the United States, the OCC’s heightened standards for large banks (12 CFR Part 30, Appendix D) require an independent risk management function with specified duties. The risk frameworks comparison covers the differences; for this engagement the twelve Basel principles are the most usable checklist.
| Basel principle (2021) | What it requires | Framework component it tests | Program tests |
|---|---|---|---|
| 1. Risk management culture | The board takes the lead in establishing a strong risk culture, with codes of conduct and training | Governance and culture | T1, T2 |
| 2. Integrated framework | An operational risk management framework fully integrated with overall risk management | Policy and taxonomy | T3 |
| 3. Board oversight | The board approves and periodically reviews the framework and ensures senior management implements it | Governance | T1 |
| 4. Appetite and tolerance | The board approves a statement of the nature, types and levels of operational risk the bank will assume | Appetite | T4 |
| 5. Senior management implementation | Governance structures and policies, procedures and processes that translate the framework into practice | Policy, roles | T2, T3 |
| 6. Identification and assessment | Comprehensive identification using tools such as event management, scenario analysis, self-assessments, benchmarking | RCSA, loss data, scenarios | T5 to T10 |
| 7. Change management | A process to identify, manage, challenge, approve and monitor change | New product and change approval | T11 |
| 8. Monitoring and reporting | Regular monitoring of the operational risk profile and material exposures, with reporting to the board | KRIs, reporting | T9, T14, T15 |
| 9. Control and mitigation | A strong control environment, segregation of duties, risk transfer such as insurance | Issue management, control inventory | T12, T13 |
| 10. ICT risk | A robust programme for information and communication technology risk aligned with the framework | Technology risk integration | Scoped to the cyber program audit; T3 tests integration |
| 11. Business continuity | Forward-looking business continuity plans linked to the framework | Resilience integration | Scoped to the resilience audit; T3 tests integration |
| 12. Disclosure | Public disclosures that let stakeholders assess the approach | Reporting | T16 |
The nine components of a framework
Whatever the organization calls them, a framework has nine moving parts, and each is a unit of the audit. The table lists what good looks like for each and where the detailed site guide is.
| Component | What good looks like | Owner | Guide |
|---|---|---|---|
| 1. Governance and policy | Board-approved framework policy; a risk committee with a charter; defined first, second and third line roles; the operational risk function independent of the business | CRO or head of operational risk | IA’s role in ERM |
| 2. Taxonomy and appetite | A risk taxonomy used consistently across RCSA, losses, KRIs and scenarios; an appetite statement with measurable tolerances and a breach process | Operational risk function; the board for appetite | Risk appetite statements |
| 3. Risk and control self-assessment | A cycle that covers all material processes, rates inherent and residual risk on a documented scale, links controls to risks, is challenged by the second line and produces actions | Business units, facilitated by the function | RCSA guide |
| 4. Loss event data | A database of internal loss events above a threshold, complete against independent sources, classified to the taxonomy, with root causes, recoveries and near misses; external data where used | Operational risk function; business units report | This guide, loss data section |
| 5. Key risk indicators | Indicators with owners, thresholds, data sources, a reporting cadence and an escalation rule when thresholds are breached | Business units; the function aggregates | KRI tables; non-financial KRIs |
| 6. Scenario analysis | Documented scenarios for severe but plausible events, with governance over assumptions, used in capital, insurance or resilience decisions | The function with business experts | This guide, scenarios section |
| 7. Issue and action management | One log of operational risk issues from all sources (RCSA, losses, audit, exams), with owners, dates, validation and escalation | The function | Issue log; validation |
| 8. Change and new product approval | A process that assesses operational risk before new products, systems, outsourcing and reorganizations go live | The function and product committees | Third-party program; change management |
| 9. Reporting and challenge | Periodic reports to management and the board that aggregate the above, state the profile against appetite and record second line challenge of first line assessments | The function; the risk committee | This guide, reporting and challenge sections |
Scoping the engagement: hours, depth and what to leave out
A full framework audit at a mid-size bank runs 350 to 500 hours; at a non-bank with a lighter framework, 150 to 250. Three scoping rules keep it tractable. Leave out the components that have their own engagements in the plan (cyber, business continuity, third-party management) and test only their integration into the framework. Choose two or three business units as the depth sample for RCSA and KRI testing rather than testing every unit’s self-assessment. And decide at planning whether loss data completeness will be tested, because it is the most expensive test and the most productive; the planning memo should state the reconciliation sources chosen.
| Area | Depth | Hours (bank, 420 total) | Hours (non-bank, 200 total) |
|---|---|---|---|
| Planning and criteria | Framework policy, prior exam findings, committee papers for the year | 40 | 25 |
| Governance, taxonomy, appetite (T1 to T4) | Full | 50 | 30 |
| RCSA (T5, T6) | Three business units in depth; all units for coverage | 60 | 35 |
| Loss data (T7, T8) | Completeness reconciliation to four sources; classification sample | 80 | 35 |
| KRIs (T9) | All indicators for design; breaches for the year | 35 | 15 |
| Scenario analysis (T10) | All scenarios for governance; two for re-performance | 35 | 10 |
| Change, issues, controls (T11 to T13) | Samples | 45 | 20 |
| Reporting, challenge, disclosure (T14 to T16) | Full year of reports | 40 | 15 |
| Reporting phase | — | 35 | 15 |
The sixteen-test work program
Each test follows the five-element form used in the work program template: objective, procedure, population, evidence and what would be a deviation. The table gives the program at the level of the test; the sections that follow expand the three tests that carry the most weight.
| # | Test | Procedure | Evidence | Deviation |
|---|---|---|---|---|
| T1 | Board oversight | Inspect board and risk committee minutes for the year: framework approval or review, appetite approval, profile reporting, breach discussion | Minutes, papers, attendance | Framework not reviewed within the policy’s cycle; profile reports not presented; breaches not discussed |
| T2 | Roles and independence of the function | Inspect the organization chart, the CRO’s reporting line, the function’s charter and resourcing; interview the head | Org chart, charter, budget, headcount | Function reports into the business it challenges; vacancies above 20 percent for a year |
| T3 | Policy, taxonomy and integration | Inspect the framework policy; trace the taxonomy into the RCSA tool, the loss database, the KRI library and the scenario register; confirm ICT, continuity and third-party risk are integrated | Policy, taxonomy, system configurations | Different taxonomies in different tools; components that cannot be aggregated |
| T4 | Appetite and tolerances | Inspect the appetite statement; confirm tolerances are measurable; trace a year of measurements; inspect breach escalations | Appetite statement, metrics, escalation records | Qualitative-only appetite; breaches not escalated per policy (appetite guide) |
| T5 | RCSA coverage and cycle | Reconcile the process inventory to RCSAs completed in the cycle; inspect the schedule | Process inventory, RCSA register | Material processes with no RCSA within the cycle |
| T6 | RCSA quality | For three units, re-perform the assessment: challenge ratings against loss history and audit findings; test control-to-risk linkage; inspect second line challenge records and actions raised | RCSA workpapers, challenge logs | Residual ratings inconsistent with losses; no challenge evidence; no actions from red ratings |
| T7 | Loss data completeness | Reconcile the loss database to independent sources (see the next section) | GL write-off accounts, legal provisions, insurance claims, complaints, incident tickets, fraud cases | Reportable events missing from the database |
| T8 | Loss data quality | Sample 40 events: classification to the taxonomy, date of discovery versus occurrence, gross and net amounts, recoveries, root cause, near-miss flagging, timeliness of reporting | Event records and source documents | Misclassification above 10 percent; events reported more than 30 days after discovery |
| T9 | Key risk indicators | Inspect the KRI library for owners, thresholds, sources and cadence; trace a year of breaches to escalation; test three indicators’ data to source | KRI library, reports, source data | Indicators without thresholds; breaches without escalation; indicators not recalculated from source |
| T10 | Scenario analysis | Inspect the scenario register and governance; re-perform two scenarios’ assumptions; confirm use in capital, insurance or resilience decisions | Scenario workshops, assumptions, decision records | Scenarios stale beyond the policy cycle; assumptions unchallenged; outputs unused |
| T11 | Change and new product approval | Sample 15 new products, systems or outsourcings from the year; confirm the operational risk assessment preceded go-live | Approval records | Go-live without assessment; assessment after the fact |
| T12 | Issue management | Reconcile issues from RCSA, losses, audit and exams to one log; sample 30 for ownership, dates, validation, escalation | Issue log | Issues outside the log; closures without validation; past-due without escalation |
| T13 | Control inventory and risk transfer | Confirm key controls for top risks are inventoried and tested by someone; inspect insurance coverage against the scenario register | Control inventory, test results, policies | Top risks with no tested control; insurance gaps against severe scenarios |
| T14 | Management reporting | Inspect a year of operational risk reports: aggregation across components, profile against appetite, trend, actions | Reports | Reports list events without profile or appetite view |
| T15 | Second line challenge | Inspect records of the function’s challenge to first line assessments, ratings and KRI explanations (see the challenge section) | Challenge logs, emails, committee papers | No evidence the function ever changed a first line view |
| T16 | Regulatory and public disclosure | Reconcile disclosed operational risk information (Pillar 3 or annual report) to the framework’s own data | Disclosures, source data | Disclosures inconsistent with internal reporting |
The loss data completeness test
The loss database is the framework’s memory, and it is almost always incomplete, because reporting a loss is unpleasant and the business units that suffer them have little incentive to do it. Completeness cannot be tested from the database itself; it is tested by starting from places where losses leave independent traces and checking whether each trace has a database entry. Banks subject to the Basel standardized approach for operational risk capital have an extra reason to care: the internal loss multiplier uses ten years of loss data, and supervisors test its quality.
| Independent source | What to extract | Reconciliation | Typical result |
|---|---|---|---|
| General ledger write-off and loss accounts | All postings above the reporting threshold to operational loss, write-off, fraud loss, legal settlement and customer remediation accounts | Match each posting to a database event by amount, date and unit | The largest source of unreported events; 20 to 40 percent unmatched in a first-time test |
| Legal provisions and settlements | Litigation register; provisions booked; settlements paid | Match to events classified as legal | Legal often keeps its own register and reports nothing to risk |
| Insurance claims | Claims filed and recoveries received | Match to events; check recoveries recorded net | Recoveries recorded, underlying gross loss not |
| Customer complaints and remediation | Complaints with compensation paid | Match compensation above threshold | Remediation programs recorded as one event or not at all |
| IT incident tickets (severity 1 and 2) | Major incidents with business impact | Match to business disruption events, including near misses | Near misses never recorded |
| Fraud and investigation case files | Closed cases with confirmed loss | Match to internal and external fraud events | Cases recorded at investigation close, months late |
| Regulatory fines and exam findings | Penalties paid; MRAs | Match to compliance events | Usually complete; fines are visible |
The test produces a completeness rate (events found in independent sources that exist in the database, divided by all events found) and a list of unreported events with amounts. Both go in the finding; the rate is the condition, the list is the evidence, and the cause is almost always a reporting process that depends on the loss-suffering unit volunteering. The journal entry analytics patterns apply to the ledger extraction, and the fraud protocol applies if the test surfaces a loss nobody investigated.
Testing the RCSA and the key risk indicators
The RCSA test (T6) is a re-performance. For each of the three depth units, the auditor takes the unit’s residual risk ratings and asks three questions: are they consistent with the unit’s own loss history and with audit findings in the last three years; does each key control listed actually map to the risk it is listed against and has it been tested by anyone; and did the second line change anything when it reviewed the assessment. A unit that rates every risk green, has had two High audit findings and three loss events in the period, and shows no second line adjustment has an RCSA that is theater, which is the failure mode the RCSA guide is built to prevent.
The KRI test (T9) is a design and data test. Design: every indicator has an owner, a data source, a calculation, thresholds (amber and red), a cadence and an escalation rule, and the indicator leads the risk rather than lagging it (open high-severity incidents leads; losses booked lags). Data: three indicators are recalculated from source, which finds the indicators that have been reported from a spreadsheet nobody refreshes. Escalation: every red breach in the year is traced to an escalation record and an action; the common finding is a dashboard full of red that generated nothing. The KRI tables give example indicators and thresholds by risk type.
Testing scenario analysis
Scenario analysis imagines severe but plausible operational losses (a multi-day core system outage, a large internal fraud, a data breach, a mis-selling event) and estimates their frequency and severity for use in capital, insurance and resilience decisions. The audit tests governance rather than the estimates: is there a register of scenarios aligned to the taxonomy and the top risks; are workshops documented with participants, assumptions and sources; has the second line challenged the assumptions; have the scenarios been refreshed within the policy cycle (usually annually); and are the outputs used, meaning a decision somewhere cites them. Re-performing two scenarios’ assumptions against external loss data or the organization’s own history catches the estimates that were anchored on last year’s number. The third-party resilience and cyber resilience guides contain scenario libraries that can serve as benchmarks.
Evidence of second line challenge
Supervisors examine challenge directly, and so should the audit. The Three Lines Model describes second line roles as providing “monitoring and challenge” and the Basel principles describe the framework as “reviewed and challenged by the second line of defence”. The test is simple: find the records in which the operational risk function disagreed with a first line assessment and changed the outcome. Acceptable evidence includes RCSA ratings adjusted by the function with the reason, KRI explanations rejected and resubmitted, new product approvals declined or conditioned, loss events reclassified, and committee papers in which the function’s view differs from the business’s. A function with no such record in a year has not challenged anything, whatever its charter says; the internal audit vs compliance guide draws the same distinction for compliance monitoring.
Testing what the board is told
The framework exists to inform decisions, so the last tests look at the reports. A good quarterly operational risk report aggregates the components: the profile by taxonomy category, the position against appetite with breaches named, the top risks and their trend, losses in the period against history, KRI status with escalations, scenario updates, open issues by age and severity, and the second line’s own assessment of the first line. A report that lists the quarter’s loss events and stops is an incident log, not risk reporting. The audit reconciles a year of reports to the underlying data (do the loss totals match the database, do the KRI statuses match the library) and reads the minutes to see whether anything in the report produced a decision. The presenting to the audit committee guide has the reporting principles; they apply to the CRO’s report as much as the CAE’s.
The coordination protocol: complement, not duplicate
Once the framework has been audited, the question for the plan is how the second line’s testing and internal audit’s testing fit together, so that the business is not tested three times for the same thing and so that the board understands whose assurance it is receiving. Standard 9.5 requires the chief audit executive to coordinate with other assurance providers and to assess the basis for any reliance; the Three Lines Model describes the mechanism as alignment through communication, cooperation and collaboration. A protocol that works has five clauses, agreed between the CAE and the head of operational risk and shown to the audit and risk committees.
| Clause | What it says | Why |
|---|---|---|
| One assurance map | Risks, first line owners, second line testing, internal audit coverage and external assurance on one page, refreshed annually with the plan | Gaps and duplication become visible (GRC framework guide) |
| Reliance by component, with a basis | Internal audit states for each framework component whether it relies on second line work fully, partially or not at all, citing this audit’s results | Standard 9.5; the board sees derived versus direct assurance |
| Re-performance, not repetition | Where internal audit relies on second line testing, it re-performs a sample rather than re-testing the population | Reliance is verified at a fraction of the cost |
| Shared issue log and shared calendar | One log of operational risk issues from every source; test schedules coordinated so a unit is not visited by both functions in the same quarter | Removes the duplication the business notices most |
| Distinct reporting | The CRO’s report and the CAE’s report go to the board separately, each labeled as whose assurance it is; the CAE may comment on the second line’s report | The board must not mistake management’s assurance for independent assurance |
The protocol is also where this audit’s result bites. A framework rated at maturity level 2 on RCSA cannot support reliance on RCSA results, and the plan should say so; a level 4 KRI component can carry full reliance for monitoring, with internal audit re-performing three indicators a year. Reliance decisions are revisited after each framework audit and recorded in the plan the board approves under Standard 9.4.
Rating the framework: a maturity scale
Engagement ratings use the function’s standard scale (severity matrix and rating rule), but a framework audit benefits from a maturity view per component, because “Needs Improvement” says less to a board than “loss data is at level 2 of 5”. A five-level scale that works:
| Level | Name | Governance and taxonomy | RCSA | Loss data | KRIs and scenarios | Reporting and challenge |
|---|---|---|---|---|---|---|
| 1 | Initial | No policy; roles undefined | None or ad hoc | No database | None | Incident lists only |
| 2 | Developing | Policy exists; taxonomy inconsistent across tools | Some units; unchallenged | Database with known gaps; completeness untested | Indicators without thresholds; scenarios ad hoc | Reports list events; no appetite view |
| 3 | Defined | Board-approved policy; one taxonomy; appetite with tolerances | All material units on a cycle; challenge recorded | Completeness tested against the ledger; above 80 percent | Thresholds and escalation; annual scenarios | Profile against appetite; breaches escalated |
| 4 | Managed | Integrated with ERM, ICT, continuity, third-party | Ratings reconciled to losses and audit findings | Completeness above 90 percent across four sources; near misses captured | Leading indicators; scenarios used in capital and insurance | Decisions traceable to reports; second line view distinct |
| 5 | Optimizing | Framework reviewed against external benchmarks annually | Continuous assessment with data feeds | External data integrated; root cause analytics | Predictive indicators; scenario outputs drive strategy | Board uses the report to set appetite |
The findings that recur
| Finding | Component | Usual rating | Root cause |
|---|---|---|---|
| Loss events above threshold missing from the database | Loss data | High | Reporting depends on the unit that suffered the loss; no reconciliation to the ledger |
| KRIs without thresholds, or red breaches with no escalation | KRIs | Medium | Indicators chosen for availability, not relevance; no escalation rule |
| RCSA ratings inconsistent with losses and audit findings | RCSA | Medium to High | Self-assessment without challenge; rating scale undocumented |
| Appetite statement qualitative; breaches not escalated | Appetite | Medium | Appetite written for the annual report, not for use |
| Taxonomies differ between the RCSA tool and the loss database | Taxonomy | Medium | Tools bought at different times; no owner for the taxonomy |
| New products launched before assessment | Change | High in regulated firms | Approval process bypassed under commercial pressure |
| Issues from exams and losses tracked outside the log | Issues | Medium | Separate registers in compliance and legal |
| Scenarios stale; outputs unused | Scenarios | Low to Medium | Treated as a capital exercise; no owner for use |
| No evidence of second line challenge | Challenge | High for supervised firms | Function under-resourced or reporting into the business |
Template: the framework assessment record
Operational risk management framework assessment — [organization], [period]. Criteria: [Basel Committee Principles for the Sound Management of Operational Risk (2021); the organization’s Operational Risk Framework Policy v[x]; OCC heightened standards where applicable]. Scope: [components 1 to 9; ICT, continuity and third-party risk tested for integration only, covered by engagements [refs]]. Depth units: [names]. Tests performed: [T1 to T16 per the program, with any exclusions and reasons]. Component results: Governance and policy [level, finding refs]; Taxonomy and appetite [level, refs]; RCSA [level, refs; units re-performed]; Loss data [completeness rate by source: ledger x%, legal x%, insurance x%, complaints x%; events unreported n, $amount; refs]; KRIs [indicators n, with thresholds n, breaches n, escalated n; refs]; Scenario analysis [scenarios n, refreshed within cycle n, re-performed 2; refs]; Issue management [issues reconciled n, outside log n; refs]; Change approval [sampled n, assessed before go-live n]; Reporting and challenge [reports reconciled, challenge records n; refs]. Engagement rating: [Satisfactory / Needs Improvement / Unsatisfactory] under the function’s rating rule. Overall maturity: [level], with the lowest component [name, level]. Prepared [name, date]; reviewed [name, date].
Worked example: Lakeshore Bancorp’s operational risk framework audit
Lakeshore Bancorp, the $9 billion publicly traded regional bank holding company used in examples across this site, has a 22-person internal audit function, a chief risk officer who reports to the board risk committee, an operational risk function of six, and a lead bank supervised by the OCC. Its FY26 plan included a 420-hour audit of the operational risk management framework, the first since the function was created in 2021, prompted by a supervisory observation that the bank’s loss data appeared thin for its size. The criteria were the Basel principles and the bank’s own framework policy; the depth units were commercial lending, deposit operations and the digital channels team.
The loss data completeness test drove the result. Reconciling the database to the ledger’s write-off and remediation accounts, the litigation register, insurance claims and severity-1 incident tickets for two years found 61 reportable events in the sources against 38 in the database, a completeness rate of 62 percent; the 23 unreported events totaled $1.9 million, including a $640,000 customer remediation program that legal had run without telling risk and nine severity-1 outages that nobody had recorded as near misses. The RCSA re-performance found deposit operations rated every risk residual-low in a year in which it had generated eleven loss events and two High audit findings, with no second line adjustment on file. Of 60 key risk indicators, 42 had no red threshold and the 18 that did had breached 27 times with four escalations. The scenario register’s last refresh was 2023. The appetite statement was quantitative for losses (annual operational losses below $4 million) but the breach in Q3 FY25, when a single wire fraud took the total to $4.3 million, had been reported to the risk committee without the escalation memo the policy required. Challenge evidence existed for new product approvals (three declined or conditioned in the year) and for nothing else.
| Component | Result | Maturity level | Finding and rating |
|---|---|---|---|
| Governance and policy | Board-approved policy; function independent; risk committee charter current | 3 | None |
| Taxonomy and appetite | One taxonomy; appetite quantitative; Q3 breach not escalated per policy | 3 | F4 Medium: appetite breach escalation |
| RCSA | Coverage complete; deposit operations ratings unsupported; no challenge adjustments | 2 | F2 High: RCSA ratings inconsistent with losses and findings, unchallenged |
| Loss data | 62 percent complete; 23 events, $1.9 million unreported; near misses absent | 2 | F1 High: loss data completeness |
| KRIs | 42 of 60 without red thresholds; 4 of 27 breaches escalated | 2 | F3 Medium: KRI design and escalation |
| Scenario analysis | Register last refreshed 2023; outputs unused | 2 | F5 Medium: scenarios stale |
| Issue management | Legal and compliance issues outside the log (14 found) | 2 | F6 Medium: issues outside the log |
| Change approval | 15 of 15 assessed before go-live; three conditioned | 4 | None |
| Reporting and challenge | Reports reconcile; profile against appetite present; challenge evidence limited to new products | 3 | F7 Medium: challenge not evidenced outside product approval |
| Overall | — | 2 to 3 | Needs Improvement; 2 High, 5 Medium; 14 actions |
The report was rated Needs Improvement under the bank’s rating rule. Management’s actions included a quarterly loss data reconciliation to the ledger owned by the operational risk function, thresholds for every indicator by the following quarter, and a challenge log. The OCC’s next examination cited the audit’s completeness test and converted its earlier observation into a matter requiring attention on loss data, the lifecycle of which is described in the MRA guide; the bank’s re-test a year later found completeness at 91 percent.
Questions about auditing operational risk management
How is this different from auditing operational risk?
Operational risk is audited in every process engagement, where the controls over a specific process are tested. This engagement audits the framework that is supposed to see operational risk across all processes: the taxonomy, self-assessments, loss data, indicators, scenarios, issue tracking, reporting and second line challenge.
Does a non-bank need an operational risk framework?
It needs the components under some name: a way to identify and rate risks, record losses, watch indicators, track fixes and report to the board. COSO ERM and ISO 31000 provide the criteria; the Basel principles are a useful checklist even outside banking because they are concrete.
What is the single most productive test?
Loss data completeness, reconciled from the general ledger’s write-off and remediation accounts. It is the test most frameworks have never run on themselves, and a first-time result of 60 to 80 percent complete is normal.
Should internal audit rely on the second line’s RCSA results?
Only after this audit has established that the RCSA is reliable: coverage complete, ratings consistent with losses and findings, and challenge evidenced. Standard 9.5 requires an assessment of the basis for reliance, and this engagement is where that assessment is done.
How often should the framework be audited?
Every two to three years in full, with the loss data completeness test annually in organizations whose capital or supervisory standing depends on it. Supervisors of large banks expect the risk function itself to be in the audit universe on a cycle.
What criteria apply to operational risk capital?
The Basel III standardized approach, which jurisdictions have implemented on different timetables, calculates operational risk capital from a business indicator and, where the supervisor applies it, an internal loss multiplier based on the bank’s loss history. Loss data quality therefore has a direct capital consequence, and the audit should reference the supervisor’s current rule; check the jurisdiction’s implementation status at the time of the engagement.
Where do cyber, continuity and third-party risk fit?
They are operational risks with their own frameworks and usually their own engagements (cyber, resilience, third parties). This audit tests only that they are integrated: the same taxonomy, losses and indicators flowing into the same reports, and the same issue log.
Related guides
- Operational risk: a guide for internal auditors — what operational risk is and how it is categorized.
- How to run an RCSA that is not theater — the self-assessment component.
- Key risk indicators: example tables by risk type — the KRI component.
- Non-financial risk indicators — indicators for operational, compliance and reputation risk.
- Risk appetite statements that guide decisions — the appetite component.
- The risk register — the register the framework feeds.
- Risk management frameworks compared — COSO, ISO 31000 and NIST as criteria.
- Internal audit’s role in ERM — the boundary between assurance and running the framework.
- OCC risk categories — the supervisory vocabulary.
- The MRA and MRIA lifecycle — what happens when the supervisor finds the gap first.
- The finding and issue log template — the issue management component.
- Issue validation — closing issues properly.
- Auditing cybersecurity programs — the ICT risk component in depth.
- Auditing business continuity and resilience — the continuity component in depth.
- Third-party risk management end to end — the outsourcing component in depth.
- Finding severity ratings — rating the findings and the engagement.
- How to audit payment operations and wire transfers — a process audit, for contrast.
- Risk taxonomy — building the categories everything else uses.
- Risk assessment matrix — design, calibration and the pitfalls.
- The Organizational Resilience Topical Requirement — the 20 requirements and how to assess them.
- Designing a KRI Program: Selection, Thresholds, Escalation — turning a metric wishlist into an early warning system with thresholds that trigger action
- Operational Resilience Explained: Services and Tolerances — important business services, impact tolerances, scenario testing and the regimes compared
- Risk Appetite vs Tolerance vs Capacity vs Limits, Explained — the four layers defined and cascaded, with a testable appetite statement
- Operational Risk Framework: Taxonomy, Tools, Governance — taxonomy, loss data, RCSAs, KRIs, scenarios and governance.
Leave a Reply