TOD vs TOE: Test of Design vs Operating Effectiveness

A test of design asks whether a control, performed exactly as described by a competent person, would actually prevent or detect the error it exists for. A test of operating effectiveness asks whether it was performed that way, by that kind of person, every time it was supposed to be, across the period you are opining on. They are different questions, they need different evidence, and they fail in different ways: a design failure means the control cannot work even when everyone does their job, and an operating failure means a control that could work did not. Most weak testing on audit files comes from blurring the two, either by treating a walkthrough as if it proved operation, or by sampling forty instances of a control that was never capable of catching the risk.

This guide separates the two tests properly. It sets out the questions each answers and the evidence each needs; where the standards (PCAOB AS 2201, the ISAs, COSO and the Global Internal Audit Standards) place them; how to test design as an argument rather than a checklist; how to test operation with attributes, evidence and consistency; what each test does to sample sizes and hours; when design-only is enough and when it is not; how to document both on one workpaper; the review-note classics; and a worked example from a beverage distributor’s route cash audit in which a control failed design at nine depots and operation at three.

This guide was rewritten in September 2026 from a shorter version published in August 2026. Standard references (AS 2201 paragraphs 42 to 45, ISA 315 and 330, COSO 2013, Global Internal Audit Standards 14.1 to 14.3) were checked against the current texts; sample-size figures follow the sample sizes guide.

In this guide

Two questions, two tests: the comparison
Where the standards put design and operation
Testing design: an argument, not a checklist
The implementation step: proving the control exists before you sample it
Testing operating effectiveness: attributes, evidence, consistency
What each test does to sample sizes and hours
When design-only is enough, and when both are required
Documenting both tests on one workpaper: the template
The review-note classics
Worked example: the depot reconciliation control at MidState
Related guides

Two questions, two tests: the comparison

Design effectiveness is a judgment about the control as specified: its purpose, its mechanics, its precision, its frequency, the competence and authority of the person who performs it, the information it relies on, and what happens when it finds something. Operating effectiveness is a conclusion about the control as performed: whether it ran, when it ran, who ran it, what it caught, and whether it did so consistently across the period. The table sets the two side by side; everything else in this guide expands a row of it.

Dimension	Test of design (TOD)	Test of operating effectiveness (TOE)
The question	If this control operated exactly as described, by a competent person with authority, would it prevent or detect the error or fraud it exists for, at the precision required?	Did it operate as designed, by such a person, consistently throughout the period, and did it deal with what it found?
What is examined	The control’s specification and one or a few instances, to confirm the specification is real	A sample of instances across the period, or the whole population where it is small or automated
Main procedures	Inquiry, observation, inspection of the control description and evidence, walkthrough of one transaction end to end	Inquiry plus inspection and re-performance of sampled instances; observation where the control leaves no trail; re-performance for controls that depend on judgment or calculation
Point in time or period	Usually as at a date, updated if the design changes	Over the period covered by the opinion, with roll-forward if testing stops early
Typical output	A reasoned conclusion that the design is effective, or a design deficiency with the gap identified	An attribute test result: instances tested, deviations found, conclusion on the population, and an evaluation of any deviation
How it fails	The control cannot achieve the objective even when performed perfectly: wrong precision, wrong timing, wrong person, wrong data, no follow-up	The control could achieve the objective but was skipped, performed late, performed by the wrong person, performed on bad data, or performed without acting on exceptions
What a failure means	A design deficiency; operating tests of that control are usually pointless until it is redesigned	An operating deficiency; the design may be fine and the fix is people, timing or discipline
Sample size	One to a handful of instances	Driven by frequency, risk and the tolerable deviation rate; the 25, 40 and 60 conventions and the statistics behind them

Where the standards put design and operation

The distinction is not a house style; it is written into every framework an internal auditor is likely to be measured against, under slightly different names. PCAOB Auditing Standard 2201, the standard for audits of internal control over financial reporting, tests design effectiveness in paragraphs 42 and 43 (whether the controls, if operated as prescribed by persons with the necessary authority and competence, satisfy the control objectives and can effectively prevent or detect misstatements, using inquiry, observation, inspection and walkthroughs) and operating effectiveness in paragraphs 44 and 45 (whether the control is operating as designed and whether the person performing it has the authority and competence to perform it effectively, using inquiry, observation, inspection and re-performance). The financial statement auditing standards use a three-part vocabulary: ISA 315 (Revised 2019) has the auditor evaluate the design of relevant controls and determine whether they have been implemented, and ISA 330 governs tests of operating effectiveness when the auditor intends to rely on them; the AICPA’s AU-C 315 and 330 mirror this. COSO’s 2013 framework asks whether components and principles are present and functioning, which maps to design and implementation, and whether they operate together, which is the operating dimension at the entity level.

The Global Internal Audit Standards do not use the words design and operating effectiveness as terms of art, but the requirements assume the distinction: Standard 13.4 requires evaluation criteria (the design against which the control is judged), 14.1 requires sufficient, reliable, relevant and useful information, and 14.2 and 14.3 require analyses and evaluation of findings in which the condition is compared to criteria and the cause and effect determined. A finding that a control is badly designed and a finding that a well-designed control was not performed have different causes and different effects, and the Domain V guide shows how the engagement standards treat them. The vocabulary table below keeps the terms straight across frameworks.

Framework	Design	Implementation / existence	Operation
PCAOB AS 2201 (ICFR audits)	Design effectiveness, paragraphs 42 to 43	Covered within design testing via walkthroughs (paragraphs 37 to 41)	Operating effectiveness, paragraphs 44 to 45; extent of testing paragraphs 46 to 47
ISA 315 (Revised 2019) and ISA 330; AU-C 315 and 330	Evaluate the design of controls	Determine whether the control has been implemented (design and implementation, D and I)	Tests of controls when relying on operating effectiveness (ISA 330)
COSO 2013	Present (the component or principle exists in the design of the system)	Present, at the level of a specific control being in place	Functioning, and the five components operating together
Global Internal Audit Standards 2024	13.4 Evaluation Criteria; 14.2 comparison of condition with criteria	14.1 Gathering Information (walkthroughs and inspection)	14.1 to 14.3 applied to a sample or population over the period
SOX 404 management assessment	Design assessed and documented for each key control	Walkthroughs performed by management or its testers	Operating effectiveness tested as of year end, with roll-forward from interim

Testing design: an argument, not a checklist

A design test is a piece of reasoning with evidence attached, and the reasoning has a fixed shape: here is what could go wrong, here is the control that is supposed to stop it, and here is why the control, as specified, would stop it at the precision the risk demands. Auditors who treat design as a checkbox (control documented, yes; control owner named, yes) miss the point, which is that a control can be perfectly documented, owned and performed and still not address the risk. A monthly review of aggregate revenue by a controller who looks for variances over five per cent is designed to catch a five per cent problem; if the risk is a two per cent misstatement, the control is well documented, well owned, well performed and ineffective by design.

The seven questions below are the design test. Answer each with evidence from the control description, the walkthrough and the instance you inspected, and the conclusion writes itself. A “no” on any of them is a design gap to report, not a reason to test operation harder.

Design question	What you are checking	Evidence that answers it	Typical design failure
1. Objective: what could go wrong that this control addresses?	That the control is tied to a specific risk or assertion, not to a process step	The risk and control matrix; the walkthrough narrative; your own statement of the what-could-go-wrong	The control addresses a different risk from the one it is mapped to
2. Precision: at what threshold would the control catch the problem?	Thresholds, tolerances, the level of aggregation reviewed, the criteria for investigation	The review threshold in the procedure; the investigation criteria; the actual level of detail in a completed instance	The threshold is above the amount that matters; review is at a level too aggregated to reveal the error
3. Timing and frequency: does it run often enough and early enough?	Whether the control’s frequency matches the risk’s frequency and whether detection happens in time to correct	The schedule; dates on completed instances; the correction window	A quarterly review of a daily risk; a detective control that fires after the reporting date
4. Performer: does the person have the competence and authority?	Whether the performer can recognize the problem and has the standing to act on it	Role and reporting line; training or experience; the evidence of judgment in a completed instance	A junior preparer reviewing their own work; a reviewer without the knowledge to see the error
5. Information: is the input the control relies on complete and accurate?	Whether the reports, data and system outputs the control uses can be relied on	The source of each input; the controls over it, or the auditor’s own test of it (see the IPE testing guide)	The reconciliation is performed on a report that excludes the population it is meant to cover
6. Follow-up: what happens when the control finds something?	Whether exceptions are investigated, resolved and escalated, and whether that is part of the control	The procedure’s exception steps; examples of exceptions and their resolution	Exceptions are noted and never resolved; there is no defined path when the control finds a problem
7. Dependency: does the control rely on other controls that must also work?	Automated components, system access, other controls whose failure would defeat this one	The system dependency chain (see the ITGC vs application controls guide); the related controls in the matrix	A manual review that depends on a system report nobody has validated; an approval workflow that a routing rule bypasses

Write the design conclusion as an argument with the seven answers in it, not as a rating. “Design effective: the monthly depot cash reconciliation, performed by the depot administrator and reviewed by the regional finance manager, compares driver-collected cash per the route-accounting module to bank deposits at the depot level, investigates differences over 200 dollars within five days, and escalates unresolved items to regional finance; the reconciliation uses the module’s collections report, which we tested for completeness in workpaper C-4; the reviewer is independent of collection and deposit at this depot” is a design test. “Design: effective” with a tick is a ritual, and the Audit Society summary explains why the distinction matters more than it looks.

The implementation step: proving the control exists before you sample it

Between design and operation sits a step the financial statement standards name and the SOX world folds into the walkthrough: implementation. A control can be well designed on paper and not exist in practice, because the procedure was written for a process that has since changed, because the person named left, or because the control was designed for an audit and never actually adopted. The implementation test is the walkthrough of one real instance, end to end, with the person who performs the control, inspecting the artifacts they produce. It answers a narrow question: is this control, as designed, actually in place? It does not answer whether it operates consistently; that is what the sample is for, and the walkthrough vs test of controls guide explains why five walked-through transactions are still a walkthrough and not a test.

Two practical rules. Walk through the control with the performer, not the owner, because the owner describes the design and the performer reveals the implementation. And walk through a recent instance you selected, not one they offer, because an offered instance is the one that went well. If the walkthrough shows the control is not in place, stop: you have a design or implementation deficiency, and an operating sample would only measure how consistently the control does not exist.

Testing operating effectiveness: attributes, evidence, consistency

An operating test is an attribute test: for each sampled instance, did the control exhibit the attributes that make it the control? The attributes come from the design test. If the design conclusion says the reconciliation is prepared by the administrator, reviewed by the regional manager within five days, investigates differences over 200 dollars and escalates unresolved items, then the attributes to test are exactly those: prepared by the right role, reviewed by the right role, reviewed within the window, differences investigated at the threshold, unresolved items escalated. Each attribute is a yes or no for each instance, and a “no” is a deviation regardless of whether the outcome happened to be fine.

Evidence strength matters more in operating tests than anywhere else in fieldwork, because the question is what actually happened. The hierarchy, from weakest to strongest, is inquiry (the performer says it was done), observation (you watched it done once, which proves that instance and that day), inspection (you examined the artifact the control left behind, with attention to whether the artifact proves performance or only completion), and re-performance (you did the control yourself on the same inputs and compared results). Inquiry alone never supports an operating conclusion. Inspection of a signature supports the conclusion that a signature exists; inspection of the reviewer’s questions, corrections or notes supports the conclusion that a review occurred. Re-performance is the standard for controls that depend on judgment or calculation, and for management review controls whose evidence of review is thin, which the management review controls guide covers in depth.

Consistency is the third element and the one most often skipped. A control that operated effectively in the eight months you tested and stopped in month nine is not effective for the year. Spread the sample across the period, include the periods of stress (year end, system changes, staff turnover, the month the performer was on leave), and roll forward from an interim test to the period end with inquiry and a small additional sample rather than assuming. If the performer changed during the year, treat the two performers as two populations.

Procedure	What it proves	What it does not prove	Use it for
Inquiry	What the performer believes or asserts	That anything happened	Understanding the control; corroborating other evidence; never alone
Observation	That the control operated on the occasion observed	Operation on any other occasion; the performer knew they were watched	Controls that leave no trail (physical counts, access checks, segregation in practice)
Inspection	That the artifact exists and, depending on the artifact, that the control was performed	Performance, when the artifact only shows completion (a tick, a signature, a status field)	Most sampled instances; strongest when the artifact shows judgment or action
Re-performance	That the control, done independently on the same inputs, produces the same result	That the performer did it, unless combined with inspection	Reconciliations, calculations, judgmental reviews, IT-dependent controls

What each test does to sample sizes and hours

The design test needs one instance, sometimes two or three where the control has variants, because its evidence is the specification and the reasoning, not the frequency. The operating test needs a sample sized by the control’s frequency, the risk it addresses and the deviation rate you can tolerate, and the common conventions exist because those inputs produce stable answers. The frequency table is the one most functions use; the statistical basis for the larger figures is in the sample sizes guide, which explains why 25, 40 and 60 are the numbers they are and when a different one is right.

Control frequency	Population in a year	Common operating sample (lower risk)	Common operating sample (higher risk)	Design sample
Annual	1	1	1	1
Quarterly	4	2	2	1
Monthly	12	2	3 to 5	1
Weekly	52	5	10 to 15	1
Daily	about 250	20 to 25	30 to 40	1
Multiple times a day / per transaction	Thousands	25	40 to 60	1 to 3
Automated (application control)	Every transaction	1, plus evidence that ITGCs over the system were effective	1, plus ITGC evidence; more if ITGCs are weak	1

Hours follow the same logic. In a typical process audit the design tests consume a quarter to a third of the fieldwork and most of the thinking; the operating tests consume the rest of the hours and most of the sampling. Functions that spend ninety per cent of fieldwork on operating samples usually have a design step that is too thin, which shows up later as findings that the control “operated as designed” while the risk went unaddressed. The automated controls testing guide explains why an automated control needs one instance and an ITGC conclusion rather than forty instances, which is the largest single hours saving available in most test plans.

When design-only is enough, and when both are required

Design-only testing is legitimate in a defined set of situations, and it is a shortcut everywhere else. It is enough when the engagement’s objective is about design: a pre-implementation review of a new process, an advisory engagement on a control framework, a readiness assessment before a control has operated for long enough to sample. It is enough when the control has not yet operated, or has operated too few times to sample, in which case the report must say so. It is enough for a control that the engagement does not rely on, where the auditor is documenting the control environment rather than concluding on it. And it is, in practice, all that can be done for a control that fails design: once the design gap is established, testing operation only measures how consistently an ineffective control was performed, and the hours are better spent quantifying the effect.

Both tests are required whenever the engagement gives assurance that a risk is controlled: any assurance engagement with a rating or opinion, any control relied upon in a SOX assessment, any control the function’s plan says it will test, and any control whose failure would change the engagement conclusion. The failure mode to avoid is the report that says “controls are adequately designed and operating effectively” on the strength of walkthroughs, which asserts a conclusion the evidence cannot support; Standard 14.5 requires conclusions supported by the analysis, and a reviewer or external assessor will ask for the operating sample behind that sentence. The table gives the decision.

Situation	Design	Implementation	Operation	Report language
Assurance engagement with a rating or opinion	Required	Required	Required for every relied-upon control	Rating supported by both tests; coverage stated
SOX key control	Required, documented	Walkthrough	Required as of year end, with roll-forward	Effective / deficiency evaluated (see the control deficiency evaluation guide)
Pre-implementation or readiness review	Required	Where the control exists	Not possible; state it	“Design assessed; operating effectiveness not tested because the control has not yet operated”
Advisory engagement on a control framework	Required	Optional	Optional and usually out of scope	Advisory conclusions on design; no assurance on operation
Control that fails the design test	Completed; deficiency reported	Not needed	Not performed; quantify effect instead	“Design deficiency; operating effectiveness not tested”
Control with fewer than the sample size of instances in the period	Required	Required	Test all instances	State the population and that all were tested
Automated control	Required, including the configuration	One instance	One instance plus ITGCs over the system	Effective, conditional on the ITGC conclusion

Documenting both tests on one workpaper: the template

The cleanest files keep design and operation on one workpaper per control, in that order, so that a reviewer can see the argument before the sample and can see that the attributes tested are the attributes the design conclusion identified. The template below is the structure; the workpaper example shows a completed one and the walkthrough documentation template covers the walkthrough section.

Control test workpaper: design and operating effectiveness

Header: Control reference and name | Process and risk addressed (what could go wrong) | Control owner and performer | Frequency | Type (preventive/detective; manual/automated/IT-dependent) | Key or non-key | Period covered | Preparer and reviewer, dates.

Section A, design (as at date): Control description as documented | Walkthrough reference and instance walked (date, transaction, performer) | The seven design questions with evidence for each: objective, precision, timing, performer, information, follow-up, dependencies | Design conclusion (effective / deficiency, with the gap) | If deficient: effect on the risk, and whether operating testing proceeds.

Section B, operating effectiveness (period): Attributes tested (numbered, each traceable to Section A) | Population definition, source and completeness check | Sample method and size, with rationale (frequency, risk, tolerable deviation) | Sample selection reference | Results matrix: instance, date, performer, attribute results, evidence reference | Deviations: description, cause, whether isolated or systematic, effect | Roll-forward procedures if tested at interim | Operating conclusion.

Section C, overall: Combined conclusion (designed and operating effectively / design deficiency / operating deficiency) | Finding reference if any | Reviewer notes and clearance.

The review-note classics

Review note	What went wrong	The fix
“Walkthrough performed, control operating effectively”	A single instance treated as an operating conclusion	Walkthrough supports design and implementation only; add the sample or change the conclusion
“Design effective” with no reasoning	A rating in place of an argument; the precision question was never asked	Answer the seven design questions in writing; state the precision and the what-could-go-wrong
Forty instances tested, all signed, no deviations, risk unaddressed	Operation tested on a control that fails design; the sample measured signatures	Test design first; if it fails, stop and quantify effect
Attributes do not match the design conclusion	The sample tested existence of a document rather than the elements that make the control work	Derive attributes from Section A; test the reviewer’s action, not the reviewer’s initials
Sample drawn from January to August for a December opinion	No coverage of the period of highest risk; no roll-forward	Spread the sample; include year end; roll forward with inquiry and additional instances
Deviation “not a finding because the outcome was correct”	A control that did not operate was excused because nothing went wrong that time	A deviation is a deviation; evaluate cause and rate; the outcome is luck, not control
Inquiry as the only evidence for an operating attribute	The performer’s word substituted for inspection	Inspect or re-perform; inquiry corroborates, it never carries
Automated control tested with a sample of 40	Hours spent proving a system does the same thing forty times	Test one instance and the ITGCs; document the dependency

Worked example: the depot reconciliation control at MidState

MidState Beverage, the three-state drinks distributor used as the running example across this site, collects about 31 million dollars a year in cash and checks through 300 drivers at 12 depots. The control that matters most is the daily depot cash reconciliation: driver-collected cash per the route-accounting module is agreed to the depot’s bank deposit, differences over 200 dollars are investigated within five days, and unresolved items are escalated to regional finance. The FY27 route cash audit (520 budgeted hours, closed at 548) tested that control at all 12 depots, and the file is a clean illustration of why design and operation are separate tests.

Design came first, depot by depot, because the control is performed locally and the design varies with who performs it. The seven questions produced a design deficiency at 9 of the 12 depots: the reconciliation was prepared and reviewed by the same depot administrator who also received the drivers’ cash and prepared the deposit, so the performer had neither independence from the activity nor, in practice, any reason to find a difference. Precision, timing, information and follow-up were adequately designed everywhere; the failure was the performer question. The team stopped operating tests at those nine depots and spent the hours on effect instead, which is how the audit quantified the exposure that became finding F1 (High) and, through the confirmation work, F5. At the three depots where a regional finance manager reviewed the reconciliation independently, design was effective and the operating test proceeded.

Depots	Design conclusion	Operating test	Result	Finding
9 of 12	Design deficiency: preparer and reviewer not independent of cash receipt and deposit	Not performed as an attribute test; hours redirected to quantifying effect (60 reconciliations re-performed across all twelve depots, five per depot, plus 70 monetary-unit and 20 targeted customer confirmations)	14 of the 60 re-performed reconciliations failed, 12 of them at the nine deficient depots; confirmation exceptions traced to statements never sent	F1 High (reconciliation not independent); F5 Medium (1,130 of 4,200 customers receive no statement)
3 of 12	Design effective: independent regional review, 200-dollar threshold, five-day window, escalation defined	Attribute test, 5 instances per depot per quarter (15 per depot, 45 in total), attributes: prepared by administrator, reviewed by regional manager within five days, differences over 200 dollars investigated, unresolved items escalated	2 deviations in 45: one review 11 days late in a holiday week, one difference of 340 dollars investigated but not documented	Observation; no finding (isolated, cause identified, no effect)

Two lessons the file records. Testing operation at the nine deficient depots would have produced forty-five neatly signed reconciliations and no findings, because the administrator performed the control diligently; it was the design that made the diligence worthless. And the two deviations at the well-designed depots were not excused because the outcomes were fine: the late review and the undocumented investigation were recorded, their causes identified, and the conclusion drawn that they were isolated. The report’s coverage statement, which the audit committee asked for a quarter later, says exactly which depots were tested for what, and it is the reason nobody at the committee thinks the control operates at all twelve.

Comments

Leave a Reply Cancel reply

Dark Towers by David Enrich: Summary and Lessons for Auditors

Blind Spots by Bazerman and Tenbrunsel: Summary and Lessons for Auditors

Black Box Thinking by Matthew Syed: Summary and Lessons for Auditors

The Field Desk Series

Test of Design vs Operating Effectiveness (TOD vs TOE): Two Questions, Two Tests