Attribute sampling is the statistical method for estimating how often a control fails to operate, by testing a random sample of the times it should have operated and projecting the deviation rate to the whole population with a stated confidence. It is the method behind most tests of controls: did the approval happen, was the reconciliation reviewed, was the callback made. Its outputs are two: a conclusion about whether the control’s deviation rate is below the rate the auditor can tolerate, and a measured allowance for the chance that the sample misled. Most auditors use it by looking up a sample size in a table, testing, and declaring the control effective if the sample passes; the method is more useful, and more honest, when each step is understood, because the steps where judgment enters (what counts as a deviation, how much reliance the test supports, what rate was expected) decide the answer before any item is tested.
This guide works the method end to end. It covers defining the attribute and the deviation, defining and reconciling the population, choosing the confidence level and the tolerable deviation rate from the reliance the test supports, estimating the expected deviation rate, computing the sample size (with tables reproduced from the binomial calculation rather than copied from a textbook), selecting the sample randomly with a documented seed, evaluating results through the upper deviation limit, writing the conclusion, what to do when deviations appear and why “test more until it passes” is not an option, discovery sampling for rare events and sequential sampling for efficiency, the workpaper, and a worked case from MidState Beverage’s route cash audit in which two deviations in a sample of forty-five changed the engagement’s conclusion. It is the statistical companion to the sample size guide (which explains the 25, 40 and 60 conventions), the monetary unit sampling guide (for overstatement of balances) and the judgmental sampling guide (for when statistics do not apply).
In this guide
- When attribute sampling is the right method
- Step 1: define the attribute and the deviation
- Step 2: define and reconcile the population
- Step 3: choose confidence and the tolerable deviation rate
- Step 4: estimate the expected deviation rate
- Step 5: compute the sample size
- Step 6: select the sample
- Step 7: perform the test and record deviations
- Step 8: evaluate through the upper deviation limit
- Step 9: conclude, and what the conclusion changes
- When deviations appear
- Discovery sampling and sequential sampling
- The workpaper
- Worked case: MidState Beverage’s route cash reconciliations
- Questions about attribute sampling
- Related guides
When attribute sampling is the right method
| Situation | Method | Why |
|---|---|---|
| Testing whether a control operated (yes or no) across many occurrences | Attribute sampling | The question is a rate of deviation in a population of binary outcomes |
| Testing whether a balance is overstated in dollars | Monetary unit sampling | The question is an amount, and large items matter more |
| Looking for at least one instance of a rare, serious event (fraud, prohibited transaction) | Discovery sampling (below) | The question is whether the rate exceeds a small critical value |
| The population can be tested in full by query | Full-population analytics (the SQL guide) | No sampling risk; sampling is for what cannot be tested in full |
| The control operates a few times a year | Test all occurrences or a judgmental sample | Statistical projection is meaningless for populations under about 50 |
| Items differ in risk and the auditor wants the risky ones | Judgmental sampling | No projection is possible; the conclusion is about the items tested |
| The auditor wants to say “the deviation rate is X percent, plus or minus” | Attribute sampling with a two-sided interval | Rarely needed in controls testing, which asks whether the rate is below a limit |
Step 1: define the attribute and the deviation
The attribute is the characteristic the control should produce in each item (“the daily depot cash reconciliation was prepared by the depot cashier and reviewed and signed by the depot manager on the same business day”). The deviation is the absence of that characteristic, and it must be defined before testing with every edge case decided: is a review signed the next morning a deviation (the design says same day; yes); is a review signed by the assistant manager a deviation (if the design names the manager or a designated deputy, and the assistant is designated, no); is a reconciliation with an unexplained difference that was signed anyway a deviation (yes, and a different kind: the control was performed but did not do its job, which the exceptions guide treats separately). The attribute comes from the control description in the risk and control matrix; where the description is vague, the test cannot be precise, and the design principles guide explains why that is a finding about the control rather than about the test. One attribute per sample is the rule; testing three attributes on one sample is three samples that happen to share items, each evaluated on its own.
Step 2: define and reconcile the population
The population is every occasion on which the control should have operated in the period: every business day at every depot for a daily reconciliation, every vendor bank change for a verification control, every new hire for an onboarding approval. It is defined by period and scope, listed from a source that is complete (the system’s record of occasions, not the control performer’s log of the ones they performed), and reconciled: the number of items in the listing agrees to an independent count, with the IPE tests applied to the listing. A sample drawn from an incomplete population tells the auditor about the wrong population, and the most common version of the error is sampling from the file of completed reconciliations, which by construction contains no missing ones. Where the population is heterogeneous (twelve depots with different staffing), the auditor decides whether to sample it as one population (one conclusion about the control across the organization) or to stratify (conclusions by stratum), and the choice depends on what the conclusion needs to say: a control that fails at three depots and works at nine is one finding if the population is pooled and two if it is stratified.
Step 3: choose confidence and the tolerable deviation rate
| Parameter | Meaning | Common choices | Set by |
|---|---|---|---|
| Confidence level | The probability that the sample’s conclusion is right; one minus the risk of concluding the control is effective when it is not (the risk of overreliance) | 95 percent where the test is the main basis for relying on the control (SOX key controls, high-risk processes); 90 percent where other evidence contributes or reliance is moderate | The reliance the test supports; the engagement’s risk assessment |
| Tolerable deviation rate (TDR) | The maximum population deviation rate the auditor can accept and still conclude the control is effective | 5 percent for key controls the auditor relies on heavily; 10 percent for controls with moderate reliance or compensating controls; higher rates for low-reliance tests | The control’s importance and what other controls exist |
| Expected population deviation rate (EPDR) | The rate the auditor expects to find, from prior tests, walkthroughs and the control environment | 0 percent where the control has a clean history and is automated or rigorously performed; 1 to 2 percent for manual controls with occasional lapses; higher expected rates make statistical testing inefficient | Evidence, honestly assessed |
Confidence and tolerable rate are judgments about reliance, not statistics, and the sample size guide explains how the 25, 40 and 60 conventions derive from them: 25 corresponds roughly to 90 percent confidence with a 10 percent tolerable rate and no expected deviations, 40 to 90 percent and 5 to 6 percent, and 60 to 95 percent and 5 percent. The expected rate is the parameter auditors most often get wrong by optimism: an expected rate of zero when the walkthrough found a lapse produces a sample that will fail and a re-plan under pressure.
Step 4: estimate the expected deviation rate
The expected rate decides how many deviations the plan can absorb and still conclude: a sample sized for zero expected deviations fails the moment one appears, while a sample sized for 2 percent expected (with a 10 percent tolerable rate at 95 percent confidence, 46 items) allows one deviation. Estimate it from the last test’s result, the walkthrough, the control’s design strength and the performer’s workload; where the prior year found two deviations in sixty (3.3 percent), expecting zero this year is not a plan but a hope. The trade-off is explicit: a higher expected rate means a larger sample, and when the expected rate approaches the tolerable rate the sample size climbs steeply (expecting 3 percent against a 5 percent tolerable rate at 95 percent confidence requires 311 items), which is the method’s way of saying that a control expected to fail that often cannot be relied on and should be redesigned rather than tested.
Step 5: compute the sample size
The sample size is the smallest n such that, if the true deviation rate were equal to the tolerable rate, the probability of observing no more than the expected number of deviations is at most one minus the confidence level. The tables below are computed from that binomial rule (for large populations; the finite population correction reduces sizes slightly for populations under about 1,000 and can be ignored otherwise) and agree with the standard published tables to within rounding. The second figure in each cell is the number of deviations the plan can absorb.
| 95 percent confidence; expected rate → | 0 percent | 1 percent | 2 percent | 3 percent |
|---|---|---|---|---|
| Tolerable rate 4 percent | 74 (0) | 117 (1) | 261 (5) | Impractical |
| Tolerable rate 5 percent | 59 (0) | 93 (1) | 124 (2) | 311 (9) |
| Tolerable rate 6 percent | 49 (0) | 49 (0) | 103 (2) | 150 (4) |
| Tolerable rate 8 percent | 36 (0) | 36 (0) | 58 (1) | 77 (2) |
| Tolerable rate 10 percent | 29 (0) | 29 (0) | 46 (1) | 46 (1) |
| Tolerable rate 15 percent | 19 (0) | 19 (0) | 19 (0) | 30 (1) |
| 90 percent confidence; expected rate → | 0 percent | 1 percent | 2 percent | 3 percent |
|---|---|---|---|---|
| Tolerable rate 4 percent | 57 (0) | 96 (1) | 166 (3) | Impractical |
| Tolerable rate 5 percent | 45 (0) | 45 (0) | 105 (2) | 209 (6) |
| Tolerable rate 6 percent | 38 (0) | 38 (0) | 64 (1) | 110 (3) |
| Tolerable rate 8 percent | 28 (0) | 28 (0) | 48 (1) | 48 (1) |
| Tolerable rate 10 percent | 22 (0) | 22 (0) | 22 (0) | 38 (1) |
| Tolerable rate 15 percent | 15 (0) | 15 (0) | 15 (0) | 15 (0) |
Where the expected rate rounds to zero deviations for the computed size (1 percent expected with a 10 percent tolerable rate gives 29 items expecting 0.29 deviations, rounded to zero), the plan absorbs no deviations and the auditor should say so in the memo; planning for one deviation costs the step up to the next column. The sampling memo template records the parameters and the reasoning.
Step 6: select the sample
Statistical conclusions require a selection method that gives every item a known chance of selection. Random selection assigns a random number to every item in the reconciled population and takes the lowest n (or uses a generator against the item numbers), with the seed and the method recorded so a reviewer can reproduce the sample; the random sampling guide shows the spreadsheet mechanics and the SQL guide the hashed-seed method. Systematic selection takes every kth item from a random start (k equals population size divided by sample size) and is acceptable when the population has no cyclical pattern that could align with the interval. Haphazard selection (the auditor picks items without conscious bias) is permitted by some standards for non-statistical sampling but does not support a statistical conclusion and should not be dressed as one. Items selected that turn out not to be in the population (a voided transaction, a day the depot was closed) are replaced by the next random item and the replacement documented; items that cannot be located are deviations, not replacements, because a control whose evidence cannot be found did not demonstrably operate.
Step 7: perform the test and record deviations
Each item is tested against the attribute as defined in step 1 and the result recorded as conforming or a deviation, with the deviation’s nature described: missing entirely, performed late, performed by the wrong person, performed but the exception not followed up. The workpaper example shows the layout and the tick marks guide the notation. Two disciplines matter. The deviation definition does not move during testing; a reviewer who was two days late is a deviation if the attribute said same day, however reasonable the delay, and the auditor who wants to treat lateness as acceptable changes the attribute definition openly in the memo and re-evaluates every item, not quietly for the one that failed. And every deviation is investigated for its cause before evaluation, because the evaluation step treats deviations as a rate while the auditor needs to know whether they are isolated, systematic or indicative of something worse, which the exceptions guide covers.
Step 8: evaluate through the upper deviation limit
The sample deviation rate (deviations divided by sample size) is the best single estimate of the population rate, but the conclusion is drawn from the upper deviation limit: the highest population rate that is consistent with the observed result at the chosen confidence, computed as the rate at which the probability of observing that many or fewer deviations equals one minus the confidence. The upper limit is compared with the tolerable rate: at or below it, the auditor concludes the control’s deviation rate does not exceed the tolerable rate at the stated confidence; above it, the auditor cannot so conclude. The allowance for sampling risk is the gap between the sample rate and the upper limit, and it shrinks as the sample grows. The table gives upper limits for common sizes, computed from the binomial distribution.
| Sample size | 0 deviations, 95 / 90 percent | 1 deviation, 95 / 90 | 2 deviations, 95 / 90 | 3 deviations, 95 / 90 | 4 deviations, 95 percent |
|---|---|---|---|---|---|
| 25 | 11.3 / 8.8 | 17.6 / 14.7 | 23.1 / 19.9 | 28.2 / 24.8 | 33.0 |
| 30 | 9.5 / 7.4 | 14.9 / 12.4 | 19.5 / 16.8 | 23.9 / 20.9 | 28.0 |
| 40 | 7.2 / 5.6 | 11.3 / 9.4 | 14.9 / 12.8 | 18.3 / 15.9 | 21.4 |
| 45 | 6.4 / 5.0 | 10.1 / 8.4 | 13.3 / 11.4 | 16.3 / 14.2 | 19.2 |
| 60 | 4.9 / 3.8 | 7.7 / 6.3 | 10.1 / 8.6 | 12.4 / 10.8 | 14.6 |
| 90 | 3.3 / 2.5 | 5.2 / 4.3 | 6.8 / 5.8 | 8.4 / 7.3 | 9.9 |
| 100 | 3.0 / 2.3 | 4.7 / 3.8 | 6.2 / 5.2 | 7.6 / 6.6 | 8.9 |
Reading the table: a sample of 60 with one deviation has a sample rate of 1.7 percent and an upper limit of 7.7 percent at 95 percent confidence, so the control passes a 10 percent tolerable rate and fails a 5 percent one; the same sample with two deviations (3.3 percent) has an upper limit of 10.1 percent and fails a 10 percent tolerable rate by a tenth of a point, which is the kind of result that tempts auditors to round, and should not. Published tables may differ from these figures by a tenth because of rounding conventions; the method is the same.
Step 9: conclude, and what the conclusion changes
The conclusion is a sentence with the parameters in it: “Based on a random sample of 60 daily reconciliations from a population of 3,024 across twelve depots, with one deviation (a review signed two days late at depot 7), the upper deviation limit at 95 percent confidence is 7.7 percent, which does not exceed the tolerable rate of 10 percent; the control is concluded to be operating effectively for the period.” Or its negative: “the upper deviation limit of 11.4 percent exceeds the tolerable rate of 5 percent; the sample does not support a conclusion that the control operated effectively.” The negative conclusion changes the engagement: the control cannot be relied on, so the risk it addresses is unmitigated for the period, substantive or analytical work on the underlying population expands, the finding is written with the deviation pattern as its condition (the condition guide) and the rate as part of its effect, and the severity follows from what the control was protecting. For SOX, a failed key control test feeds the deficiency evaluation. The qualitative evaluation sits beside the statistical one: a single deviation that turns out to be a deliberate override by a manager is a different matter from one late signature, whatever the upper limit says.
When deviations appear
| Option | When it is legitimate | When it is not |
|---|---|---|
| Conclude the control is not effective | The upper limit exceeds the tolerable rate; the deviations are not isolated to a correctable cause | Never illegitimate; it is the honest answer most of the time |
| Investigate and isolate | Deviations share a cause confined to a sub-population (one depot, one month before a system fix); the auditor splits the population, concludes separately on each, and reports the failing stratum | Used to explain away deviations one by one until none remain |
| Expand the sample | Planned in advance as a sequential design (below); or the expected rate was underestimated and the auditor re-plans with the observed rate, tests the additional items, and evaluates the combined sample with all deviations counted | Testing more items until the cumulative rate falls below the limit, then stopping; this inflates the true risk of overreliance and is the method’s cardinal sin |
| Rely on a compensating control | Another control addresses the same risk and is tested and effective (the compensating controls guide) | The compensating control is asserted, not tested |
| Redefine the attribute | The original definition was wrong (the design names a deputy the auditor missed), changed openly in the memo with every item re-evaluated | Changed to make the failing item pass |
Discovery sampling and sequential sampling
Discovery sampling is attribute sampling with an expected rate of zero and a very low tolerable rate, used when the auditor wants to find at least one instance of a rare and serious condition (a fictitious vendor, a prohibited transaction) if its rate in the population is at or above a critical level. The sample size is the n at which the probability of finding none, if the true rate equals the critical rate, is one minus the confidence: at 95 percent confidence, a critical rate of 2 percent needs 149 items, 1 percent needs 299 and 0.5 percent needs 598. Finding one instance ends the sample and starts an investigation (the first 48 hours guide); finding none supports the conclusion that the rate is below the critical level at the stated confidence, which is weaker than it sounds and should be reported as such. Sequential (stop-or-go) sampling plans the test in stages: an initial sample sized so that zero deviations allow a conclusion, with pre-specified additional stages if one or two deviations appear, each stage’s size computed so that the overall risk of overreliance stays at the planned level. It is efficient when the control is expected to be clean and it is the legitimate form of “test more”, because the decision rules were fixed before the first item was tested; its illegitimate cousin, expanding ad hoc until the sample passes, is not sequential sampling but hope.
The workpaper
| Section | Content |
|---|---|
| Objective and control | The control reference from the RCM; the attribute as defined; the deviation conditions decided in advance |
| Population | Definition (period, scope, occasions); source listing; reconciliation to an independent count; IPE conclusion on the listing |
| Parameters | Confidence, tolerable rate, expected rate, with the reasoning for each tied to reliance and prior results; the computed sample size and the deviations it can absorb |
| Selection | Method (random, systematic); generator and seed, or interval and random start; replacement rules; the list of selected items |
| Testing | Item-level results with tick marks; description of each deviation and its investigated cause |
| Evaluation | Sample deviation rate; upper deviation limit at the chosen confidence; comparison with the tolerable rate; qualitative evaluation of the deviations |
| Conclusion | The sentence with the parameters; the effect on reliance and on other procedures; the finding reference if any |
| Review | Preparer and reviewer sign-offs; the review notes cleared |
Worked case: MidState Beverage’s route cash reconciliations
MidState Beverage, the three-state drinks distributor used in examples across this site, collects about $31 million a year in cash and checks through 300 route drivers from roughly 4,200 customers, reconciled daily at each of twelve depots. Its FY27 route cash audit (engagement FY27-01) tested the daily depot reconciliation control in two stages. The first was a stratified sample of 60 reconciliations, five per depot, which found 14 failures and established a design deficiency at nine of the twelve depots (the reconciliation was not independent of the cashier who prepared it), as the design versus operating guide recounts. The second stage, the attribute sample worked here, tested operating effectiveness at the three depots where the design was sound.
| Step | What the team did |
|---|---|
| 1. Attribute | “The daily cash reconciliation for the depot was prepared by the cashier, agreed to the route accounting module and the deposit listing, and reviewed and signed by the depot manager or the designated deputy on the same business day, with any difference over $50 explained on the form.” Deviation conditions decided in advance: missing form; unsigned or signed after the business day; signed by someone other than the manager or designated deputy; unexplained difference over $50 |
| 2. Population | Business days in FY27 at the three depots: 252 days × 3 = 756 reconciliation occasions, listed from the depot operating calendar (not from the file of completed forms) and reconciled to the route accounting module’s daily close records; two depot closure days removed, population 754 |
| 3. Parameters | Confidence 90 percent (the second stage supported moderate reliance; the design conclusion already limited reliance across the function); tolerable rate 5 percent (the control was key to the cash risk, which the risk assessment rated High); expected rate 0 percent, on the basis that the three depots had passed the design test and the walkthroughs found no lapse |
| 4. Sample size | 45 (90 percent, 5 percent tolerable, 0 percent expected), absorbing zero deviations; the memo recorded that a single deviation would prevent a positive conclusion |
| 5. Selection | Random: items numbered 1 to 754, random numbers generated in the sampling tool with seed recorded in the memo, lowest 45 selected; 17, 13 and 15 items fell to the three depots by chance; no replacements needed |
| 6. Testing | 43 conforming. Two deviations: one form at depot 4 signed by the assistant manager, who was not the designated deputy (the designation named the operations supervisor); one form at depot 9 signed on the following business day with no note |
| 7. Evaluation | Sample deviation rate 4.4 percent; upper deviation limit at 90 percent confidence for 2 deviations in 45: 11.4 percent, exceeding the 5 percent tolerable rate (and the 10 percent rate the team briefly considered) |
| 8. Investigation | Depot 4: the designated deputy had been on leave for three weeks and the manager had informally delegated to the assistant; a designation gap, not a one-off. Depot 9: the manager had been traveling; the cashier held the form. Neither deviation involved an unexplained difference |
| 9. Conclusion | “The sample does not support a conclusion that the reconciliation review control operated effectively at the three depots during FY27 at a 5 percent tolerable deviation rate; the deviations indicate that the control’s deputy arrangements do not cover manager absence.” Reported as part of finding F1 (reconciliation not independent at nine of twelve depots, with deputy coverage inadequate at the remaining three); the engagement’s overall rating of Unsatisfactory stood |
The case shows the method doing its job. The team’s expected rate of zero was an honest estimate that proved wrong, and the plan said in advance what a deviation would mean, so there was no temptation to test fifteen more items in search of a pass; the two deviations were investigated and turned out to share a cause (manager absence without covered delegation) that the design principles guide would call a failure mode nobody had named; and the conclusion changed what the function did next, expanding the monetary unit sample of customer balances and the targeted confirmations because the reconciliation control could not be relied on to have caught errors. The management action, a deputy designation covering every manager absence with a monthly report of reconciliations signed by anyone other than the manager or deputy, was validated the following year with a fresh sample of 45 at the same parameters: zero deviations, upper limit 5.0 percent, control concluded effective by a margin the team described as exactly as thin as the method requires.
Questions about attribute sampling
What sample size do I need for a test of controls?
It depends on three choices: confidence (usually 90 or 95 percent), the tolerable deviation rate (usually 5 or 10 percent) and the expected rate. At 95 percent confidence with a 5 percent tolerable rate and no expected deviations, 59; at 90 percent and 10 percent, 22; expecting one deviation raises the size to the next column of the table.
What is the upper deviation limit?
The highest population deviation rate consistent with the sample result at the chosen confidence; the sample rate plus an allowance for sampling risk. The control passes if the upper limit does not exceed the tolerable rate. A sample of 60 with one deviation has an upper limit of 7.7 percent at 95 percent confidence.
Can I test more items if the sample fails?
Only under a sequential plan fixed before testing, or by re-planning openly with a higher expected rate and evaluating the whole combined sample with every deviation counted. Adding items until the cumulative rate passes and then stopping inflates the real risk of overreliance and is not a legitimate procedure.
What is the difference between attribute sampling and monetary unit sampling?
Attribute sampling estimates how often a control deviates in a population of yes-or-no outcomes; monetary unit sampling estimates the dollar overstatement in a population of balances, giving large items a higher chance of selection. Controls tests use the first; substantive tests of balances use the second.
Does the population size matter?
Very little above about 1,000 items; the sample size is driven by confidence, tolerable rate and expected rate. For populations under a few hundred the finite population correction reduces the size somewhat, and for populations under about 50 statistical sampling is not meaningful and the auditor tests all items or samples judgmentally.
What is discovery sampling?
Attribute sampling designed to find at least one instance of a rare, serious condition if its rate is at or above a critical level: 299 items to be 95 percent confident of finding one instance when the rate is 1 percent. Finding one starts an investigation; finding none supports, weakly, that the rate is below the critical level.
How do I document an attribute sample?
With the attribute and deviation definitions, the population and its reconciliation, the three parameters with reasoning, the computed size, the selection method and seed, item-level results, the deviation investigations, the upper limit calculation and a conclusion sentence that carries the parameters. The sampling memo template on this site has the structure.
Related guides
- Audit sample sizes: 25, 40 and 60 — where the conventions come from.
- Monetary unit sampling — the method for overstatement of balances.
- Judgmental sampling — when statistics do not apply.
- Random sampling in Excel and Google Sheets — the selection mechanics.
- The foundations of random sampling in internal auditing — why randomness matters.
- Sampling techniques for high-risk areas — strategies by risk.
- Audit sampling memo template — recording the parameters.
- Handling audit test exceptions — investigating deviations.
- Test of design vs operating effectiveness — the two tests and the MidState design stage.
- IPE testing — reconciling the population listing.
- Audit workpaper example — the testing layout.
- Audit tick marks — the notation.
- Audit review notes — clearing the reviewer’s questions.
- Control deficiency evaluation — when a SOX key control fails.
- Compensating controls — the legitimate fallback.
- Control design principles — the deputy failure mode the case found.
- Writing audit finding conditions — the deviation pattern as condition.
- SQL for auditors — hashed-seed selection and full-population alternatives.
- When internal audit finds fraud — what follows a discovery sample’s find.
- Testing Controls in Small Populations: Annual to Monthly — what a sample of one, two or four can support, and what to do instead of pretending.
Leave a Reply