Sooner or later every internal auditor is told to substantively test something: an accrual, a reserve, a revenue cutoff, the cash a driver says he turned in. Most of us learned the phrase from external audit and carry a shorthand version around: pick forty items, vouch them, tick, conclude. That version collapses under the first hard question a reviewer or an audit committee member asks. How do you know the population was complete? What did you decide an exception was before you found one? What do two exceptions in forty say about the three million dollars you did not look at? A test that cannot answer those questions is not evidence about the balance; it is evidence that forty invoices existed.
This guide is the method. It gives you an assertion-by-assertion table of substantive procedures with a worked example and the evidence each one yields (22 rows), a test design template, a fully worked accrued-liabilities cutoff test with a monetary-unit sample of 40, two exceptions, a projection and an upper misstatement limit, a worked substantive analytical procedure on fleet fuel with an expectation, two thresholds and an investigation, and a failure table built from review notes. It was rewritten in September 2026 to reflect the Global Internal Audit Standards, effective 9 January 2025, and it sits alongside the guides on audit evidence and tests of design versus operating effectiveness, which cover the control side of the same engagement.
In this guide
- What substantive testing is, and what a test of controls cannot tell you
- Assertions and the procedure that answers each one
- Designing a test that holds up: seven decisions and a template
- Worked example: accrued liabilities cutoff at MidState Beverage
- Worked example: a substantive analytical procedure on fleet fuel
- When to test the whole population with analytics
- Documenting the conclusion so it survives review
- Common failures in substantive testing
- Adapting the method: small populations, estimates, interim work and pressure to move on
What substantive testing is, and what a test of controls cannot tell you
A substantive test is a procedure whose object is the number: you examine a transaction, a balance or a disclosure and come away with a statement about whether it is misstated, by how much and in which direction. A test of controls has a different object, an activity, and produces a statement about whether the activity operated. The two get confused because they often look at the same document. Pulling a June invoice to check for the approver’s signature is a test of controls; pulling the same invoice to establish that the service was received before June 30 and sits in the June accrual is a substantive test. The design versus operating effectiveness guide covers the first; this one covers the second.
The distinction matters because the two fail independently. A control can operate exactly as designed on a population that is wrong: at MidState Beverage the controller reviewed and signed every depot accrual form that reached her desk, and five of the twelve depots never sent one. A balance can also be right in the absence of any control, which is why “no exceptions” in a substantive sample says nothing about next quarter. When you need both statements, run both tests and write two conclusions.
| Dimension | Test of controls | Substantive test |
|---|---|---|
| Question answered | Did the control operate as designed throughout the period? | Is the amount misstated, by how much, and in which direction? |
| Object examined | An activity: an approval, a review, a reconciliation, a three-way match | A number: a transaction, a balance, a disclosure, or the physical quantity behind it |
| Population | Occurrences of the control (every settlement review, every accrual form) | Items making up the amount, or the source events that should have produced it |
| What an exception means | The control did not operate for that occurrence; you project a deviation rate | The item is misstated; you project a misstatement amount |
| Sample size logic | Attribute sampling: tolerable and expected deviation rates give the familiar 25, 40 and 60 | Value-based: population value, a confidence factor and a threshold, plus 100 percent strata for large items |
| What a clean result proves | The control operated at the tested frequency during the period | The amount is not misstated by more than the threshold, in the direction tested, at the stated confidence |
| What it cannot prove | That the amount is right | That the control operated, or that the amount will be right next period |
| How the conclusion reads | “0 deviations in 40 occurrences; the control operated effectively in FY26.” | “Unrecorded liabilities at June 30, 2026 are not expected to exceed $125,000 at 80 percent confidence.” |
Substantive procedures come in two families. Tests of details examine individual items: inspection of documents, confirmation with third parties, physical observation, recalculation, reperformance, and vouching or tracing between a record and the event behind it. Substantive analytical procedures examine a relationship: an independent expectation of what the amount should be, compared with what was recorded, with the difference investigated. The unit of evidence, an item versus a relationship, drives how each is designed, sized and evaluated.
External auditors size substantive work from the risk of material misstatement. Internal auditors do not opine on financial statements, but the same arithmetic decides when you should be testing substantively at all, and five situations account for nearly all of it: ICFR support, where the external auditor relies on your work under SOX 404 and the test has to meet their methodology; quantifying a failed control, because a deficiency’s severity turns on the magnitude of misstatement that could result (see control deficiency evaluation); fraud allegations, where the only question is what happened to the money; operational engagements where the “balance” is hours-of-service logs, keg deposits or route settlements; and any engagement where management is about to act on a number nobody has independently tested. The Global Internal Audit Standards put it plainly in Standard 14.1: the information you gather must be relevant, reliable and sufficient. When control evidence is thin, substantive procedures are how you reach sufficient.
Assertions and the procedure that answers each one
An assertion is the specific claim a recorded number makes, and it is the most useful design tool you have because it tells you which direction to test. The seven used below follow AU-C 315 and ISA 315 (Revised 2019), which separate assertions about transactions (occurrence, completeness, accuracy, cutoff, classification) from assertions about balances (existence, rights and obligations, completeness, valuation); the PCAOB’s AS 1105 compresses them into five. Internal auditors are bound by neither, and the habit matters more than the taxonomy. “Test accounts payable” is not a test. “Establish that liabilities incurred on or before June 30 are recorded at June 30” is one, with a direction, a population and a stopping rule.
Direction of testing is the most common design error I see in review. To test existence or occurrence you start from the recorded item and go outward to the thing that proves it happened; that is vouching. To test completeness you start from the event and go inward to the record; that is tracing. A sample drawn from the ledger can never test the completeness of the ledger, because whatever is missing from the ledger cannot be in the sample. MidState’s route cash engagement learned this the hard way: sampling cash receipts postings found nothing wrong, because the $18,400 the Dayton driver diverted over five months was never posted. The test that found the pattern started from signed delivery tickets and customer remittance stubs and asked where each one landed.
| Assertion | Substantive procedure | MidState Beverage example | Evidence obtained and its weight |
|---|---|---|---|
| Existence (inventory) | Observe the count; test-count floor to sheet and sheet to floor | June 30 count at the Columbus depot: 60 SKUs counted each way, breakage and out-of-code product noted | Auditor’s own observation; the strongest evidence there is, but only for that date and place |
| Existence (receivables) | Confirm directly with the debtor, the auditor controlling the request and the reply address | The 45 largest chain-account balances at June 30 ($4.1 million of $9.7 million); every difference resolved to invoice level | Third-party response received by the auditor; strong for existence, weak for valuation because customers do not confirm they will pay |
| Existence (cash) | Obtain bank statements or confirmations directly from the bank | June 30 statements for the 14 depot deposit accounts pulled under the auditor’s own read-only bank login and agreed to the reconciliations | External evidence obtained directly; silent on deposits in transit |
| Existence (fixed assets) | Physically verify items selected from the register | 30 vehicles selected from the fleet register and sighted; VIN, plate, condition and location recorded | Auditor observation; also yields valuation evidence for idle or wrecked units |
| Occurrence (revenue) | Vouch recorded sales to delivery evidence and to the settlement of the cash | 40 invoices vouched to the signed delivery ticket, the handheld scan record and the route settlement in which the cash or check came in | Internal documents with an external element (the customer’s signature); moderate |
| Occurrence (journal entries) | Vouch manual entries to the business event and the approval | All 23 manual entries over $25,000 in the June close vouched to their calculation and the event behind them | Strong where the entry ties to an external document, weak where the support is the poster’s own spreadsheet |
| Completeness (liabilities) | Search subsequent invoices and disbursements for liabilities incurred before period-end | The worked example below: 2,361 invoices and payments from July 1 to August 7 screened for June service dates | Vendor invoices and service tickets; strong when the service date comes from a document rather than the invoice date |
| Completeness (revenue) | Trace from source records into the ledger | 40 signed delivery tickets pulled from depot files and traced to the invoice and the revenue posting | Direction is the point: starting outside the ledger is the only way to find what it lacks |
| Completeness (cash receipts) | Trace receipts to the deposit and to the posting | For 30 depot-days, driver settlement sheets and customer remittances traced to the validated deposit slip and the receivables posting | Bank validation plus internal records; strong, and the test that would have caught the Dayton diversion inside a month |
| Completeness (records) | Sequence-check the numbered documents | FY26 delivery ticket range tested for gaps and duplicates across 100 percent of records; every void over $500 inspected | System data; strong for the record set, weak for the validity of the voids |
| Completeness (payables) | Reconcile vendor statements to the subledger plus accruals | June statements from the 25 largest non-product vendors reconciled to accounts payable plus accruals at June 30 | External document; strong, but only for vendors that send statements |
| Accuracy (depreciation) | Recalculate independently | FY26 depreciation for all 342 fleet units recomputed from cost, in-service date and policy lives, and compared by unit | Auditor recalculation; strong; differences are usually policy-life or in-service-date errors |
| Accuracy (pricing) | Reperform against the authorized master data | 40 invoices re-priced against the price file in effect on the invoice date; volume discounts and container deposits recomputed | Master data plus recalculation; strong if the price-file change log has been tested |
| Accuracy (payroll) | Recompute gross pay from source hours and rates | 40 driver pay records recomputed from handheld route hours, approved overtime and the rate in the HR system | Recalculation from system data; strong, subject to the reliability of the time source |
| Cutoff (revenue) | Test the last and first transactions around period-end to delivery dates | The last 25 delivery tickets before and the first 25 after June 30, each agreed to the period the goods left the depot | Customer-signed dates; strong; test both sides of the date or you have tested one direction only |
| Cutoff (purchases and inventory) | Match receiving reports either side of period-end to inventory and payables | Receiving reports dated June 26 to July 5: goods received by June 30 confirmed in both inventory and the goods-received-not-invoiced accrual | Receiving documents plus postings; strong; the classic error is goods in inventory but not in payables |
| Cutoff (accruals) | Search subsequent invoices for service periods that predate the cutoff date | The worked example below | Service tickets and timesheets; the invoice date is not evidence of the service period |
| Valuation (receivables) | Test the allowance inputs and look at subsequent cash | Aging recomputed from invoice dates; 40 balances over 90 days checked for collection through August 15; allowance compared with the uncollected remainder | Subsequent cash is external evidence of collectability; strong; recomputing the aging catches re-dated invoices |
| Valuation (inventory) | Identify damaged, out-of-code or slow stock and compare with the reserve | SKUs past their best-by date and damaged cases listed at the count and priced at cost ($218,000) against the $140,000 obsolescence reserve | Observation plus recalculation; strong; it produces a number the reserve has to answer to |
| Rights (inventory) | Separate owned stock from supplier-owned items | Brewer-owned kegs, pallets and coolers at depots confirmed excluded from inventory; supplier agreements read | Contracts; strong; counting supplier-owned assets overstates inventory and equity together |
| Obligations (leases) | Read the agreements and recompute the liability | Six fleet leases: term, payments and rate agreed to the ASC 842 schedule and the June 30 liability recomputed | Contracts plus recalculation; strong |
| Classification (repairs versus capital) | Test large repair invoices against the capitalization policy | The 30 fleet maintenance invoices over $5,000 in FY26 classified under the policy: engines and transmissions capitalize, tires and brakes expense | Invoices plus policy; strong; expect the errors to run in one direction |
Two things in the last column deserve emphasis. Evidence has weight, and the weight comes from who produced it and who handed it to you: a confirmation you received, a statement you pulled yourself and a contract outrank an invoice held by the auditee, which outranks an accrual form, which outranks an explanation; the audit evidence guide sets out the hierarchy. And every population you select from and every schedule you foot is information produced by the entity, so a test built on an unvalidated report inherits every error in that report. Completeness of the population is an assertion in its own right, and the IPE testing guide covers proving it before you select a single item.
Designing a test that holds up: seven decisions and a template
Seven decisions, made in this order and written down before the first item is pulled. A workpaper that cannot show all seven does not hold up, however carefully each item was examined, because a reviewer cannot tell whether the conclusion follows from the work. GIAS Standard 13.6 expects the procedures in a work program before fieldwork; the work program guide shows the house format, and the seven decisions are what its substantive rows need to contain.
1. Objective and assertion. One sentence, one direction. “Determine whether liabilities for goods and services received on or before June 30, 2026 are recorded in accounts payable or accrued liabilities at June 30, 2026” tests completeness and cutoff in the understatement direction. If the sentence needs “and” between two assertions, check that one procedure really answers both; existence and completeness almost never share a procedure, because they run in opposite directions.
2. Population and its completeness. Define the frame you will select from, reconcile it to a control total that did not come from the same report, and name what the frame cannot contain so you can cover it separately. For a subsequent-disbursements search that means the invoice register tied to subledger activity, plus the wire log, plus the tray of invoices that were received but never entered. The item that hurts you is the one that never made it into the file you sampled from.
3. Threshold. The amount of misstatement at which your conclusion changes, fixed before selection and written into the planning memo. In ICFR support it sits below the external auditor’s performance materiality, so that anything you find is something they would care about; in an operational engagement it is the amount that would change a decision. A useful check is whether the CFO would book an adjustment of that size without argument; if not, the threshold is too high. Under GIAS Standard 13.4 this is your evaluation criterion, and the reader of the report is entitled to know it.
4. Selection method and size. Examine 100 percent of any item large enough to breach the threshold on its own, and usually everything above a lower cut, because a 100 percent stratum needs no projection. Sample the rest by value or at random with a documented method (the free mySampler tool, or an Excel workbook you retain with the seed), never by hand from a printout. Size the sample with the value-based formula most methodologies use: population value, multiplied by the confidence factor for zero expected misstatements, divided by the threshold. The 25, 40 and 60 sizes are attribute sizes for tests of controls; importing 25 into a value test of a $3 million population is a reliable way to produce a test that cannot support its own conclusion. If you expect misstatements the zero row is too small; the AICPA Audit Sampling guide’s tables expand it, and the sampling techniques guide walks through the choices.
| Misstatements found in the sample | 95 percent confidence | 90 percent confidence | 80 percent confidence | 75 percent confidence |
|---|---|---|---|---|
| 0 | 3.00 | 2.31 | 1.61 | 1.39 |
| 1 | 4.75 | 3.89 | 3.00 | 2.70 |
| 2 | 6.30 | 5.33 | 4.28 | 3.93 |
| 3 | 7.76 | 6.69 | 5.52 | 5.11 |
| Sample size for a $3,066,100 stratum, $125,000 threshold, zero expected misstatements (value × zero-row factor ÷ threshold) | 74 | 57 | 40 | 35 |
The factors are Poisson upper limits rounded up to two decimals, the same numbers published in the AICPA guide and built into every monetary-unit sampling tool. Use 95 percent when the test is the only evidence on the assertion; 90 or 80 when a 100 percent stratum, an analytic or tested controls carry part of the assurance and the workpaper says which part; never below 75 for a balance you intend to write a conclusion about. State the confidence level in the conclusion itself.
5. Procedure. The steps, the documents, and the definition of “agreed” for each attribute, including which document wins when two disagree: a service ticket signed by the depot manager beats the invoice date, a bill of lading beats the invoice line, a bank validation beats a deposit slip. Write it so that a colleague who has never seen the process could perform it, which is also what makes the workpaper reperformable at review.
6. Exception definition. Written before the first item, because the urge to reclassify the first exception as “not really an exception” is strong and always sounds reasonable in the moment. State what counts (any misstatement in the tested direction, of any amount), what does not (an over-accrual when you are testing understatement, a misspelled vendor name, a document that is late but correct), and how partial misstatements are measured (the unrecorded portion, not the whole invoice). Log the non-exceptions anyway; they are often findings of another kind.
7. Evaluation and projection. Decide in advance how the result will be evaluated: known misstatement from the 100 percent strata, projected misstatement from the sampled strata, an allowance for sampling risk, and the comparison with the threshold. Decide the response tree too: pass, expand the sample, ask management to sweep the population, or fail. An evaluation method chosen after the results are known was chosen to fit them.
Substantive test design: [account or balance], [period-end date]. Engagement [reference]. Prepared by [name], [date]. Reviewed by [name], [date].
Objective and assertion. Determine whether [population] is [complete / recorded in the correct period / accurately stated / existing] at [date]. Direction tested: [overstatement / understatement]. Assertions not addressed by this test: [list].
Population and completeness. Frame: [report, system, parameters, run date, run by]; [count and value]. Reconciled to [independent control total], difference [amount], explained by [reason]. Items the frame cannot contain and how they are covered: [list]. IPE validation at [workpaper reference].
Threshold. [Amount], derived from [performance materiality / the decision the number supports / a regulatory limit], agreed with [name] on [date], planning memo reference [reference].
Selection. Items at or above [cut] examined 100 percent: [count, value]. Remaining stratum [count, value] sampled by [monetary-unit / random] selection at [confidence] percent using [tool, seed or random start]: sample of [n], interval [amount].
Procedure. For each item: (1) obtain [document]; (2) establish [attribute] from [source of truth, in order of precedence]; (3) agree to [record]; (4) record the evidence reference. “Agreed” means [definition, including any tolerance].
Exception definition. An exception is [definition]. Partial misstatements are measured as [method]. Logged but not counted: [list].
Evaluation. Known misstatement (100 percent strata) plus projected misstatement (sampled strata) plus the allowance for sampling risk at [confidence] percent, compared with the threshold. If the upper limit exceeds the threshold: [expand to n / management sweep / report as a finding].
Conclusion. [Written after evaluation; states the assertion, the direction, the date, the threshold and the confidence level.]
Worked example: accrued liabilities cutoff at MidState Beverage
MidState Beverage’s fiscal year ends June 30. The FY27 audit plan carries an ICFR support engagement and a review of the financial close under turnover, and the two collided in July 2026: the controller who chased the twelve depot accrual forms every month left in May, two depot managers were new, and the June close was run by an assistant controller on a 2013 ERP with accruals assembled from depot spreadsheets. The CFO asked internal audit to perform the search for unrecorded liabilities before the external auditors arrived in September. The design follows the template above; the numbers are the engagement’s.
Objective. Determine whether liabilities for non-product goods and services received on or before June 30, 2026 are recorded in accounts payable (GL 2000) or accrued liabilities (GL 2150 accrued expenses and GL 2160 accrued freight) at June 30, 2026: completeness and cutoff of liabilities, understatement direction. Product purchases were scoped out because they are matched to receiving reports and accrued automatically through the goods-received-not-invoiced account, and receiving cutoff was tested in the warehouse inventory engagement; the scoping decision was written into the objective, not discovered later. Accrued liabilities at June 30 stood at $2,384,600.
Population. The frame came from three sources. The AP invoice register for July 1 to August 7, 2026 (the ledger lock date) listed 3,412 invoices totaling $12,731,900, which internal audit tied to subledger posting activity for the same dates before doing anything else; removing the 1,094 product invoices ($7,819,500) left 2,318 non-product invoices totaling $4,912,400. The treasury log added two manual payments with no AP invoice. The scanning queue added 41 invoices received but not yet entered on August 7, a population that no ERP report will ever contain.
| Source | Items | Value | Treatment |
|---|---|---|---|
| AP register, non-product invoices posted July 1 to August 7, 2026, at or above $40,000 (stratum A) | 27 | $1,846,300 | Examined 100 percent; no projection |
| AP register, non-product invoices below $40,000 (stratum B) | 2,291 | $3,066,100 | Monetary-unit sample of 40 at 80 percent confidence |
| Invoices received but not yet entered at August 7 (scanning queue) | 41 | $63,200 | Examined 100 percent |
| Manual checks and wires July 1 to August 7 with no AP invoice | 2 | $18,400 | Examined 100 percent |
| Total frame | 2,361 | $4,994,000 | Register tied to subledger activity; queue counted by the auditor; wires agreed to the bank statement |
Threshold and selection. The threshold was $125,000, set in the planning memo below the external auditor’s performance materiality. Internal audit chose 80 percent confidence for stratum B because stratum A, 38 percent of the value, was examined in full and because the vendor statement reconciliations for the 25 largest non-product vendors gave separate evidence on the same assertion. The formula gives the sample, $3,066,100 × 1.61 ÷ $125,000 = 39.5, rounded to 40, and the interval, $3,066,100 ÷ 40 = $76,652.50. A random start of $41,377 was generated and recorded, the stratum was sorted by invoice number with a cumulative value column, and the invoice containing each successive multiple of the interval was selected; the 40 invoices totaled $391,200. Because every item in the stratum is below the interval, none could be selected twice.
- Obtain the invoice and any packing slip, service ticket, timesheet or bill of lading attached to it in the imaging system.
- Establish the service or receipt date from the attachment, never from the invoice date; allocate an invoice that straddles June 30 by days or by units delivered.
- If the date is on or before June 30, locate the liability in the June 30 AP subledger or the accrual schedule by vendor, depot and amount, and record the reference. Recorded in full: no exception. Not recorded, or recorded for less: the misstatement is the unrecorded portion.
- If the date is after June 30, the item is a July expense and passes; record the evidence anyway.
- Resolve disputes about whether a service was complete by the document that evidences delivery, not by asking the vendor or the depot manager.
Exception definition. Any portion of a liability for goods or services received on or before June 30, 2026 that was not recorded in AP or accrued liabilities at June 30. Over-accruals were logged for the close review but not counted, because the direction tested is understatement. Stratum A produced no exceptions; the queue held three June-service invoices totaling $4,900, all in the accrual; both manual payments were July rent. The sample produced two exceptions, shown with six of the passing items below.
| Item | Vendor and invoice | Amount | Service or receipt period (source) | Recorded at June 30 | Unrecorded |
|---|---|---|---|---|---|
| 1 | Kessler Refrigeration Service, dated July 9 | $7,850 | Compressor replacement, Toledo depot cooler; service ticket signed June 24 | Not in AP; not on the Toledo accrual form, which was never submitted | $7,850 (exception) |
| 2 | Riverbend Staffing, dated July 6 | $4,120 | Warehouse temporary labor, week ended June 28 (timesheets) | Accrued $2,600, estimated from the prior week’s invoice | $1,520 (exception) |
| 3 | Buckeye Crossdock Freight, dated July 3 | $3,940 | LTL shipments June 26 to 30 (bills of lading) | Accrued $3,940 in GL 2160 | None |
| 4 | Toledo depot electric utility, dated July 8 | $2,215 | Service June 5 to July 6; June portion 26 of 32 days | Accrued $1,800 by pro-ration | None |
| 5 | Hartwell Tire and Brake, dated July 14 | $1,285 | Tires fitted to unit 217 on July 9 (work order) | Not recorded; July expense | None |
| 6 | Capitol Ridge Property Management, dated July 1 | $9,500 | July rent, Springfield depot (lease) | Not recorded; July expense | None |
| 7 | Marlowe Uniform Rental, dated July 2 | $612 | Rental week June 22 to 28 (delivery manifest) | Accrued $612 | None |
| 8 | Scioto Bulk Fuels, dated July 2 | $26,900 | 7,400-gallon delivery to the Dayton tank on June 29 (delivery ticket) | Accrued $26,900 from the depot fuel receipt log | None |
Evaluation. In monetary-unit sampling each exception is expressed as a tainting, the misstated fraction of the item, and each tainting is projected across one sampling interval, on the reasoning that the selected dollar stood for every dollar in its interval. The allowance for sampling risk has two parts: basic precision, which is the upper limit you would have with no misstatements at all, and an incremental allowance for each misstatement found, computed from the change in the confidence factor as the count rises, with the taintings ranked largest first.
| Step | Computation | Amount |
|---|---|---|
| Sampling interval | $3,066,100 ÷ 40 | $76,652.50 |
| Tainting, item 1 (Kessler) | $7,850 unrecorded ÷ $7,850 invoice | 100.0 percent |
| Tainting, item 2 (Riverbend) | $1,520 unrecorded ÷ $4,120 invoice | 36.9 percent |
| Projected misstatement, item 1 | 1.000 × $76,652.50 | $76,652 |
| Projected misstatement, item 2 | ($1,520 ÷ $4,120) × $76,652.50 | $28,280 |
| Projected misstatement, stratum B | $76,652 + $28,280 | $104,932 |
| Basic precision at 80 percent | 1.61 × $76,652.50 | $123,411 |
| Incremental allowance, item 1 | (3.00 − 1.61 − 1.00) × 1.000 × $76,652.50 | $29,894 |
| Incremental allowance, item 2 | (4.28 − 3.00 − 1.00) × ($1,520 ÷ $4,120) × $76,652.50 | $7,918 |
| Upper misstatement limit at 80 percent | $104,932 + $123,411 + $29,894 + $7,918 | $266,155 |
| Known misstatement, 100 percent strata | Stratum A, queue and manual payments | $0 |
| Threshold | Planning memo | $125,000 |
| Same sample evaluated at 95 percent, for comparison | Factors 3.00, 4.75 and 6.30 | $407,933 |
Decision. The projected misstatement alone, $104,932, is 84 percent of the threshold before any allowance for sampling risk, and the upper limit of $266,155 is more than twice it. The sample does not support a conclusion that unrecorded liabilities are below $125,000, and arguing about the two items does not change that; they are what the population looks like when you hold it up to the light. Management’s first instinct was to book $9,370 for the two invoices and close the point. The projection is the answer to that instinct: the stratum is carrying something like $105,000 of unrecorded liabilities and could be carrying $266,000, against a balance of $2,384,600.
Expanding the sample was considered and rejected, and the arithmetic comes up in every one of these conversations. Another 40 items with no further exceptions would halve the interval to $38,326, cut the projected misstatement to $52,466 and bring the upper limit to about $133,000, still above the threshold. Only a sample of 120 with no exceptions in the additional 80 items would reach roughly $88,700, and a stratum that produced two taintings in 40 is not one where 80 clean items is a good bet. Expansion is the right response to a marginal sample and the wrong one to a clear failure. Internal audit asked management to sweep the population instead: faster, and it fixes the books rather than the sample.
The sweep and the outcome. The AP supervisor and the assistant controller screened all 2,291 stratum B invoices and the 41 queued invoices by service or receipt date in four days. Of the 214 invoices with pre-July service dates, 162 were recorded at June 30, 38 were not recorded at all ($186,200) and 14 were under-accrued ($28,500): an unrecorded total of $214,700, 7.0 percent of the stratum’s value and 9.0 percent of the accrual balance. Internal audit reperformed the screen on 30 items, including the 15 largest that management had classified as recorded, with no differences, and the adjustment took accrued liabilities from $2,384,600 to $2,599,300. Note where the actual sits: twice the projected point estimate and inside the 80 percent upper limit. The point estimate was the center of a wide interval; the upper limit was the number that told the truth. The finding was rated on the site’s severity scale and written with the five Cs: condition, a $214,700 understatement; cause, five of twelve accrual forms not submitted and no screening of July invoices by service date before the ledger lock; effect, liabilities understated and FY26 pre-tax income overstated by the same amount; action, a 12-of-12 form checklist gating the close and a service-date screen of every invoice posted in the first 20 days after period-end.
Conclusion as written in the workpaper. Based on the procedures at W/P 4.2 to 4.6, liabilities for non-product goods and services received on or before June 30, 2026 were understated by $214,700, identified through a management sweep of the full population after a monetary-unit sample of 40 produced a projected misstatement of $104,932 and an upper misstatement limit of $266,155 at 80 percent confidence against a $125,000 threshold. Following the adjustment recorded on August 21, 2026 and internal audit’s reperformance of the sweep on 30 items with no differences, unrecorded non-product liabilities at June 30, 2026 are not expected to exceed $125,000. This conclusion does not extend to product purchases, which were tested in the warehouse inventory engagement, or to the operating effectiveness of the accrual control, which is reported as a finding at 4.7.
Worked example: a substantive analytical procedure on fleet fuel
An analytical procedure is substantive only when four conditions hold: the relationship between the amount and its drivers is plausible and predictable; the inputs are independent of the amount being tested and reliable in their own right; the expectation is precise enough for the threshold you need to support; and every difference above the threshold is investigated and corroborated rather than explained. Comparing this year’s balance with last year’s fails the first condition for most accounts, which is why a fluctuation analysis is a risk assessment tool and not evidence. Fleet fuel at MidState passes all four: miles driven, fuel efficiency and price per gallon determine the expense, and each has a source with nothing to do with the general ledger.
Expectation. Expected fuel expense for each month equals telematics miles for the 342 legacy fleet units, divided by 6.8 miles per gallon, multiplied by the regional average diesel price for the month less the $0.06 per gallon discount in the fuel card agreement. Miles come from the telematics platform’s monthly export by vehicle, reconciled to the fleet register. The 6.8 came from the FY24 and FY25 fleet averages (6.77 and 6.83) and was checked against the manufacturers’ figures for the two truck classes. The price came from a public regional diesel series averaged by month; the figures in the table are the example’s inputs, not a quotation of any published series. Recorded expense is GL 6210 by month, tied to the trial balance. The two acquired distributors run their own fleets on their own ledgers and were excluded, and the exclusion is stated in the conclusion.
Thresholds. Two gates. Each month was compared against $15,000, about 4.7 percent of an average month’s expectation and roughly the size of the model’s own noise: a 0.1 mile-per-gallon error in the efficiency assumption moves the annual expectation by about $57,000 and a five-cent price error moves it by about $53,000, so a precision of a few percent per month is the best this model can do. Dividing the $125,000 engagement threshold across twelve months would have given $10,400 and flagged noise every other month. The second gate is the annual net difference against $125,000, which catches the monthly gate’s arithmetic weakness (twelve months at $14,999 in the same direction would be $180,000). Both gates were set before the comparison was run.
| Month | Telematics miles | Expected gallons (÷ 6.8) | Net price per gallon | Expected expense | Recorded (GL 6210) | Difference | Status against $15,000 |
|---|---|---|---|---|---|---|---|
| July 2025 | 641,200 | 94,294 | $3.65 | $344,174 | $348,374 | +$4,200 | Within |
| August 2025 | 652,800 | 96,000 | $3.62 | $347,520 | $344,420 | −$3,100 | Within |
| September 2025 | 612,400 | 90,059 | $3.60 | $324,212 | $331,012 | +$6,800 | Within |
| October 2025 | 598,900 | 88,074 | $3.56 | $313,542 | $310,842 | −$2,700 | Within |
| November 2025 | 571,300 | 84,015 | $3.53 | $296,572 | $298,472 | +$1,900 | Within |
| December 2025 | 583,600 | 85,824 | $3.49 | $299,524 | $294,124 | −$5,400 | Within |
| January 2026 | 548,200 | 80,618 | $3.52 | $283,774 | $287,374 | +$3,600 | Within |
| February 2026 | 559,700 | 82,309 | $3.58 | $294,666 | $296,766 | +$2,100 | Flagged on the first pass at −$17,906; cleared after 38,000 duplicate miles were removed |
| March 2026 | 576,900 | 84,838 | $3.66 | $310,508 | $285,608 | −$24,900 | Flagged; investigated |
| April 2026 | 601,300 | 88,426 | $3.73 | $329,831 | $352,431 | +$22,600 | Flagged; investigated |
| May 2026 | 628,400 | 92,412 | $3.70 | $341,924 | $347,024 | +$5,100 | Within |
| June 2026 | 645,100 | 94,868 | $3.75 | $355,754 | $352,854 | −$2,900 | Within |
| FY26 total | 7,219,800 | 1,061,735 | $3.62 weighted | $3,842,001 | $3,849,301 | +$7,300 (0.19 percent) | Annual gate of $125,000 met |
Dollar and gallon figures are computed from unrounded inputs, so a column may differ from its total by a few units. The first pass flagged February at −$17,906, and the problem was in the model, not the ledger: nine trucks had their telematics units replaced in February and both the old and the new device reported for most of the month, adding 38,000 miles that were never driven. The vendor’s device inventory export listed the duplicate identifiers with their active dates, the fleet manager’s monthly report had not de-duplicated them, and removing the miles cut the February expectation by $20,006, leaving a residual of +$2,100. This is the normal experience with a first analytic: the first thing it finds is a flaw in your own inputs. Fix the model, document the fix and the fact that it was made after seeing the difference, and never absorb the difference into an explanation.
March and April were the genuine flags, opposite in sign and similar in size, which is the signature of a cutoff error between months. The fleet manager’s explanation was that a bulk delivery had been “invoiced late.” That is a hypothesis, and it was corroborated with two documents that do not depend on him: the Scioto Bulk Fuels delivery ticket for 7,500 gallons to the Dayton depot tank dated March 27, and the depot’s tank gauge log showing the receipt that day. The $27,400 invoice was dated April 2 and posted on April 6 with no March accrual. Moving it back leaves residuals of +$2,500 for March and −$4,800 for April, both within the gate. For the year, recorded expense of $3,849,301 sits $7,300 (0.19 percent) above the expectation of $3,842,001, the absolute monthly residuals after investigation sum to $45,100, and both gates are met. The conclusion, scoped to the legacy fleet and the $125,000 threshold, was that FY26 fleet fuel expense is fairly stated.
Two observations came out of a test that passed. The March delivery was not accrued at month-end, while the June 29 delivery of 7,400 gallons to the same tank was accrued correctly at year-end (item 8 in the cutoff sample): year-end closes get attention that monthly closes do not, and an analytic run on annual totals alone would have shown +$7,300 and never revealed it. And the duplicate devices had been inflating the fleet manager’s miles-per-gallon reporting to operations for a month, which went to the fleet and DOT advisory engagement in the FY27 plan. Analytics built from drivers rather than from last year’s number are the fastest route to this kind of operating insight.
When to test the whole population with analytics
Sampling exists because examining every item was once impossible. For any attribute that is machine-readable in complete, validated data it no longer is, and a full-population procedure removes sampling risk for that attribute. The conditions are strict, and each is a place where the procedure quietly stops being substantive: the data must be validated as IPE, the attribute must be codable without judgment, the exception definition must be a rule the code can apply, and the flagged items must still be examined by a person. What the procedure proves is exactly the attribute you coded, and nothing adjacent to it.
| Full-population procedure | Assertion | What it proves | What it does not prove | Follow-up still needed |
|---|---|---|---|---|
| Sequence-gap and duplicate test on FY26 delivery ticket numbers | Completeness of revenue records | Every number in the range is accounted for once | That tickets outside the range were not issued; that voided tickets were legitimate | Inspect every void over $500 and 25 random voids to the customer’s confirmation of non-delivery |
| Duplicate payment scan (same vendor and amount within seven days; normalized invoice numbers) | Occurrence and accuracy of payables | Which payments are candidates | That a candidate is a duplicate; recurring legitimate charges look identical | Inspect every candidate to the invoice and the vendor statement; see AP analytics |
| Journal entry scan: entries after the ledger lock, round amounts, non-accounting posters, weekend postings | Occurrence and classification | Which entries carry the risk characteristics you defined | That an entry without those characteristics is proper | Vouch every flagged entry and 25 unflagged entries; see journal entry analytics and journal entry testing |
| Recompute the three-way match tolerance on 100 percent of product invoices | Accuracy of purchases | Price and quantity variances are within tolerance for every invoice | That the receiving quantity was right in the first place | Tie 25 receiving reports to the physical count records |
| Service-date versus posting-period comparison on 100 percent of invoices | Cutoff of liabilities | Every invoice with a populated service date is in the right period | Anything about invoices whose service-date field is blank or defaulted; MidState’s 2013 ERP has no such field, which is why the cutoff test above had to be a sample | Sample 40 invoices with blank or defaulted fields to the source document |
| Re-price 100 percent of FY26 invoice lines against the price file effective on the invoice date | Accuracy of revenue | Every line was billed at the price in the file | That the price file was authorized | Test 25 price-file changes to their approvals; see revenue recognition |
The hybrid design in the last column is the practical answer for most engagements: run the full-population procedure, examine every flag, and pull a small random sample of unflagged items to confirm that “no flag” means “no problem.” That second sample is what separates an analytic from a screen. It is also the design that moves cleanly into a continuous procedure, which the continuous auditing versus monitoring guide covers, provided you remember that a scan that finds nothing has proved only that nothing matched the rule.
Documenting the conclusion so it survives review
GIAS Standard 14.6 requires documentation sufficient to support the engagement results, and for a substantive test that has a precise meaning: a reviewer who has never spoken to you must be able to see the population and how its completeness was established, the threshold and where it came from, the selection parameters (method, confidence, interval, random start or seed), an evidence reference for every item including the ones that passed, each exception with its measurement, the evaluation arithmetic, and a conclusion that follows from it. The workpaper example shows the layout; the order of the seven design decisions is the order of the workpaper. Tickmarks with no document reference behind them are the most common review note on substantive work, and a conclusion that asserts more than the test examined is the second.
| Conclusion as drafted | Why it fails review | Conclusion that holds up |
|---|---|---|
| “Accrued liabilities are complete at June 30, 2026.” | No scope, no threshold, no confidence level; asserts a certainty no sample can deliver, and covers product purchases the test excluded | “Unrecorded non-product liabilities at June 30, 2026 are not expected to exceed $125,000 at 80 percent confidence, after the $214,700 adjustment recorded on August 21, 2026.” |
| “No exceptions noted; the accrual control is operating effectively.” | Converts a substantive result into a control conclusion; the items were never examined for whether the control operated | “No misstatements were identified in the 40 items examined. This test does not address whether the accrual review control operated; see W/P 3.4.” |
| “Fuel expense appears reasonable.” | “Reasonable” is not a criterion, and “appears” tells the reader the auditor did not decide | “FY26 fleet fuel expense for the legacy fleet of $3,849,301 is within $7,300 (0.19 percent) of an independent expectation; no monthly difference exceeded $15,000 after investigation of March and April; the procedure supports the account within the $125,000 threshold.” |
| “Two errors were found and corrected by management; no further action.” | Corrects the known misstatements and ignores what they imply about the population | “Two misstatements in a sample of 40 project to $104,932 with an upper limit of $266,155 against a $125,000 threshold; the sample does not support the balance. See the management sweep at W/P 4.6.” |
When the test fails, separate the two things you have found. The adjustment is a number, and the report should give both the known and the projected figures with the confidence level, because a committee that hears “$9,370” and one that hears “$214,700 after a full sweep, from a sample that projected $104,932” will react differently, and only the second reaction is warranted. The control weakness is a finding, written with condition, criteria, cause, effect and corrective action, rated on its own merits, and tracked to closure through issue validation. GIAS Standard 14.5 asks for conclusions that weigh the significance of what was found against the engagement objectives; a conclusion that states the assertion, the direction, the date, the threshold and the confidence level meets that bar, and “no exceptions noted” does not.
Common failures in substantive testing
These come from review notes, quality assessments and the occasional restatement post-mortem. Each produces a workpaper that looks finished and proves less than it claims, and the table runs roughly in the order of the seven design decisions, because that is where each failure is introduced.
| Failure | What it looks like | Why it matters | Fix |
|---|---|---|---|
| Testing completeness from the ledger | A sample of recorded accruals, each vouched to an invoice, concluding “accruals are complete” | Nothing missing from the ledger can appear in a sample of the ledger; the test proves existence and calls it completeness | Start from the source: subsequent invoices, delivery tickets, receiving reports, remittances. Write the direction into the objective |
| Population from an unvalidated report | An ERP extract used as the frame with no tie to a control total; parameters and run date not recorded | Every filter, date range and permission gap in the report becomes a hole in the population, invisible to the sample | Tie the frame to an independent total, record the parameters, and cover what the report cannot contain (the scanning queue, the wire log) |
| Threshold set after the results | The threshold appears only in the conclusion, conveniently above the projected misstatement | A conclusion cannot be independent of the result if the criterion was chosen to fit it | Fix the threshold in the planning memo with its derivation, signed by the manager before selection |
| Attribute sizes in a value test | “Sample of 25 per methodology” against a $3 million stratum with a $125,000 threshold | 25 items at 95 percent supports a threshold of about $368,000 on that stratum, three times what the conclusion claims | Size from population value, confidence factor and threshold, and put the arithmetic in the workpaper |
| Invoice date used as the cutoff date | All July-dated invoices treated as July expense | The exception in the worked example was dated July 9 for a June 24 service; the invoice date measures the vendor’s billing cycle, not the liability | Establish the service or receipt date from the attachment; where none exists, request it and log the item as unsupported |
| Exception redefined after it is found | “Timing difference, not an error” written beside the first exception | Every cutoff error is a timing difference; the reclassification empties the test of meaning | Write the exception definition before fieldwork and apply it without discussion; log disagreements for the manager |
| Known errors corrected, projection ignored | Management books the two invoices found; the workpaper closes with “corrected, no further action” | The sample was there to tell you about the population; two exceptions in 40 are the population speaking | Project, add the allowance for sampling risk, compare with the threshold, and follow the response tree agreed at planning |
| Explanation accepted as corroboration | “Per the fleet manager, the difference relates to a late invoice” | An explanation is a hypothesis; a plausible one is still unverified, and plausible explanations are what a fraudster supplies | Corroborate with a document independent of the person explaining: the delivery ticket, the device inventory, the bank statement |
| Expectation built from the number being tested | “Fuel expense per gallon purchased” compared with recorded fuel expense | The expectation and the amount share a source, so they agree by construction | Build expectations from operational drivers (miles, headcount, cases, days) and external prices |
| Conclusion drift | “No exceptions” becomes “the balance is fairly stated” becomes “controls are effective” by the time the report is drafted | Each restatement adds an assertion the test did not examine, and the committee acts on the last version | Copy the workpaper conclusion into the report verbatim, with its scope, threshold and confidence level |
Adapting the method: small populations, estimates, interim work and pressure to move on
Small populations. Below roughly a hundred items, or wherever a handful of items carry most of the value, sampling is the wrong tool. Examine everything, or examine everything above a cut and describe the remainder as untested rather than projecting from three items to twenty. MidState’s six fleet leases and 23 large manual entries were never candidates for a sample. The same logic governs the top stratum of any population: a known misstatement in a 100 percent stratum is added to the projected misstatement from the sampled stratum in full, with no projection and no allowance, because there is no sampling risk in a stratum you examined completely.
Estimates. An allowance, a reserve or a model-based accrual has no population of items to sample. The substantive test is of the estimate’s three parts: the inputs (are the aging buckets right, is the obsolescence list complete), the method (does the model do what the policy says, and does it match what the same model produced last year), and the outcome, a retrospective review comparing last year’s estimate with what actually happened. The MidState inventory reserve is an example: the count produced $218,000 of out-of-code and damaged product against a $140,000 reserve, a number the reserve has to answer to in a way that no vouching of the reserve calculation could produce. The management review controls guide covers the control side of estimates; the substantive side is the retrospective review.
Interim testing and roll-forward. Testing at March to save time in July is legitimate only with a plan for the gap: a procedure that covers April to June (a substantive analytic on the activity, a review of the intervening large items), or a year-end sample sized on its own. A cutoff test in particular cannot be rolled forward, because its population is defined by the period-end date; the receiving reports that matter for June 30 did not exist in March.
Non-financial objects and entities off the system. The seven decisions apply unchanged when the “balance” is a set of DOT hours-of-service records, keg deposits or route settlements: define the population and its completeness (every route-day, not every settlement that synced), fix the threshold in the unit that matters, select, define the exception, evaluate, conclude. For the two acquired distributors not yet on the ERP, the frame comes from their legacy ledgers, the consolidation spreadsheet is itself an item to test, and the top stratum is 100 percent because the populations are small and the data unfamiliar. Walk the process before designing any of it; the process walkthroughs guide is where the population definition actually gets written, because the walkthrough is where you learn that the Toledo accrual form is never submitted and that the scanning queue exists.
Pressure to move on. Every failed substantive test produces a version of the same conversation, and it helps to have the sentence ready. To the CFO: “The two invoices are $9,370. The sample says the population is carrying about $105,000 and could be carrying $266,000; a four-day sweep by AP will replace both estimates with a fact, and the auditors will run the same test in September on whatever number we leave them.” To the audit committee: “The adjustment was $214,700, identified because a 40-item sample projected to $104,932 against a $125,000 threshold, which is why we did not accept management’s initial correction of the two items found.” Both sentences contain the projection, the threshold and the outcome, and neither contains the word “reasonable.” That is the whole discipline: a substantive test that holds up is one whose numbers can be said aloud to the people who have to act on them, in the order they were decided, without anyone having to take your word for it.
Related guides
- Audit evidence — the hierarchy of evidence weight behind the fourth column of the assertion table.
- Test of design versus operating effectiveness — the control side of the same engagement and the conclusions that belong to it.
- Audit sample sizes: 25, 40 and 60 — attribute sizes for tests of controls, and why they do not transfer to value tests.
- Sampling techniques for high-risk areas — stratification, monetary-unit selection and the trade-offs between methods.
- IPE testing — proving the completeness and accuracy of the report you sample from.
- Journal entry testing — substantive procedures for manual entries, including the close population.
- Journal entry analytics — full-population scans and the follow-up sample that makes them evidence.
- How to audit revenue recognition — occurrence, cutoff and pricing procedures for the revenue side.
- Process walkthroughs — where the population and exception definitions are first discovered.
- Tools — the free mySampler and RCM workbench, alongside the full list of guides and the topic index.
- Monetary Unit Sampling (MUS): Size, Selection, Evaluation — sample sizes, selection and evaluation worked through.
- Audit Fieldwork Explained: The Week-by-Week Guide — from kickoff to closing, week by week.
Leave a Reply