Nicole Forsgren, Jez Humble and Gene Kim’s Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations is the research behind the delivery measures now widely known as the DORA metrics. Published by IT Revolution in March 2018, it reports four years of survey research, conducted with Puppet for the annual State of DevOps reports, and argues three things: that software delivery performance can be measured with four metrics, two for speed and two for stability; that the best performers do better on both at once rather than trading one against the other; and that a defined set of 24 capabilities, technical, architectural, managerial and cultural, predicts that performance. For internal auditors the book matters for a reason that goes beyond technology management. Its finding on change approval, that approval by an external body such as a change advisory board was associated with slower delivery and no fewer failures, goes to the centre of how IT general controls over change are designed and tested.
This guide summarises the book without reproducing it, sets out its structure from the published contents, explains the four metrics and the capabilities as the book and the State of DevOps reports state them, follows the metrics through their changes up to DORA’s 2026 definitions, maps the findings to IT audit and change management, and ends with a worked example of a food manufacturer’s one-person audit function and its co-source IT auditors reviewing a proposal to replace a change advisory board with peer review.
In this guide
- The book in one page
- Who the authors are, and how the book came about
- The structure: three parts and sixteen chapters
- Measuring delivery: the four key metrics
- The 24 capabilities and the culture model
- Change approval: the finding auditors most need to read
- How the DORA metrics have changed since 2018
- The nine ideas worth keeping
- The vocabulary: fourteen terms defined for practitioners
- What it means for internal audit: an IT audit map
- The criticisms, and what the book leaves out
- Editions, reading time and how to read it
- A one-hour discussion guide for an audit team
- Worked example: Brightwater Foods reviews its change controls
- Frequently asked questions
- Related guides
The book in one page
| Question | The answer, in our words |
|---|---|
| What is it? | A research report written for managers: the findings of the State of DevOps survey programme on what drives software delivery performance, with the methods explained. IT Revolution published it on 27 March 2018. |
| What does it measure? | Software delivery performance, through four metrics: delivery lead time and deployment frequency for throughput, and time to restore service and change fail rate for stability. The measures were chosen because they describe outcomes for the whole system rather than the output of one team. |
| What did it find? | That high performers are better on all four measures at once, so speed and stability are not a trade-off; that delivery performance predicts organisational performance; and that 24 capabilities in five groups, from version control and test automation to a generative culture and transformational leadership, predict delivery performance. |
| What is the finding for controls? | That approval of changes by an external body, such as a manager or a change advisory board, was associated with longer lead times, less frequent deployment and slower recovery, and not with lower change failure. The book recommends lightweight approval, such as peer review, instead. |
| How was the research done? | By surveys of technology professionals, analysed with psychometric methods that test whether survey questions measure what they claim to. The book’s second part explains why the authors think surveys can be trusted for this purpose. |
| What happened next? | The research programme, DORA, became part of Google Cloud; its annual reports added new metrics, renamed one and, in 2025, changed their title and focus to AI-assisted software development. The four metrics, now five, are widely known as the DORA metrics. |
| Why read it as an auditor? | Because it is evidence about which change controls work, because its metrics are a ready-made set of key risk indicators for technology delivery, and because development teams may cite it when they ask audit to accept peer review in place of a change board. |
Who the authors are, and how the book came about
Nicole Forsgren holds a doctorate in management information systems and led the research; the author note on the book’s Google Books record describes her as the lead investigator on the DevOps studies behind it and notes that her research has been funded by bodies including NASA and the National Science Foundation. She is now a Partner at Microsoft, where she leads the Developer Experience Lab, sits on the board of ACM Queue and is a co-author of the second edition of The DevOps Handbook; her later book, written with Abi Noda, is Frictionless: 7 Steps to Remove Barriers, Unlock Value, and Outpace Your Competition in the AI Era. Jez Humble, at the time of publication, was a co-author of Continuous Delivery, Lean Enterprise and The DevOps Handbook, had worked for the US federal government at 18F, was DORA’s chief technology officer and taught at the University of California, Berkeley. Gene Kim, who has studied high-performing technology organisations since 1999, founded the software company Tripwire and was its chief technology officer for thirteen years; he co-wrote The Phoenix Project and The DevOps Handbook, his other books include The Unicorn Project, Wiring the Winning Organization and Vibe Coding, the last two written with Steven J. Spear and Steve Yegge respectively, and he has organised the DevOps Enterprise Summit, now the Enterprise Technology Leadership Summit, since 2014.
The book grew out of the State of DevOps research, which DORA (DevOps Research and Assessment) ran in partnership with Puppet before 2018, as an extension of Puppet’s original State of DevOps surveys. It draws on four years of that work, the reports for 2014 to 2017, and, according to the Shingo Institute’s citation, on data from more than 23,000 technology professionals. Martin Fowler and Courtney Kissler wrote forewords, and the final part, a case study of the Dutch bank ING, was written by Steve Bell and Karen Whitley Bell. The book won a Shingo Publication Award; the Shingo Institute’s announcement describes it as a research-based book and case study. In an interview with InfoQ in June 2018, Humble described the research as being at the beginning of a journey and invited others to test and extend it.
The structure: three parts and sixteen chapters
| Part | Chapters | What it contributes |
|---|---|---|
| Front matter | Forewords by Martin Fowler and by Courtney Kissler; a quick reference to the capabilities that drive improvement; preface | The capability list at a glance |
| Part One: What We Found | 1 Accelerate; 2 Measuring Performance; 3 Measuring and Changing Culture; 4 Technical Practices; 5 Architecture; 6 Integrating Infosec into the Delivery Lifecycle; 7 Management Practices for Software; 8 Product Development; 9 Making Work Sustainable; 10 Employee Satisfaction, Identity, and Engagement; 11 Leaders and Managers | The findings: the metrics, the capabilities and what they predict |
| Part Two: The Research | 12 The Science Behind This Book; 13 Introduction to Psychometrics; 14 Why Use a Survey; 15 The Data for the Project | The methods, and the case that surveys can measure these things |
| Part Three: Transformation | 16 High-Performance Leadership and Management, by Steve Bell and Karen Whitley Bell | A case study of ING in the Netherlands |
| Back matter | Conclusion; Appendix A Capabilities to Drive Improvement; Appendix B The Stats; Appendix C Statistical Methods Used in Our Research; bibliography; index | The capability definitions and the statistics |
The contents are those of the O’Reilly catalogue record. Part One holds the findings, and chapter 2, on measuring performance, and chapter 7, on management practices, are the two an auditor should read most carefully. Part Two is unusual in a management book: four chapters explaining survey design, the psychometric tests that check whether a set of questions measures a single underlying idea, and the data. An auditor who is asked to rely on a survey, or on a culture assessment built from one, will find it a useful introduction. Part Three changes voice, from the research team to two guest authors describing one bank’s reorganisation.
Measuring delivery: the four key metrics
The book starts from the problem that measures of software work have usually been poor. Measures of output reward busy work, and measures set separately for developers and for operations staff, throughput for one and stability for the other, set the two against each other. A good measure, the authors argue, should describe a global outcome, so that teams are not rewarded for pushing problems onto each other, and it should measure outcomes rather than output. The four metrics they chose meet both tests. The table gives each metric as the book describes it, what it shows, and where an auditor would find the evidence.
| Metric | What it measures | What it tells you | Where an auditor finds the evidence |
|---|---|---|---|
| Delivery lead time | The time taken to implement, test and deliver a change; the book distinguishes this delivery part from the whole lead time from a customer’s request to its fulfilment | How quickly the organisation can respond, including with a fix | Version control and deployment logs: commit time against production deployment time |
| Deployment frequency | How often the organisation deploys to production, used as a proxy for batch size, which is hard to measure directly | Smaller, more frequent batches mean less risk per change and faster feedback | Deployment pipeline records |
| Time to restore service | How long it takes to restore service after an incident, on the view that failure in complex systems is inevitable | Resilience: how quickly mistakes are contained | Incident tickets and monitoring alerts |
| Change fail rate | The percentage of changes to production that degrade service or need remediation, such as a hotfix, rollback or patch | The quality of what is released | Change records linked to incidents, rollbacks and hotfixes |
The first two measure throughput and the second two stability, a grouping the research programme had adopted by 2015. The central finding is that the groups do not trade off: the high performers in the data were faster and more stable at the same time. The 2017 State of DevOps Report, the last year of data the book draws on, put numbers on the gap. Its comparison is set out below, as the report states it.
| Measure (2017 report) | High performers | Low performers | Gap reported |
|---|---|---|---|
| Deployment frequency | On demand, multiple deploys a day | Between once a week and once a month | High performers deployed 46 times more often |
| Lead time for changes | Less than one hour | Between one week and one month | 440 times faster from commit to deploy |
| Mean time to recover | Less than one hour | Between one day and one week | 96 times faster |
| Change failure rate | 0 to 15 percent | 31 to 45 percent | Five times lower |
The report also restated the programme’s 2014 finding that high performers were twice as likely to exceed their own goals for profitability, market share and productivity, which is the basis for the book’s claim that delivery performance matters to the organisation and not only to the technology function. Two cautions apply to reading the table. The clusters are statistical groupings of each year’s survey respondents, so their boundaries can change from one report to the next; and the 2017 report itself noted that its low performers’ averages for deployment frequency and lead time were worse than the medium group’s although their medians were the same. The numbers describe the population that answered the survey, not a benchmark any particular organisation must meet.
The 24 capabilities and the culture model
The book argues for capability models rather than maturity models. A maturity model, in the authors’ account, implies an end point that an organisation can declare it has reached, assumes every team progresses through the same stages, tends to measure tools or technical proficiency rather than outcomes, and is static while technology changes. A capability model instead identifies the practices that predict better outcomes and lets each team improve the ones that matter most in its context. The 24 capabilities fall into five groups, which the table sets out with what each group means for an audit.
| Group | Capabilities | What it means for an audit |
|---|---|---|
| Continuous delivery (8) | Version control for all production artefacts; deployment automation; continuous integration; trunk-based development; test automation; test data management; shifting security left; continuous delivery | These are control activities: version control is the change log, automated tests are preventive controls, and the pipeline enforces the process |
| Architecture (2) | A loosely coupled architecture; an architecture for empowered teams, who choose their own tools | Smaller blast radius for each change; scoping of which systems a change can affect |
| Product and process (4) | Gathering and implementing customer feedback; making the flow of work visible through the value stream; working in small batches; team experimentation | Batch size as a risk factor; the link to the Lean Startup summary‘s small-batch argument |
| Lean management and monitoring (5) | Lightweight change approval; monitoring of applications and infrastructure to inform decisions; checking system health proactively; limits on work in progress; visualising work | Change approval design, monitoring and alerting: the territory of IT general controls and incident response |
| Culture (5) | A generative culture; supporting learning; collaboration among teams; resources and tools that make work meaningful; transformational leadership | Culture as a predictor of performance and of whether problems are reported; see the culture audit guide |
The culture measure comes from Ron Westrum, who studied how organisations in high-risk fields such as aviation and healthcare handle information. Westrum’s typology has three types. In a pathological, power-oriented organisation, cooperation is low and messengers are punished; in a bureaucratic, rule-oriented one, cooperation is modest and bad news is ignored; in a generative, performance-oriented one, cooperation is high, risks are shared and failure leads to inquiry rather than blame. DORA measures it with survey statements scored on a seven-point scale and averaged. The book found a generative culture associated with better delivery and organisational performance, and argued that culture can be changed by changing practices, such as those of continuous delivery and lean management. For an audit function the typology is a usable lens on whether bad news travels, and the Fearless Organization summary covers the related idea of psychological safety, which DORA’s 2019 report found also predicts productivity.
Change approval: the finding auditors most need to read
The finding that matters most to an IT auditor concerns change approval. In the research behind the book, the authors compared approaches to approving changes. Approval by an external body, a manager or a change advisory board outside the team, was negatively associated with lead time, deployment frequency and restore time, and had no association with change fail rate; the book concludes that such approval was worse than having no change approval process at all. Lightweight change approval, such as review by peers within the team, is one of the 24 capabilities the book found to drive better performance; DORA’s later guidance pairs it with automated testing in the deployment pipeline. DORA restated the point in its 2019 report: heavyweight approval had a negative effect on delivery performance, and the research found no evidence that formal external review lowers change fail rates.
A regulator has since looked at the same question with different data. In February 2021 the UK Financial Conduct Authority published a review of more than a million production changes made in 2019 by 23 financial services firms. About 1.6 percent of changes caused an incident. Change advisory boards approved more than 90 percent of the major changes they reviewed, and some firms’ boards had not rejected a single change in the year, leading the FCA to suggest that many boards acted more as a flight-control function than as assurance. Firms releasing smaller, more frequent changes had higher success rates, and firms whose governance arrangements had been in place for more than a year had a lower proportion of change-related incidents; firms using DevOps showed a slightly higher failure rate, though the FCA said the sample was too small to draw firm conclusions.
| Question in a change management audit | What the book and the later evidence suggest | What the auditor tests |
|---|---|---|
| Is every production change authorised? | Authorisation can be given by a peer reviewer in the development platform rather than by a board | That approval is recorded for every change, by someone other than the author, and that the platform prevents bypass |
| Is there segregation of duties? | DORA notes that peer review can meet the common regulatory requirement that someone other than the author approves a change | Branch protection, administrator overrides and who can deploy; see the segregation of duties guide |
| Are changes tested? | Automated tests in the pipeline are a preventive control applied to every change | Pipeline configuration, test coverage of critical functions, and whether failing tests block deployment |
| Does the change board add assurance? | External approval was associated with slower delivery and no fewer failures; the FCA found boards that rejected nothing | Rejection rates, time spent per change, and whether the board reviews the risky changes or all of them |
| Are emergency changes controlled? | A process fast enough for normal changes needs no separate emergency route | The volume of emergency changes and whether each was reviewed after the event |
| Is privileged access controlled? | Changes made outside the pipeline bypass every control in it | Direct production access, break-glass use and its review; see the privileged access guide |
None of this means that internal audit should recommend abolishing change boards. The finding concerns external approval as the main control over routine changes; DORA’s own guidance gives the board a different job in a continuous delivery organisation, coordinating across teams and deciding business trade-offs, and recommends extra scrutiny for high-risk changes. Principle 11 of COSO’s 2013 Internal Control framework asks organisations to select and develop general controls over technology; it states a principle rather than prescribing a mechanism such as a board. The objective of a change control is that changes are authorised, tested and implemented by appropriate people, and how that is achieved is a design choice. The change management audit guide, the SDLC and DevOps pipeline audit guide and the guide to testing automated controls set out how to test a pipeline as a control.
How the DORA metrics have changed since 2018
The research continued after the book under DORA, which is now a programme run by Google Cloud with its research led by a team at Google. Its definitions have changed several times, and an auditor reviewing a technology dashboard should know which version is in use. The table follows the changes as DORA itself records them.
| Year | Change | What it means in practice |
|---|---|---|
| 2014 | The first study tested four candidate measures but used three, because change fail rate did not fit the statistical model that year | The four-metric set was not fixed from the start |
| 2015 | Measures grouped into throughput (deployment frequency, lead time) and stability (time to restore, change fail rate); high performers strong on both | The basis of the book’s no-trade-off finding |
| 2018 | Availability added as a measure of operational performance; the umbrella term became software delivery and operational performance; an elite cluster defined | Delivery measures joined by an operations measure |
| 2019 | Heavyweight change approval reported to hurt performance; psychological safety reported to predict productivity | The change approval finding restated |
| 2021 | Availability broadened to reliability, covering latency, performance and scalability | Operational measure widened |
| 2023 | Mean time to restore renamed and redefined as failed deployment recovery time, counting only failures caused by changes to production | Outages from external causes no longer count |
| 2024 | Deployment rework rate added as a fifth metric; metrics regrouped into throughput and instability | The four keys became five |
| 2025 | Annual report renamed from Accelerate State of DevOps to State of AI-assisted Software Development; AI found to act as an amplifier, improving throughput but often at the cost of stability where foundations are weak; an AI Capabilities Model of seven capabilities published in December | The research focus moved to AI |
DORA’s current guide, updated in January 2026, defines five metrics. Throughput is measured by change lead time, from a change being committed to version control to its running in production; deployment frequency; and failed deployment recovery time, the time taken to recover from a deployment that needs immediate intervention. Instability is measured by change fail rate, the share of deployments that need immediate intervention such as a rollback or hotfix, and deployment rework rate, the share of deployments that are unplanned and made because of an incident in production. Two differences from the book matter for an auditor. Recovery time now counts only failures caused by deployments, so a dashboard built on the old definition and one built on the new will not agree; and failed deployment recovery time has moved from the stability group into throughput. Any key risk indicator built on these measures should state which definition it uses; the KRI examples guide shows the thresholds and owners such an indicator needs.
One naming clash needs care. DORA, the research programme, has nothing to do with the EU’s Digital Operational Resilience Act, Regulation (EU) 2022/2554, also known as DORA, which has applied to EU financial entities since January 2025 and covers ICT risk management, incident reporting and resilience testing. An audit report that cites DORA metrics for an EU bank should say which one it means; the guide to the EU regulation covers the other DORA.
The nine ideas worth keeping
- Measure outcomes, not output. Output measures reward activity; lead time, deployment frequency, recovery time and change failure describe what the system delivers.
- Measure the whole system. Separate targets for developers and operations staff set them against each other; global measures make them share the result.
- Speed and stability are not a trade-off. In the research, the high performers were both faster and more stable than the rest.
- Batch size is a risk. Deployment frequency stands in for batch size; small, frequent changes are easier to test, understand and reverse.
- Capabilities, not maturity levels. There is no end state; there are practices that predict better outcomes, to be improved where they matter most.
- External approval is a weak control. Approval by a body outside the team was associated with slower delivery and no fewer failures; lightweight approval by peers was associated with better performance.
- Culture can be measured and changed. Westrum’s typology turns how organisations handle information into survey scores, and changing practices changes culture.
- Security belongs in the pipeline. Building security into tools and early design beats reviewing it at the end.
- Deployment pain is a signal. Fear of deploying and burnout were lower where delivery was automated and done in small batches, and both are signs of control risk as well as of poor working conditions.
The vocabulary: fourteen terms defined for practitioners
| Term | What the authors mean | What it looks like in an audit |
|---|---|---|
| Software delivery performance | The combination of throughput and stability measured by the four metrics | A set of key risk indicators for technology change |
| Delivery lead time | The time to implement, test and deliver a change | Commit-to-production times from version control and pipeline logs |
| Deployment frequency | How often code is deployed to production; a proxy for batch size | Deployment counts by system and period |
| Time to restore service | How long it takes to restore service after an incident | Incident duration; now failed deployment recovery time |
| Change fail rate | The share of changes that degrade service or need remediation | Changes linked to incidents, rollbacks and hotfixes |
| Throughput and stability | The two groups of measures; now throughput and instability | Balanced indicators rather than speed alone |
| Capability model | A set of practices that predict outcomes, improved continuously | An assessment against capabilities rather than a maturity level |
| Maturity model | Staged levels with an end point, which the authors reject | A maturity score in a management report, to be read with caution |
| Westrum typology | Pathological, bureaucratic and generative cultures, by how information flows | A lens on whether bad news reaches decision-makers |
| Lightweight change approval | Peer review within the team, supported by automation | Recorded approvals in the development platform |
| Continuous delivery | Keeping software deployable at all times through automation and small changes | The pipeline as a control system |
| Trunk-based development | Developers merging small changes into a shared main branch frequently | Short-lived branches and frequent, reviewable merges |
| Loosely coupled architecture | Systems that teams can test and deploy independently | A smaller scope of impact for each change |
| Deployment pain | The fear and disruption associated with releasing changes | Weekend release windows and large, risky releases |
What it means for internal audit: an IT audit map
The book is written for technology leaders, but its findings change several standard IT audit procedures. The table maps them. The ITGC primer for non-IT auditors, the guide to the dependency between ITGCs and application controls and the IT audit plan guide describe the surrounding programme.
| Audit activity | What the book implies | Procedure it changes |
|---|---|---|
| Change management ITGC | The control objective can be met by peer review and an enforced pipeline | Test the pipeline as the control, using the questions in the change approval table above, and reconcile production changes to pipeline deployments |
| IT audit planning | Delivery measures show where change risk concentrates | Use change fail rate and emergency change volume by system to help rank systems in the IT risk assessment |
| Reliance by the external auditor | A redesigned control has to be understood before it replaces the old one | Agree the testing approach for a new change control with the external auditor before it takes effect |
| Transformation programmes | Capability models measure outcomes; maturity scores can measure activity | Ask whether a DevOps or cloud programme reports the delivery measures or only a maturity level |
| Technology KPIs and KRIs | The four metrics are outcome measures | Use them as indicators, with definitions stated and data traced to logs; see the analytics evidence guide |
| Incident management | Recovery time is a delivery measure | Link incidents to the changes that caused them; see the incident response audit guide |
| Third-party and SaaS change | The same questions apply to changes the organisation does not make | Test how vendor releases are notified, tested and approved |
| Culture and risk reporting | Generative cultures surface problems | Ask how failed changes are reviewed and whether the reviews look for causes rather than culprits |
The map has a limit that auditors should state. The book’s evidence is about software delivery performance, not about the financial statement assertions or the regulatory obligations that change controls also serve. A pipeline that deploys quickly and rarely fails can still deploy an unauthorised change to a pricing table. The auditor’s job is to test that the control objective, authorised, tested and appropriately implemented change, is met, and the book’s contribution is to show that a board is not the only, or necessarily the best, way to meet it. The SOX ITGC scoping guide covers the systems where the financial reporting stakes apply.
The criticisms, and what the book leaves out
The main criticism concerns the research method, and the authors anticipated it. The data come from surveys rather than from system logs, and the analysis shows association rather than proving cause. In the InfoQ interview the authors explained that randomised experiments were impractical, so they used a quasi-experimental design with validated survey measures and statistical controls. A critique by the software engineering manager Junade Ali, published in January 2024, went further: DORA does not publish its raw data; self-reported surveys may be biased by respondents who feel good about their work answering positively about both speed and stability; the link between speed and reliability is correlational; the outcome measures emphasise speed over the things users and engineers care about most, such as data security and accuracy; and the reports have been produced first for Puppet and then for Google Cloud, both of which benefit when developers deploy faster.
Three further points are this guide’s own. First, the metrics are easily turned into targets, and a target invites gaming: change fail rate falls if hotfixes are not counted, and deployment frequency rises if changes are split. An auditor reviewing a dashboard of DORA metrics should test the definitions and the data as for any key indicator. Second, the research describes software teams, and its findings transfer least well to regulated systems where a change has external consequences, such as payments, pricing or safety; the FCA’s review found DevOps firms with a slightly higher failure rate, while warning that its sample was too small to conclude anything. Third, the change approval finding is easy to misread as a case against any approval at all. The book’s argument is for lightweight approval, not none, and DORA’s later guidance keeps a role for review of high-risk changes. None of these undermines the book’s value for auditors, which is to put evidence where there had mostly been habit.
Editions, reading time and how to read it
| What | Details | Best for |
|---|---|---|
| Paperback | IT Revolution, 27 March 2018, 288 pages, ISBN 9781942788331 | The standard edition |
| Ebook and audiobook | Ebook and Kindle editions, and an audiobook, from the same publisher | Reading on the move; the tables and figures are easier in print |
| Catalogue variant | Some records give the subtitle as The Science Behind DevOps: Building and Scaling High Performing Technology Organizations | Finding the book in a library |
| Recognition | Shingo Publication Award; the Shingo Institute described it as a research-based book and case study | Context for the book’s standing in lean circles |
| Reading plan | Chapters 1 and 2 for the metrics; chapter 7 for management practices; Appendix A for the capabilities; skim Part Two unless relying on survey evidence | An IT auditor with one long evening |
| Keeping current | DORA’s reports and its guide to the metrics, published free at dora.dev | Checking the definitions in use today |
| Companion reading | The Phoenix Project summary for the novel Gene Kim co-wrote about the same problems; the Field Guide to Understanding Human Error summary for blameless review of failed changes | Placing the research |
A one-hour discussion guide for an audit team
- Ten minutes: the four numbers. Ask whether the technology function can state its lead time, deployment frequency, recovery time and change fail rate for its most important system, and from what data.
- Fifteen minutes: the change board. Take the last quarter’s change advisory board minutes. Count the changes approved, rejected and deferred, and estimate the minutes spent per change.
- Ten minutes: the bypass. Discuss how a change could reach production without passing through the approved route, and whether the function has ever tested for one.
- Fifteen minutes: the peer-review proposal. Suppose development asks to replace board approval with peer review for routine changes. List the evidence the function would need to accept it.
- Ten minutes: the plan. Decide whether a review of change controls and delivery metrics belongs in the next plan, and how co-source IT specialists would be used; see the co-sourcing provider guide.
Worked example: Brightwater Foods reviews its change controls
Brightwater Foods is a fictional US food manufacturer with $180 million of revenue, three plants, a single internal auditor and a co-source firm that provides its IT audit work. Its customer ordering portal and the integration that sends orders and prices to the ERP system are built in-house by a development team of five, which moved to an automated deployment pipeline in 2026. Every change still went to a weekly change advisory board chaired by the IT director. In late 2026 the IT director proposed that routine portal changes be approved by peer review in the pipeline, with the board reviewing only high-risk changes. The chief financial officer asked internal audit for a view before agreeing, because the company’s external auditors rely on the IT general controls over systems that feed the financial statements. The internal auditor scoped a review for the first quarter of 2027, and the co-source firm’s IT auditors measured the four metrics from the 2026 pipeline, version control and incident data before testing the controls.
| Test | What the review found | Result |
|---|---|---|
| Is every production change traceable to the pipeline? | 412 production changes in 2026: 403 deployed through the pipeline and 9 made directly in the production database by the lead developer, who held administrator rights | Finding: direct changes stopped; a break-glass account with logging and next-day review created |
| Is every pipeline change approved by someone other than its author? | 389 of the 403 pipeline deployments had a recorded approval by a second developer; 14 were merged by their authors using an administrator override of the branch protection | Finding: override removed for all developers; overrides now require the IT director and are reported monthly |
| Do failures differ by route? | 26 changes (6.3 percent of 412) caused incidents needing a rollback or hotfix: 24 of the 403 pipeline changes (6.0 percent) and 2 of the 9 direct changes (22 percent). One direct change loaded a price file with an error that under-billed customers until it was corrected three days later | Observation: the direct route was the riskiest, not the pipeline |
| What does the board add? | The board approved every change it saw in 2026 and rejected none; its minutes showed about two minutes per change; waiting for the weekly meeting accounted for most of the median lead time of just over four days | Observation: the board added delay without visible challenge |
| Are the metrics reported correctly? | The IT dashboard reported a change fail rate of 2.4 percent because hotfixes were not counted as failures | Finding: DORA definitions adopted and documented; the dashboard reperformed by the co-source firm |
| Are high-risk changes identified? | No criteria distinguished routine portal changes from changes to pricing logic or the ERP integration | Finding: risk criteria agreed with finance; high-risk changes stay with the board, with a test plan reviewed by the finance systems lead |
The review supported the proposal, with conditions. The internal auditor’s report to the audit committee concluded that peer review in an enforced pipeline could meet the change control objective for routine portal changes, but only once the administrator override and the direct database route were closed, because those, rather than the absence of a board, were where the control had failed in 2026. High-risk changes, defined as those touching pricing logic, the ERP integration or security configuration, would continue to need board approval and a documented test plan. The chief financial officer shared the revised design with the external auditors before it took effect, and they planned to test the peer-review evidence and the pipeline configuration in their next cycle.
The arrangements took effect in April 2027. By the third quarter the median lead time for routine changes had fallen to about one day, there had been no self-approved merges and no direct database changes outside the break-glass process, and the board was spending its meeting on the few changes that could affect what customers were charged. The internal auditor’s contribution was to frame the question as one about the control objective rather than the control’s form, and the co-source firm’s was the technical testing that showed where the real weaknesses were. The example is fictional; the evidence it draws on, about external approval, batch size and the routes around a control, is the evidence this summary sets out.
Frequently asked questions
Does Accelerate say change advisory boards should be abolished?
No. It reports that approval by a body outside the team was associated with slower delivery and no reduction in failures, and recommends lightweight approval such as peer review supported by automated testing. DORA’s later guidance keeps a role for review of high-risk changes and for coordination across teams.
What are the four key metrics?
Delivery lead time, deployment frequency, time to restore service and change fail rate. DORA has since renamed time to restore as failed deployment recovery time, added deployment rework rate as a fifth metric and grouped the five into throughput and instability.
Can peer review satisfy segregation of duties for SOX or other ITGC testing?
It can meet the objective that someone other than the author approves a change, provided the platform enforces it and the approvals are recorded; DORA makes the same point. The auditor must test the enforcement, especially administrator overrides and direct access, and agree the approach with the external auditor where they rely on the control; see the guide to manual, automated and IT-dependent controls.
Is DORA the same as the EU’s Digital Operational Resilience Act?
No. DORA in the book is DevOps Research and Assessment, the research programme now run by Google Cloud. The EU’s Digital Operational Resilience Act, Regulation (EU) 2022/2554, is a financial-sector regulation that has applied since January 2025; the guide to the regulation covers it.
How reliable is the research?
It rests on large surveys analysed with psychometric methods, and the authors describe it as quasi-experimental rather than proof of cause. Critics point to self-reporting, the correlational design and the absence of published raw data. Auditors should treat the findings as strong evidence about patterns, and test their own organisation’s data before relying on them.
What should an auditor read alongside it?
The Phoenix Project summary, for the novel Gene Kim co-wrote about the same problems, and DORA’s current guide to the metrics; the internal audit reading list sets out the wider library by theme.
Related guides
- The Phoenix Project: Summary and What It Means for IT Auditors — the novel behind the same movement
- How to Audit IT Change Management: From Ticket to Production — testing change controls
- How to Audit the SDLC and DevOps Pipeline — the pipeline as a control system
- IT General Controls (ITGC) Audit: The Complete Primer for Non-IT Auditors — where change management sits among the ITGCs
- Segregation of Duties Beyond the ERP: A Universal SoD Framework — separating authoring from approving
- How to Audit Privileged Access: Admin Rights, Break-Glass, and Vaulting — the routes around a pipeline
- The Best Books for Internal Auditors — where this book sits among the others
- All book summaries — the full set of summaries for auditors
Leave a Reply