The change management control that internal auditors learned to test, a ticket, a developer, a separate tester, a change advisory board and a migration performed by someone who could not write code, describes almost no modern delivery team. Code moves from a laptop to production through a pipeline, sometimes dozens of times a day, the developer who wrote it may be the person who deploys it, and the approvals live in a pull request rather than a form. None of this removed the control objectives. Changes still have to be authorized, tested, approved by someone other than the author, deployed only by a controlled mechanism, and traceable from the production state back to a request. What changed is where the evidence of each objective lives, and an auditor who asks for the old evidence will be told, accurately, that it does not exist, and will conclude, inaccurately, that the controls do not either. This guide translates ITGC intent into pipeline reality: what to test at each stage, how separation of duties works when the developer deploys, where the populations come from, and how to run the engagement, with a test program, a pipeline control inventory template and a worked example at a bank’s digital channels team.
In this guide
- The control objectives did not change; the evidence did
- The standards that describe the modern SDLC
- The pipeline as a control system, stage by stage
- Separation of duties when the developer deploys
- Requirements and approval gates in agile delivery
- Secrets, credentials and the pipeline’s own identity
- Test coverage and security testing as evidence
- Populations, reconciliation and sampling
- The test program
- The pipeline control inventory template
- Worked example: Lakeshore Bancorp’s digital channels pipeline
- Common findings and how to rate them
- Related guides
The control objectives did not change; the evidence did
Every program change control in a traditional ITGC framework serves one of five objectives, and the pipeline serves the same five with different mechanisms. The table pairs them, because the pairing is the whole method: the auditor tests the objective through whichever mechanism the organization actually uses, and rates a gap against the objective rather than against the absence of the familiar form. The change management audit guide covers the traditional mechanisms and the population work in depth; this guide assumes it and moves to the pipeline.
| Control objective | Traditional mechanism | Pipeline mechanism | Evidence to request |
|---|---|---|---|
| Changes are authorized before work starts | Change request approved by the business owner | Work item in the backlog accepted into a sprint or prioritized by the product owner; branch created from the work item | Work item history with state transitions and the product owner’s acceptance; branch-to-work-item linkage |
| Changes are tested before release | Separate test team, test plan, signed test results, UAT sign-off | Automated unit, integration and security tests run by the pipeline on every commit; coverage and quality gates that block the merge; acceptance criteria verified on the work item | Pipeline run logs with gate results; coverage reports; gate configuration; acceptance criteria and their verification on the work item |
| Changes are approved by someone other than the author | CAB or manager approval on the change form | Pull request review required by branch protection, with self-approval disabled and the number and identity of reviewers enforced by the platform | Branch protection configuration; PR history showing reviewers and approval timestamps; platform audit log of protection changes |
| Only controlled mechanisms deploy to production | Migration performed by operations staff without development access | Deployment executed only by the pipeline’s identity from an approved artifact, with human deploy rights removed or restricted to a break-glass path | Production access lists; deployment tool permissions; cloud or platform audit logs showing the deploying identity; break-glass log |
| Production state is traceable to approved changes | Change log reconciled to tickets; periodic comparison of production to a baseline | Immutable, signed artifacts with provenance; deployment log reconciled to merged PRs and work items; drift detection on infrastructure as code | Artifact registry with signatures and provenance attestations; deployment-to-PR reconciliation; drift reports |
The translation has a corollary that decides the engagement’s approach: in a pipeline, most of the controls are configurations rather than performances. Branch protection, required reviewers, quality gates, deployment permissions and artifact signing are set once and enforced by the platform on every change, which makes them automated controls, tested as the automated controls testing guide describes: confirm the configuration, test one or a few instances to prove it operates, and rely on the general controls over the platform for the period. The manual elements that remain, the reviewer’s judgment in a PR and the product owner’s acceptance, are tested as manual controls with samples. Classifying each pipeline control correctly is the first task of fieldwork and the main determinant of the budget.
The standards that describe the modern SDLC
Three references supply criteria. NIST’s Secure Software Development Framework, SP 800-218, organizes secure development into four practice groups, Prepare the Organization, Protect the Software, Produce Well-Secured Software and Respond to Vulnerabilities, each with practices, tasks and implementation examples; version 1.1 of February 2022 remains the final version, and version 1.2 was released as a draft (SP 800-218 Rev. 1, initial public draft) in December 2025 following Executive Order 14306 of June 2025, with comments closed at the end of January 2026 and finalization still pending when this guide was written. The SSDF is written for producers of software, which includes every organization with an in-house development team, and its Protect the Software group is the closest published description of pipeline integrity. SLSA, the Supply-chain Levels for Software Artifacts specification (version 1.2 is the current approved version), defines a Build track with levels L0 to L3 that describe how trustworthy an artifact’s provenance is, from none, through provenance that exists, to provenance generated and signed by the build platform, to a hardened build platform that prevents tampering during the build, and reintroduced a Source track for the integrity of the source repository. ISO/IEC 27001:2022 Annex A carries the development controls in A.8.25 to A.8.33: the secure development life cycle, application security requirements, secure architecture and engineering principles, secure coding, security testing in development and acceptance, outsourced development, separation of development, test and production environments, change management and the protection of test information. COBIT 2019 places the same ground in BAI03 (managed solutions identification and build), BAI06 and BAI07, and NIST SP 800-53 Rev. 5 in the SA and CM families, in particular CM-3 configuration change control, CM-5 access restrictions for change and SA-11 developer testing and evaluation.
For a SOX-scoped application the criteria are narrower and older: the program change ITGC has to give assurance that changes affecting financial reporting were authorized, tested and approved before they reached production, and that the migration was controlled. The pipeline satisfies that objective when the mechanisms in the first table operate and the general controls over the platform hold, and the SOX ITGC scoping guide and the ITGC versus application controls guide cover how the dependency is evaluated when they do not.
The pipeline as a control system, stage by stage
A pipeline has six stages, and each has a small number of controls that carry the objectives. The table is the map for the walkthrough: the auditor follows one change through all six with the engineer who owns the pipeline, screen by screen, and records at each stage which control operates, where it is configured and what evidence the platform keeps. The walkthrough is done on the real pipeline, not on the architecture diagram, because the diagram describes the intended pipeline and the audit is of the one that exists.
| Stage | Controls that carry the objectives | What to test | Evidence |
|---|---|---|---|
| Source | Branch protection on the production branch; required reviews with self-approval disabled; status checks required before merge; signed commits or verified authors; restricted force-push and branch deletion | Inspect protection settings on every production repository; test that a self-approved or unreviewed merge is rejected; reconcile repositories to the application inventory to find unprotected ones | Protection configuration export; platform audit log of protection changes; repository inventory; a failed attempt recorded during the walkthrough |
| Build | Builds run only on managed runners from a defined configuration; dependencies resolved from an approved registry with lockfiles; software bill of materials generated; artifact signed with provenance | Inspect runner configuration and who can change it; test that a build from an unapproved source is rejected; verify provenance and signature on a sampled artifact; check SBOM generation | Pipeline definition under version control; runner inventory; registry policy; artifact signature and provenance attestation; SBOM |
| Test | Unit and integration tests required to pass; coverage threshold enforced; static analysis and software composition analysis with severity gates; secret scanning; dynamic testing in a staging environment for exposed services | Inspect gate configuration and exemptions; sample pipeline runs for gate results; test that a failing gate blocks the merge; review the exemption list and who approved it | Gate configuration; run logs; coverage and scan reports; exemption register with approvals and expiry |
| Release | Promotion of an immutable artifact through environments; release approval by a second person or a policy engine for production; change record created automatically with links to PRs and work items | Verify the same artifact digest moves through environments; sample production releases for the approval and the approver’s independence; check the automated change record’s completeness | Release pipeline definition; environment approval configuration; release history with approvers; change records |
| Deploy | Deployment performed by the pipeline’s identity only; human production deploy rights removed; break-glass path with logging and after-the-fact review; infrastructure as code with plan and apply separated and reviewed | List every identity with production deploy rights and justify each; reconcile deployments in the platform’s audit log to pipeline runs; inspect break-glass uses and their reviews; test IaC review and drift detection | Deploy permission lists; cloud and platform audit logs; break-glass log and reviews; IaC repository and drift reports |
| Operate | Feature flags with ownership and expiry; rollback capability tested; monitoring and alerting on production changes; post-incident reviews feeding backlog | Sample feature flags for owner and age; inspect the last rollback; verify production change alerts reach someone who reviews them; trace an incident to a backlog item | Flag inventory; rollback records; alert configuration and handling; incident reviews |
Two stages deserve emphasis because they are where the objectives most often fail silently. The source stage’s protection settings can be changed by repository administrators, and in most organizations that group is large and includes the developers themselves; the general control over the pipeline is therefore the administration of the platform, and an audit that tests branch protection without testing who can switch it off has tested a control that its own subjects can disable. The deploy stage’s “pipeline identity only” rule is defeated by every human who retains production credentials from the pre-pipeline era, and by every long-lived deploy key stored on a laptop. The privileged access work in the privileged access audit guide applies to the pipeline’s administrators and to the platform’s service accounts exactly as it does to domain administrators.
Separation of duties when the developer deploys
Trunk-based development, in which every developer commits to a single main branch that is always deployable, is where auditors most often conclude that segregation of duties has collapsed, and it is where it can be strongest, because the separation has moved from people to mechanisms. The objective is that no single person can take code from their own keyboard into production without an independent check. In the traditional model the check was a different human at migration time; in the pipeline it is a combination of four things, and the audit tests that all four hold. First, the author cannot approve their own pull request, enforced by the platform, with at least one reviewer who is not the author and, for high-risk repositories, a designated code owner. Second, the merge cannot occur until the automated gates pass, so the independent check includes the tests and scans, which the author cannot waive without an approved, visible exemption. Third, the deployment is performed by the pipeline from the merged artifact, so the human who wrote the code never touches production; the developer’s production access is limited to observation, and any emergency path is break-glass with logging and review. Fourth, the people who can change any of the first three, the platform administrators, are few, named, separately monitored and not the same people who deliver most of the changes.
The universal segregation of duties framework describes this as the ladder of alternatives: where full separation is impossible, the organization substitutes preventive mechanisms and detective review, and the auditor tests the substitutes rather than lamenting the absence of the original. The detective layer in a pipeline is the reconciliation of production changes to merged and reviewed PRs, run on a cycle by someone outside the delivery team. If every production deployment in the period traces to a reviewed PR and every reviewed PR to an accepted work item, the segregation objective is met, whoever pressed the button, and the auditor says so in those words. Where the reconciliation shows deployments with no PR, PRs merged by administrators bypassing review, or gates disabled for a period, the objective failed for those changes, and the finding is written against the objective with the count.
Requirements and approval gates in agile delivery
The authorization objective is where auditors most often ask for evidence that agile teams cannot produce and do not need to. There is no requirements document signed before development, and demanding one produces a document written after the fact for the auditor. What exists instead is a backlog of work items with acceptance criteria, a prioritization decision by a product owner who represents the business, and a definition of done that the team applies before an item is closed. The authorization test is therefore whether the product owner is a person the business has actually delegated the decision to, whether the acceptance of an item into a sprint or the prioritization of a work item is recorded with that person’s identity, whether acceptance criteria exist before the work starts and are verified before the item is closed, and whether the linkage from work item to branch to PR to deployment is maintained so that a production change can be traced back to the decision. For financially significant changes, a second-level control usually exists in the form of a release review or a change board that sees the release, not the individual items, and the auditor tests that the items the board did not see were within the delegated authority. The control descriptions guide shows how to write these controls so that the description names the platform field rather than a form that does not exist.
Secrets, credentials and the pipeline’s own identity
The pipeline holds the most powerful credentials in the organization, because it has to deploy to production, and the audit of secrets management is really an audit of what the pipeline can do and who can make it do something else. Four questions cover it. Where do secrets live: in a managed vault or the platform’s secret store with access scoped to the pipeline and the job that needs them, or in repository files, pipeline variables visible to every contributor, and the laptops of the people who set the pipeline up. Secret scanning on every commit and on repository history is the control that answers the second half of the question, and a scan of history at the start of fieldwork is the cheapest high-yield test in the engagement. How does the pipeline authenticate to the cloud or the platform it deploys to: through short-lived, federated identity issued to a specific job, or through long-lived access keys stored as variables and never rotated. Who can read or change the secrets: the list is a privileged access population and is reviewed as one. And how are secrets rotated, and what happened the last time one leaked: the rotation record and the incident record are the operating evidence.
The pipeline’s identity also needs the joiner-mover-leaver treatment, because it is a non-human identity with production rights: it should have an owner, a documented purpose, the minimum permissions its jobs need, and a review on the same cycle as human privileged access, per the IAM audit guide. Where the platform is a cloud service, the platform’s own controls are third-party controls, evidenced by its SOC 2 report and the organization’s complementary controls, as the SOC 2 review guide sets out.
Test coverage and security testing as evidence
The testing objective in a pipeline is satisfied by gates, and gates have three attributes an auditor tests: the threshold, the enforcement and the exemptions. A coverage gate of 70 percent that is enforced on every merge is a control; the same threshold reported but not enforced is a metric; and an enforced threshold with a third of repositories exempted is a control over two-thirds of the population, which the report says. For security testing, static analysis, dependency scanning and secret scanning run on every commit, with a severity gate that blocks critical and high findings, and dynamic testing runs against a staging environment for anything exposed to the internet; the auditor samples pipeline runs to see the gates operate, reviews the exemption and suppression records for who approved them and whether they expired, and reconciles the scanning coverage to the repository inventory. The test data question belongs here too: production data copied into test environments is a data protection exposure and a confidentiality control failure, and the pipeline’s test data source is a walkthrough question with a one-word answer that is either acceptable or a finding, per the data privacy audit guide.
Acceptance testing did not disappear either; it moved onto the work item. The acceptance criteria written before development and the verification recorded at closure, by someone other than the developer where the platform allows it, are the user acceptance evidence, and a sample of closed work items traced to their criteria and verification is the test. Where the organization keeps a separate UAT stage for financially significant releases, that stage is tested as a manual control with the release population as the sample frame.
Populations, reconciliation and sampling
The population of changes is not the ticket system. It is the set of production deployments in the period, taken from the deployment tool’s history or, better, from the platform or cloud audit log that records what actually changed in production and under which identity, because that log includes the deployments that bypassed the pipeline. The reconciliation runs from production backward: every deployment event to a pipeline run, every pipeline run to a merged PR, every PR to a work item, and every work item to an accepted backlog entry. Each break in the chain is a category of exception with its own meaning: a deployment with no pipeline run is a bypass; a pipeline run with no PR is a direct push or an administrator merge; a PR with no work item is unauthorized work, or a hygiene failure; a work item with no acceptance is a delegation question. The counts by category are the headline of the report.
Sampling then applies to the manual elements within the reconciled population. Twenty-five to forty PRs across repositories, risk levels and authors are tested for reviewer independence, substantive review (comments, requested changes, not a bare approval seconds after the request), gate results at merge and the artifact’s path to production; the sample sizes follow the sample sizes guide and the selection is documented as the sampling memo template lays out. Automated gates and protections are tested by configuration inspection and a test of one, with the general controls over the platform’s administration tested for the period. Break-glass uses are tested in full, because the population is small and every instance is, by definition, an exception to the control.
The test program
| # | Test | Method | Pass condition |
|---|---|---|---|
| 1 | Repository and pipeline inventory | Reconcile repositories and pipelines to the application inventory and the in-scope systems | Every production application maps to identified repositories and a pipeline; orphans explained |
| 2 | Branch protection | Export protection settings for production branches; attempt a self-approved merge in a test repository configured identically | Required reviews, no self-approval, required status checks, no force-push on every production repository |
| 3 | Platform administration | List platform and repository administrators; test their joiner-mover-leaver history and the audit log of protection changes | Administrators few, named, reviewed; protection changes logged, justified and reversed where temporary |
| 4 | PR review quality | Sample 25 to 40 merged PRs | Independent reviewer, evidence of substantive review, gates passed at merge, linked work item |
| 5 | Gate enforcement | Inspect gate configuration; sample runs; test a failing gate | Coverage, static, dependency and secret scans enforced; failing gates block; exemptions approved, time-bound and reviewed |
| 6 | Artifact integrity | Trace sampled deployments to signed artifacts with provenance; confirm the same digest across environments | Artifact deployed is the artifact built from the reviewed commit; provenance verifiable |
| 7 | Production deploy rights | List all identities with production deploy or write rights across deployment tool, cloud and platform | Pipeline identity plus a documented break-glass group only; human standing rights removed |
| 8 | Deployment reconciliation | Reconcile production change events to pipeline runs, PRs and work items for the period | Every production change traces to a reviewed PR and an authorized work item; exceptions counted by category |
| 9 | Break-glass | Inspect every break-glass use in the period | Logged, justified, reviewed after the fact, access revoked, change back-filled through the pipeline |
| 10 | Secrets management | Inventory secret storage; scan repository history; inspect pipeline authentication to targets; review rotation | No secrets in code or history; federated short-lived credentials; vault access scoped; rotation on schedule |
| 11 | Infrastructure as code | Inspect IaC repositories, review requirements, plan and apply separation, drift detection | Infrastructure changes reviewed like code; production drift detected and reconciled |
| 12 | Requirements and acceptance | Sample closed work items; test product owner delegation and acceptance evidence | Acceptance criteria before work, verification at closure, prioritization by a delegated owner |
The pipeline control inventory template
One inventory per pipeline, completed during the walkthrough and kept as the engagement’s map of where each objective is carried. It doubles as the control description set for a risk and control matrix, and the RCM template takes its rows directly.
Pipeline control inventory
1. Scope. Application(s) served; repositories; pipeline definitions and their location under version control; environments; platform (source control, CI, CD, artifact registry, cloud); pipeline owner; date of walkthrough and the change followed.
2. Source controls. Production branch name; protection settings (required reviews, code owners, self-approval, required checks, force-push, deletion); signed commits; administrators who can change protection; audit log location.
3. Build controls. Runner type and management; pipeline definition review requirements; dependency sources and lockfile policy; SBOM generation; artifact signing and provenance level (SLSA Build track level claimed).
4. Test gates. Each gate with threshold, enforcement (blocking or reporting), scope (repositories covered and exempt), exemption approver and review cycle; dynamic testing scope; test data source.
5. Release and deploy controls. Environment promotion rules; production approval mechanism and approvers; deploying identity; human production rights; break-glass path, logging and review; infrastructure as code review and drift detection.
6. Secrets. Secret store; scoping; pipeline authentication method to each target; rotation; secret scanning coverage.
7. Traceability. Work item to branch to PR to run to deployment linkage; where the deployment log lives; who reconciles it and how often.
8. Classification and testing approach. For each control: automated (configuration plus test of one) or manual (sample); general controls relied on; test reference.
Worked example: Lakeshore Bancorp’s digital channels pipeline
Lakeshore Bancorp, the illustrative $9 billion regional bank used across this site, runs its core banking on a hosted platform but builds its own online and mobile banking front end with a digital channels team of 14 engineers practicing trunk-based development, with deployments to production most working days. The application is in SOX scope because it originates transactions and displays balances, and the external auditor had flagged in the prior year that the change population “could not be reconciled to the change management system”, which was true and beside the point, because the change management system was a ticketing tool the team used for infrastructure requests. Internal audit scoped a pipeline audit at 380 hours, ran the walkthrough on a live change in the first week, and built the inventory above for the two production pipelines (web and mobile).
The reconciliation ran from the cloud platform’s audit log. In twelve months there were 1,340 production deployment events. 1,322 traced to pipeline runs; 18 did not, and all 18 were manual deployments by two platform engineers using a standing role from the pre-pipeline era that had never been removed, including one on a Sunday to revert a failed release. The 1,322 pipeline runs traced to 1,296 merged PRs (some runs deployed several merges); 41 of the PRs had been merged by repository administrators with the required-review check bypassed, all 41 by the same two engineers, 29 of them in a two-week period when a flaky integration test suite had been failing and the team was “unblocking” releases. 1,296 PRs traced to 1,180 work items; the 116 without a work item were dependency bumps and configuration changes the team treated as housekeeping. The secret scan of repository history found six live credentials, four for a test environment and two for a production analytics service, committed between 2022 and 2024 and never rotated. Coverage gates were set at 70 percent and enforced, but 9 of the 41 repositories were exempt, including the payments module. The pipeline authenticated to the cloud through federated short-lived identity, which was the strongest control in the estate, and artifacts were signed with provenance that met the SLSA Build track’s L2 description.
| Objective | Result | Finding and rating |
|---|---|---|
| Authorized before work starts | 1,180 of 1,296 PRs linked to accepted work items; the 116 unlinked were housekeeping within the team’s delegated authority, but the delegation was undocumented | Medium: define and document what the team may change without a work item |
| Tested before release | Gates enforced on 32 of 41 repositories; payments module exempt; flaky suite bypassed for two weeks | High: gate exemptions unapproved and unreviewed, including a SOX-relevant module |
| Approved by someone other than the author | 41 of 1,296 PRs merged with review bypassed by two administrators | High: administrator bypass of required review; no monitoring of bypasses |
| Only controlled mechanisms deploy | 18 of 1,340 deployments performed manually under a legacy standing role | High: human production deploy rights not removed; no break-glass process |
| Traceable production state | Artifacts signed with provenance; deployment log complete; reconciliation not performed by anyone until the audit | Medium: institute a monthly reconciliation outside the delivery team |
| Secrets | Six live credentials in repository history; federated identity for the pipeline itself | High: rotate, purge history, enable scanning on push and on history |
The SOX consequence was evaluated through the dependent application controls, as the deficiency evaluation guide describes: the 18 manual deployments and 41 bypassed merges were identified individually, the eleven that touched balance display or transaction origination were re-tested by re-performance of the affected automated controls after each change, and none had altered a financial control, so the ITGC deficiency was concluded as a significant deficiency on the strength of the bypass volume and the absence of monitoring, not a material weakness. Management removed the legacy role in week three of fieldwork, created a break-glass group with logging and a 24-hour review, made the two administrators’ bypass rights subject to an alert to the CISO’s team, and brought the payments module under the coverage gate; the audit re-tested all four before the report was issued and said so, with the dates, in the findings. The reconciliation now runs monthly, performed by the second-line technology risk team, and the next year’s audit will test it as a control rather than perform it.
Common findings and how to rate them
The same findings recur across pipeline audits, and their ratings depend on the objective they break rather than on how technical they sound. Administrators who can and do bypass review are a High wherever the application matters, because they defeat the only independent check. Human production deploy rights retained from the past are a High for the same reason, and their remediation is cheap. Unapproved gate exemptions are a High when they cover in-scope modules and a Medium otherwise. Secrets in repository history are a High until rotated and then a Medium for the process gap. Missing work-item linkage is usually a Medium hygiene finding unless it hides unauthorized work. A pipeline definition that anyone can edit without review is a High, because it is the control’s own source code. The absence of any reconciliation of production changes to approvals is a Medium at a low-risk application and a High at a SOX one, because it means nobody would know if the other controls failed. And the finding auditors most often miss is the platform administration group itself: if forty people can change branch protection, the branch protection is a suggestion, and the finding severity ratings guide is the right place to calibrate what that means for the report. The ITGC primer and the IT audit plan guide cover where pipeline audits sit in the wider general controls universe and the annual plan.
Related guides
- How to audit IT change management: from ticket to production
- Testing automated controls
- ITGC versus application controls
- SOX ITGC scoping
- Segregation of duties beyond the ERP: a universal framework
- How to audit privileged access
- How to audit identity and access management
- How to review a SOC 2 report
- How to audit data privacy compliance
- Evaluating control deficiencies
- Risk and control matrix template
- IT general controls: the complete primer
Leave a Reply