Auditing DevSecOps Processes: Who Owns the Gate, the Exception and the Fix

Imagine a hypothetical product team that ships a B2B web platform several times a day. The pipeline looks mature: static analysis on every merge request, a dependency scan, an infrastructure-as-code scan, container image checks. The dashboard is green. Then a large customer asks a simple question during procurement: can you show that security in your delivery process actually works, not just that the tools are installed?

The security lead starts pulling the thread. The static analysis gate was switched from blocking to advisory some months ago after a wave of false positives, and nobody can say who made that decision or whether it was meant to be temporary. A high-severity dependency finding has an exception with no expiry date, approved by the same engineer who requested it. The Terraform code passes its scan, but during an incident someone widened a firewall rule directly in the cloud console, and the change never made it back into the repository.

Every tool in that story works. The process around them does not. That gap is exactly what a DevSecOps process audit is meant to find.

What a DevSecOps process audit actually examines

A DevSecOps process audit asks whether security is built into the way software is specified, built, tested, released and fixed. The object of review is not a single application or a single scan result. It is a chain of decisions: who defines security requirements, which checks run and with what authority, who may bypass them, how exceptions are granted and retired, and who accepts residual risk when a release goes out with known issues.

Ownership is the real unit of audit

Tools produce signals; people and roles make decisions. For each control, the audit needs a named owner who can change it, not just a team that operates it. A gate without an owner tends to drift: thresholds get relaxed, rules get muted, and nobody notices because the pipeline stays green. If the auditor cannot establish who decided a gate's current mode, that is a finding in its own right.

Process evidence versus technical assessment

It helps to separate two kinds of work that are often blended. Process audit evidence concerns decisions and records: owners, approvals, samples of real releases, the exception register, remediation tickets and their verification. A technical assessment of the pipeline looks at trust edges: who or what can modify pipeline definitions, how build runners are isolated, where secrets are exposed, and whether an artifact can be traced back to the source code it was built from.

NIST SP 800-204D addresses integrating software supply chain security into DevSecOps CI/CD pipelines, and it is a sensible reference once the work moves to those trust edges. It does not replace a process audit, and a process audit does not replace it. A good plan states which of the two is in scope and, if both are, which conclusions rest on which part. The technical side is covered in more depth in our guide to DevSecOps security assessment.

Planning: agree scope, sample and yardsticks first

NIST SP 800-115, the Technical Guide to Information Security Testing and Assessment, describes an assessment in three phases: planning, execution and post-execution. The guide itself says it is an overview of techniques with their benefits and limitations, not a comprehensive testing program. This article is not a complete DevSecOps audit program either. What the three phases provide is a structure that stops a process audit from collapsing into a tool inventory.

Planning strongly shapes the quality of an audit. The audit is an authorized assessment with agreed scope and rules of engagement. Typically in scope: security requirements, pipeline gates, finding triage and remediation, exception handling, release governance, and oversight of dependencies and suppliers. Out of scope unless explicitly authorized: testing production, social engineering and testing third-party services. A process audit does not imply permission to test systems the organization does not own or has not authorized for assessment.

Choosing the sample

Sampling beats inventory. Instead of requesting a list of every tool, pick several recent releases from different products and teams, and make sure to include at least one emergency change and one release that shipped with a known exception. Each sampled release becomes a thread the auditor follows from requirement to deployment. The auditor should choose the sample, not the team being audited: that way it reflects normal practice rather than the best-documented case.

Fixing the yardsticks before looking

Decide in advance which references findings will cite, and pin their versions. OWASP ASVS is a set of verifiable security requirements; for the selected version, such as 5.0.0, the project recommends citing requirements with a version prefix such as v5.0.0-1.2.5, because identifiers change between versions. The OWASP Top 10 is an awareness document, not a requirement set, and the Web Security Testing Guide is a testing method. That distinction changes the wording: "the team trains on the Top 10, but its stories carry no ASVS-mapped requirements" is a precise finding; "security is immature" is not.

If leadership needs current and target states described in governance terms, one option is a NIST CSF 2.0 profile for the development and delivery function. It is worth the effort only if someone will actually set priorities against the target profile.

Execution: trace real releases, not the tool list

During execution, the auditor walks each sampled release through the lifecycle and records, for every control, which technique produced the evidence. SP 800-115 describes a group of review techniques: documentation review, log review, ruleset review and system configuration review; these are the backbone of a process audit. Where the plan allows it, a limited authorized technical check on a representative application shows whether a control detects what it is supposed to. A green build badge is a configuration artifact. It does not prove that the underlying test covers the relevant risk.

The table below maps the main process areas to the owner the audit should identify, the evidence worth sampling and the typical failure mode.

Requirements and design

Start at the start. CISA's Secure by Design guidance holds that security principles should be applied in the design phase so that exploitable flaws are reduced before a product reaches customers. The audit question is concrete: where did the security requirements for the sampled release come from? A version-pinned ASVS level attached to the tickets can be audited. An informal agreement or unreferenced wiki page is weak evidence and must be corroborated through interviews, release records and observed practice.

Security gates and who may bypass them

For each gate, record its mode (blocking or advisory), the weakness classes it is meant to catch, who can change its mode or thresholds, and when that last happened. Blocking gates raise confidence but increase pressure to bypass them; advisory checks preserve speed but risk turning into noise. The audit should not judge the choice out of context. It asks whether the mode matches the risk of the weakness class, and whether changes to it were an owner's decision rather than gradual drift.

CISA also argues that products should be secure by default, with features such as multi-factor authentication, logging and single sign-on available at no extra cost. An auditor can check whether secure defaults and logging are part of release-readiness criteria rather than something added later.

Infrastructure as code and the live environment

A static scan of infrastructure code inspects the source. It cannot prove that the live environment matches that source. Manual console edits during incidents, out-of-band scripts and partially applied plans all create drift that a clean scan will never show. The process evidence to ask for is whether drift is detected and who reconciles it, plus a sample comparison of declared and live configuration for one or two critical resources. Our article on infrastructure-as-code security assessment covers the technical side.

Dependencies and the build chain

The process questions here are about ownership and change control: who decides that a vulnerable dependency may ship, who is allowed to edit pipeline definitions, and whether those edits go through the same review as application code. Deeper technical questions, such as artifact provenance and build runner isolation, belong to a software supply chain security assessment, and that is where SP 800-204D applies most directly.

Exceptions: where documented and actual process diverge

Exceptions are the most informative part of a DevSecOps audit. The documented process describes the normal path; exceptions show what happens under deadline pressure. Emergency fixes, manual overrides and deferred findings are usually where the gap between policy and practice becomes visible.

What a usable exception record contains

A usable exception record names the owner of the risk, the reason, the compensating control, an expiry date and an approver who is not the requester. When it expires, it is reviewed rather than silently renewed. The audit samples open and recently closed exceptions and checks each of these properties. An exception with no expiry date is, in practice, a permanent policy change that nobody decided to make.

Emergency changes and manual overrides

The emergency path is legitimate: a production incident does not wait for a full pipeline run. The audit question is whether the emergency path is closed after use: a retrospective review within an agreed time, tickets for the checks that were skipped, and manual infrastructure changes brought back into code. If emergency changes are frequent and never reviewed, the emergency path has become the real process.

The second table summarizes acceptance checks the audit can apply to releases and exceptions, and shows which evidence looks convincing but is not enough on its own.

Feedback and remediation: does the process learn?

A pipeline that finds problems but never changes how the team works is a reporting system, not a security process. This part of the audit looks at what happens after a finding exists.

Prioritize by weakness class, not finding count

MITRE's CWE Top 25 identifies the most dangerous and prevalent weakness classes behind recent CVE records, and MITRE stresses that understanding root causes feeds back into the development lifecycle and architectural planning, helping to eliminate entire classes of defects such as memory safety and injection issues. Alongside the Top 25, MITRE publishes a list ranking the weaknesses behind actively exploited vulnerabilities, using CISA's Known Exploited Vulnerabilities catalog. For an audit, this changes the question from "how many findings did the scanner report" to "which weakness classes does our process prevent by design, and are deadlines tighter for the classes that appear in exploitation data?"

CISA's Secure by Design alerts point the same way, addressing the elimination of cross-site scripting, OS command injection and directory traversal. ASVS contains a requirement on preventing OS command injection, which shows how a weakness class turns from something to worry about into something a team can verify.

A closed ticket is not a fix

Remediation evidence is the retest, not the merged pull request. The audit samples closed findings and asks for confirmation that the original issue was reproduced again after the fix and no longer occurs, and that the ticket links to that result. Where tickets close automatically on merge, the organization is measuring activity, not risk reduction. The mechanics are described in our article on vulnerability remediation control.

Feeding findings back into requirements

The strongest sign of a learning process is that recurring findings change upstream artifacts. If the same injection class appears in three consecutive releases, the expected response is a requirement, a design pattern or a library choice, not a third round of tickets. The audit looks for that trail: a finding class, a decision, a changed requirement or gate, and a later release in which the class did not recur.

Post-execution: turn observations into owned decisions

In SP 800-115, post-execution covers analysis, mitigation recommendations and reporting. For a process audit, the main risk at this stage is a long list of technical tickets that nobody with authority reads. Each finding should travel the same short path from observation to decision.

1. ObservationWhat was seen in the sample: which release, which control, and how it deviated from the documented process.
2. Evidence and techniqueWhich artifact supports it and how it was obtained (document review, log review, configuration review or authorized test), plus what that technique cannot show.
3. OwnerThe named person or role able to change the process, not only the team that will fix the symptom.
4. DecisionFix, accept with an expiry date and compensating control, or redesign the control; in each case, who decided is recorded.
5. RecheckHow and when the next cycle will verify the change against the same version-pinned references.

CISA stresses that technology providers should take ownership of product security at the executive level. In audit terms, every report needs an executive owner who accepts or rejects the residual risks it describes. Without that person, the report turns into a backlog.

Writing findings that survive standard updates

Findings should cite version-pinned references: ASVS v5.0.0 identifiers, CWE IDs and, where used, CSF 2.0 outcomes. Each finding should also state which sample it rests on and what the technique could not show, for example: "configuration review confirmed that the gate runs in blocking mode; it does not show that the rule set detects the weakness class in question". That keeps the report unambiguous when standards change and lets the next audit compare states against the same yardsticks.

Why there is no maturity score here

It is tempting to end a report with a number. But a numeric maturity score assembled from a few samples and interviews presents judgment as measurement and invites the wrong comparisons. A more useful criterion is whether the organization can show which weakness classes it prevents, how that was verified, and who accepts the remaining risk. If a score is used internally to track progress, it should be labelled plainly as the assessor's judgment, not a measured value.

What this kind of audit cannot tell you

Old guidance, new pipelines

SP 800-115 was published in 2008 and predates modern CI/CD practice. Its planning and analysis logic still holds, but pipeline-specific techniques have to come from elsewhere, which is one reason SP 800-204D matters on the technical side. ASVS and WSTG focus on web applications, so other kinds of systems need additional requirements.

A sample is not the whole population

A process audit draws conclusions from a sample of releases, exceptions and tickets. It can show that a control failed in a specific case or worked consistently across the sample, but it cannot prove that every release behaved the same way. The report should say how the sample was chosen.

Intent is not outcome

Adopting a framework or signing a commitment such as the CISA Secure by Design Pledge is a statement of intent, not proof of results. The audit relies on observable evidence. By the same logic, neither a process audit nor a penetration test is a certification, attestation or assurance of security; each is an informed view of how controls behaved in what was examined.

Preparing for an external process audit

If you plan to bring in an outside team, preparation is mostly about making the process traceable, not about polishing documents. Assign an owner to every gate and every type of exception. Make sure changes to gate configuration are versioned. Pull together an exception register with expiry dates. Choose an executive who will receive the report and is prepared to make decisions from it.

It also helps to separate process questions from technical ones early. If you want an outside view on both, you can discuss the audit scope and the evidence you already have with Pentest.red on the pentest scope preparation page: which systems, releases and pipeline trust edges are in question, what access is realistic and which constraints apply. How technical testing is organized is described on the penetration testing services page. Neither replaces the organization's own decisions about owners, exceptions and residual risk; an audit can only make those decisions visible.