Key takeaways
- DORA's 2024 research associated rising AI adoption with an estimated 7.2% drop in delivery stability.
- The likely cause is erosion of basics: bigger batches, shallower review, tests that lag behind code.
- Baseline the DORA metrics before scaling AI tools, so impact is measured, not assumed.
- Encode guardrails in the platform: small batches, the same gates for AI code, provenance checks.
Most engineering teams now use AI coding assistants, and most developers report feeling more productive. The question for engineering leaders is different: is the system (the path from commit to customer) getting faster and more reliable?
The 2024 Accelerate State of DevOps report from Google Cloud's DORA programme offered an uncomfortable answer. As AI adoption increased, it was accompanied by an estimated 1.5% decrease in delivery throughput and an estimated 7.2% reduction in delivery stability.[1] The DORA team's interpretation is the useful part: improving the development process doesn't automatically improve software delivery without the basics, "like small batch sizes and robust testing mechanisms".[1]
These are survey-based estimates and correlations, not proof that AI causes instability. But they match a mechanism most teams will recognise: when code becomes cheap to produce, changes get bigger, reviews get shallower and tests lag behind.
Measure the system, not the typing
DORA's software delivery metrics remain the most widely used, research-backed way to measure delivery performance.[2] The 2024 report treats them as two groups:[3]
| Throughput | Stability |
|---|---|
| Change lead time: commit to production | Change failure rate: share of deployments needing immediate intervention |
| Deployment frequency | Rework rate: unplanned deployments to fix user-facing problems |
| Failed deployment recovery time |
If you roll out AI tools without a baseline for these, you'll have opinions about the impact but no evidence.
Five practices that protect delivery as AI scales
1. Baseline before rollout
Capture four to six weeks of DORA metrics from your pipeline and incident data before widening AI tool access. Then compare teams and periods honestly.
2. Keep batches small, deliberately
Set a soft limit on pull request size, and make large AI-generated changes split into reviewable steps. Small batches were a DORA basic long before AI, and they matter more now.
3. Hold AI-written code to the same gates
Same review, same tests, same security scanning. Add checks AI makes more necessary: dependency provenance (hallucinated or typo-squatted packages) and secret scanning.
4. Invest in tests at the same rate as code
If AI doubles the rate at which code arrives, a test suite that doesn't keep pace becomes the bottleneck or, worse, the blind spot. Use AI to help write tests too, but review them as carefully as production code.
5. Treat the platform as a product
The same report found that internal developer platforms raise developer productivity, with a possible temporary dip while the platform matures, and work best when they're user-centred and preserve developer independence.[1] A good paved road is where you encode the guardrails above, so teams don't each reinvent them.
AI accelerates whatever delivery system it lands in, good or bad.
A simple scorecard
Review monthly, per team: the five DORA measures, median pull request size, test coverage on changed lines, and escaped defects. If throughput rises while stability holds, scale the tools. If stability falls, fix the basics before buying more seats.
This article is general information, not legal, regulatory or financial advice. Figures are quoted from the public sources listed below as at the publication date; check the source for updates.
Sources
- Google Cloud, "Announcing the 2024 DORA report", Google Cloud Blog, October 2024. cloud.google.com
- DORA, "DORA's software delivery metrics: the four keys" and guides. dora.dev
- DORA, Accelerate State of DevOps Report 2024. dora.dev