Module ai_assisted_development 

Measuring AI-Assisted Software Development

AI coding assistance can feel dramatically faster at the point of initial code generation while showing no net cycle-time improvement once the full pipeline, including review and correction, is measured — code that is faster to produce but slower to review, or that requires more rework, can offset or reverse the apparent gain. A genuine productivity gain shows faster cycle time with stable or improved quality; a false gain shows faster cycle time with degrading quality, exactly the trade this book warns against throughout, discovered here through the same paired-metric discipline applied elsewhere to DORA and to flow metrics.

Formula

Net cycle-time change (%) = (cycle time before − cycle time after) / cycle time before × 100
Genuine gain                 when cycle time improved AND defect rate did not worsen

Why it matters

Developer self-report of “this saved me an hour” is a useful starting hypothesis, subject to the same recall and desirability biases as any self-reported data, and it says nothing about downstream review or correction cost. Measuring the full cycle-time chain, not just the coding stage, and checking it against change failure rate or escaped defect rate rather than trusting a felt sense of speed, is what distinguishes a genuine gain from a false one.

Example

A team’s initial code-generation step feels dramatically faster, but the full pipeline, including a slower review and correction step, shows no net cycle-time improvement, and the defect rate has quietly worsened — the chapter’s named false-gain pattern.

use software_engineering::ai_assisted_development::{
    net_cycle_time_change_percent, is_genuine_productivity_gain,
};

// Full cycle time (generation + review + correction) fell from 10 hours
// to 8 hours: a 20% improvement.
let change = net_cycle_time_change_percent(10.0, 8.0).unwrap();
assert!((change - 20.0).abs() < 1e-9);

// But if the defect rate rose from 2% to 5% alongside that speedup,
// this is a false gain, not a genuine one.
assert!(!is_genuine_productivity_gain(10.0, 8.0, 2.0, 5.0));

// The same speedup with a stable or improved defect rate is genuine.
assert!(is_genuine_productivity_gain(10.0, 8.0, 2.0, 2.0));

Pitfalls

• Measuring only the generation-speed step, ignoring full cycle time — the chapter’s central named pitfall; review and correction cost can offset or reverse the apparent gain entirely.
• Treating self-reported time savings as a conclusion rather than a starting hypothesis to validate against objective cycle-time and quality data.
• Comparing only a before-and-after snapshot, without a genuine comparison group or a longer historical baseline — cannot distinguish AI assistance’s effect from any other concurrent change.
• Reporting a single blended average across task types — hides that assistance may provide strong value for boilerplate work and little or negative value for genuinely novel, complex problem-solving.

Sources

• Chapter 7.2, Measuring AI-assisted software development.

Topic doc: software-engineering-metrics/locales/en-001/chapters/07-02-measuring-ai-assisted-software-development.md

Functions

is_genuine_productivity_gain

Whether an observed cycle-time speedup is a genuine productivity gain rather than a false one.

net_cycle_time_change_percent

Net cycle-time change: the percentage change in full cycle time (generation plus review plus correction), positive meaning faster.