vyre-conform 0.1.0

Conformance suite for vyre backends — proves byte-identical output to CPU reference
Documentation
//! H6 — the mutation gate harness.
//!
//! This is the single most load-bearing piece in the whole conform
//! system. A test that the gate accepts is a test that catches bugs.
//! A test that passes the suite but survives a mutation is slop, and
//! the gate is the mechanism that rejects it.
//!
//! The mechanism is straightforward and the subtlety is in the
//! details:
//!
//! 1. Snapshot the source file under test.
//! 2. Apply a mutation from the catalog.
//! 3. Write the mutated source back.
//! 4. Run `cargo test <test_fn_name>` against the mutated crate.
//! 5. FAILED → mutation killed (good — the test caught the bug).
//! 6. PASSED → mutation survived (finding — the test missed the bug).
//! 7. Compile errors or cargo failures → skipped (not a finding).
//! 8. Restore the snapshot.
//! 9. Repeat for every mutation in the requested classes.
//!
//! The snapshot/restore mechanism uses a RAII guard so a panic or
//! early return in the middle of a probe still restores the source.
//! Failing to restore would corrupt the working tree — unacceptable.
//!
//! Parallelism is deferred to Phase 5. Phase 4 runs mutations
//! sequentially via a single workdir lock. The plan explicitly
//! allows this: "Parallelism: can run multiple mutations concurrently
//! using copy-on-write source trees. [...] Skip this optimization
//! for Phase 4; add in Phase 5 if needed."
//!
//! ## Decoupling from the mutation catalog
//!
//! This module does NOT depend on `crate::adversarial::mutations::catalog::Mutation`
//! directly. It operates on an [`AppliedMutation`] — a trait object
//! the caller supplies. `crate::adversarial::mutations` provides a thin adapter
//! that turns every `Mutation` in `MUTATION_CATALOG` into an
//! `AppliedMutation`. This keeps the gate independent of the catalog
//! shape so Phase 3.2's wiring is free to evolve.
//!
//! ## Structured feedback
//!
//! When a mutation survives, the gate produces a
//! [`StructuredFeedback`] entry naming the mutation and suggesting a
//! concrete change to the test that would kill it. Feedback goes to
//! the Prosecutor (the agent writing tests); a future L5 wiring feeds
//! it back into the agent stream as a rejection reason.

use std::fs;
use std::io;
use std::path::{Path, PathBuf};
use std::time::{Duration, Instant};

use crate::spec::types::MutationClass;

#[path = "mutation_cargo.rs"]
mod mutation_cargo;
use mutation_cargo::{assert_source_matches_original, run_cargo_test};

// ---------------------------------------------------------------------------
// Public types
// ---------------------------------------------------------------------------

/// A single mutation the gate can apply to a source file.
///
/// Implementors live in `crate::adversarial::mutations`. Callers build an
/// `AppliedMutation` for every catalog entry they want to test and
/// pass them into `mutation_probe`. The trait is deliberately minimal
/// so the catalog can plug in without coupling to the gate's internals.
pub trait AppliedMutation: Send + Sync {
    /// Short identifier, unique within a run. Used in report output.
    fn id(&self) -> &str;

    /// Human-readable description shown in failure messages.
    fn description(&self) -> &str;

    /// The mutation's class. Used by `mutation_probe` to filter which
    /// mutations to run against a given test.
    fn class(&self) -> MutationClass;

    /// Apply the mutation to the source string and return the mutated
    /// form. Returning `Err` means the mutation was not applicable to
    /// this file (not a test failure — a skip).
    fn apply(&self, source: &str) -> Result<String, MutationApplyError>;

    /// Suggested hint to the Prosecutor when this mutation survives.
    /// Returns a short, actionable sentence the agent can act on.
    fn hint(&self) -> String {
        format!(
            "test passed when I applied `{}`. Add an assertion that distinguishes the \
             original from the mutated behaviour.",
            self.description()
        )
    }
}

/// Error returned by `AppliedMutation::apply` when the mutation does
/// not apply to the given source. Not a test failure — a skip.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum MutationApplyError {
    /// The mutation's target pattern was not found in the source.
    NotApplicable {
        /// Why the mutation did not apply.
        reason: String,
    },
    /// Applying the mutation would produce unambiguously invalid
    /// syntax. The gate skips these rather than pretending to run them.
    WouldProduceInvalidSyntax {
        /// Why the mutation would break syntax.
        reason: String,
    },
}

impl core::fmt::Display for MutationApplyError {
    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
        match self {
            Self::NotApplicable { reason } => write!(f, "mutation not applicable: {reason}"),
            Self::WouldProduceInvalidSyntax { reason } => {
                write!(f, "mutation would produce invalid syntax: {reason}")
            }
        }
    }
}

/// The outcome of probing one test with one mutation.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum MutationOutcome {
    /// The test failed after the mutation — the test caught it.
    Killed,
    /// The test passed after the mutation — the test missed it.
    /// This is a finding that should be surfaced to the Prosecutor.
    Survived,
    /// The mutation could not be applied, or applying it produced
    /// code that did not compile. Not a test failure.
    Skipped {
        /// The reason the probe skipped this mutation.
        reason: String,
    },
}

/// One entry in the gate report — a mutation and its outcome.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MutationResult {
    /// Identifier of the mutation that was applied.
    pub mutation_id: String,
    /// Human-readable description of the mutation.
    pub description: String,
    /// Outcome of the probe.
    pub outcome: MutationOutcome,
    /// Wall time for this single probe.
    pub duration: Duration,
}

/// Structured hint emitted for every surviving mutation. Meant to be
/// consumed by the Prosecutor agent to strengthen its tests.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct StructuredFeedback {
    /// Identifier of the mutation that survived.
    pub mutation_id: String,
    /// Actionable suggestion for the Prosecutor.
    pub hint: String,
}

/// Full report from a mutation-probe run.
#[derive(Debug, Clone)]
pub struct GateReport {
    /// Name of the test function that was probed.
    pub test_name: String,
    /// Path to the source file the mutations were applied to.
    pub source_file: PathBuf,
    /// Every mutation that was attempted, in order.
    pub results: Vec<MutationResult>,
    /// Total wall time across every probe.
    pub total_duration: Duration,
    /// Structured feedback entries, one per surviving mutation.
    pub feedback: Vec<StructuredFeedback>,
}

impl GateReport {
    /// Mutations the test killed.
    #[inline]
    pub fn killed(&self) -> Vec<&MutationResult> {
        self.results
            .iter()
            .filter(|r| matches!(r.outcome, MutationOutcome::Killed))
            .collect()
    }

    /// Mutations that survived — the Prosecutor findings.
    #[inline]
    pub fn survived(&self) -> Vec<&MutationResult> {
        self.results
            .iter()
            .filter(|r| matches!(r.outcome, MutationOutcome::Survived))
            .collect()
    }

    /// Mutations that were skipped (not applicable or compile error).
    #[inline]
    pub fn skipped(&self) -> Vec<&MutationResult> {
        self.results
            .iter()
            .filter(|r| matches!(r.outcome, MutationOutcome::Skipped { .. }))
            .collect()
    }

    /// True when every applicable mutation was killed. A gate pass.
    #[inline]
    pub fn is_pass(&self) -> bool {
        self.survived().is_empty() && !self.killed().is_empty()
    }
}

// ---------------------------------------------------------------------------
// Snapshot RAII guard
// ---------------------------------------------------------------------------

/// Snapshots a file's contents on creation and restores them on drop.
/// If the drop handler fails to restore (disk full, permission error),
/// we emit an eprintln and set a global flag the probe checks before
/// exiting so the caller is alerted rather than silently corrupting
/// the working tree.
struct SourceSnapshot {
    path: PathBuf,
    original: Vec<u8>,
    restored: bool,
}

impl SourceSnapshot {
    fn new(path: &Path) -> io::Result<Self> {
        let original = fs::read(path)?;
        Ok(Self {
            path: path.to_path_buf(),
            original,
            restored: false,
        })
    }

    fn restore_explicit(&mut self) -> io::Result<()> {
        if self.restored {
            return Ok(());
        }
        fs::write(&self.path, &self.original)?;
        self.restored = true;
        Ok(())
    }
}

impl Drop for SourceSnapshot {
    fn drop(&mut self) {
        if self.restored {
            return;
        }
        if let Err(err) = fs::write(&self.path, &self.original) {
            eprintln!(
                "vyre-conform H6 mutation gate: FAILED TO RESTORE {} on drop: {}. \
                 The working tree may be in a mutated state. \
                 Original contents are {} bytes; write the snapshot back manually.",
                self.path.display(),
                err,
                self.original.len()
            );
        }
    }
}

// ---------------------------------------------------------------------------
// The probe
// ---------------------------------------------------------------------------

/// Run the mutation gate on a single test against a slice of
/// mutations.
///
/// Every mutation is applied in turn, `cargo test` is invoked
/// against the specified test, and the outcome is recorded. The
/// source file is always restored between probes, even on panic.
///
/// `cargo_test_args` lets the caller pass extra arguments (like
/// `--offline` or `--target-dir`) to isolate the probe from other
/// cargo invocations in the workspace.
mod probe;
pub use probe::{canary_plus_to_minus, mutation_probe};