gunnar-server 3.0.0

//! **The gix arm: `storage-git-gix`, `gix-odb` / `gix-ref` on a plain filesystem.**
//!
//! The second implementation of the same eleven. Not the default and not in the
//! appliance image: `--no-default-features --features server,engine-gix` builds
//! it alone, `--features engine-znippy,engine-gix` builds both and picks at
//! runtime. Its reason for existing is that an A/B needs two things to compare,
//! and until it landed there was one.
//!
//! The store is an ordinary **bare git repository**, so whatever this arm writes,
//! stock `git` can read — which is what makes it a usable oracle for the other.
//!
//! # Three narrowings, recorded as facts rather than as trait-change requests
//!
//! Two are the backend's own, from the two-engine record:
//!
//! 1. `extents()` returns a **pack-local** offset. With N packs an `Extent` does
//!    not identify a byte range on its own, and there is nowhere in `(u64, u64)`
//!    for a pack id. No synthetic global offset was invented, because it would
//!    index nothing.
//! 2. `TxId.push_seq` is **in-process only**. This backend keeps no ref log, so
//!    there is no durable ordering authority; it resets on restart.
//!
//! The third belongs to this file and is new with it:
//!
//! 3. **`want_objects` is refused.** Enumerating the ids one push introduced is
//!    not on `GitOps` and has no analog here — the znippy arm answers it out of
//!    its own extent index (`oids_in_extent`), and this arm has no extent index
//!    to ask. [`ingest`] therefore *fails* rather than answering `None`: `None`
//!    on that field means **not asked for**, and a policy gate handed `None`
//!    refuses, so returning it here would turn "this engine cannot tell you"
//!    into "your policy passed". A named, bounded absence, never a plausible
//!    wrong answer.
//!
//! # `seal` has no analog either, and that is not a gap
//!
//! `seal()` folds znippy's live logs into reserved **Arrow** sections. This
//! backend has no live logs to fold — `gc()` reads the packs and the ref store
//! directly — so [`maintain`] reports `sections_sealed: 0`, which is the count
//! of what happened rather than a stub value.

use std::io::BufRead;
use std::path::{Path, PathBuf};
use std::sync::Arc;

use anyhow::{bail, Context, Result};
use git_storage_trait::GitOps;
use storage_git_gix::GixStore;

use super::store_cache::{self, Cache};
use super::{BeforeStore, MaintenanceReport, OpenRepo, PushReceipt};

/// Every `GixStore` this process has opened, by store root.
static STORES: Cache<GixStore> = Cache::new();

/// **The one handle for `git_dir`** — the reading path and the ingest path get
/// the same object, so a `put_pack` refreshes the view the server serves from.
///
/// This is the fix for the defect written up in [`super::store_cache`]: `open`
/// and `ingest` used to call `GixStore::open_or_init` separately, and
/// `GixStore::reload_packs` — *"git's `reprepare_packed_git()`, called from the
/// writer, which is where git calls it"* — then refreshed a handle that was
/// about to be dropped. Everything pushed into a repository was invisible to
/// the reader for the life of the process.
///
/// It is by construction rather than by a guard (one implementation): there is no
/// "invalidate after write" step to forget, because there is nothing to
/// invalidate. One writer, one reader, one `PackSet`.
///
/// # This says nothing about two PROCESSES
///
/// `storage-git-gix`'s own docs record *"no writer lock — two processes writing
/// one store root is not defended against here"*, and that is still true and
/// still not defended against. This cache is per-process and fixes a
/// per-process defect. The file's original note — that one handle per open was
/// "the honest shape of a store that makes no such promise" — conflated the two:
/// declining an in-process cache never bought any cross-process safety, and it
/// cost correctness within the process.
fn open_shared(git_dir: &Path, hash_kind: gix_hash::Kind) -> Result<Arc<GixStore>> {
    let root = PathBuf::from(git_dir);
    store_cache::shared(&STORES, root, || {
        GixStore::open_or_init(git_dir, hash_kind)
            .with_context(|| format!("opening the gix store at {}", git_dir.display()))
    })
}

/// Release every cached handle. See [`super::shutdown`].
pub fn shutdown() {
    store_cache::clear(&STORES);
}

/// Open — creating if absent — the bare repository at `git_dir`.
///
/// Through [`open_shared`], so the handle this hands the serving path is the
/// same one [`ingest`] writes through. `ServerStores` also caches the
/// `Arc<OpenRepo>` per repository, but that cache is keyed on the repository
/// and is not what makes the write visible — the shared `GixStore` underneath
/// is.
///
/// # 🔴 This does not compile until edda implements `GitServe` for `GixStore`
///
/// [`OpenRepo::new`] requires the **reading** contract, because everything the
/// server does with a repository needs it (`UploadPack`, `ReceivePack`,
/// `gunnar-policy`'s ancestry walk, and `HEAD`, which is not a row in
/// [`GitOps::refs`]). `storage-git-gix` implements `GitOps` and not `GitServe`
/// — the arm is PAUSED — so this call is a compile
/// error in any build carrying `engine-gix`, and it is the right kind: a
/// missing half of the contract, named at the seam, rather than a repository
/// that opens and then refuses every clone at runtime.
///
/// Two things are owed by edda before this arm builds, and neither is gunnar's
/// (see `agentL-report.md`):
///
/// 1. `GitOps::put_refs_cas` — a **pre-existing** `E0046` on `GixStore`, which
///    is why `--features engine-gix` did not compile before this change either.
/// 2. `impl GitServe for GixStore`.
pub fn open(git_dir: &Path, hash_kind: gix_hash::Kind) -> Result<Arc<OpenRepo>> {
    Ok(OpenRepo::new(
        git_dir,
        hash_kind,
        open_shared(git_dir, hash_kind)?,
    ))
}

/// One push into the bare repository: `index_pack` writes `.pack` / `.idx`.
///
/// See narrowing 3 in the module docs for `want_objects`.
///
/// `progress` is carried and handed to
/// [`GitOps::put_pack_observed`](git_storage_trait::GitOps::put_pack_observed),
/// which on this arm is the trait's own default: `GixStore` has no entry loop
/// of its own — `gix_pack` owns the split — so it publishes nothing and says so
/// through `publishes_pack_progress()`. That is the right answer rather than a
/// gap: a fabricated `(0, N) → (N, N)` here would read to a console exactly
/// like the live count the znippy arm now produces.
pub fn ingest(
    git_dir: &Path,
    hash_kind: gix_hash::Kind,
    pack: &mut dyn BufRead,
    objects: super::ObjectSet,
    progress: Option<&dyn git_storage_trait::PackProgress>,
    before_store: Option<BeforeStore<'_>>,
) -> Result<PushReceipt> {
    // **`Required` is refused; `Wanted` is answered with `None`.**
    //
    // It used to refuse on a bare `want_objects`, and when the push attestation
    // started asking for the set that turned every push on this arm into
    // "the repository is unavailable". The refusal is still right for a gate —
    // a gate over objects cannot decide without them — and wrong for a record
    // that has a field for saying the set is unknown.
    if objects == super::ObjectSet::Required {
        bail!(
            "the gix storage engine cannot enumerate the objects a push introduced: that is not \
             one of the eleven, and this backend keeps no extent index to answer it from. The \
             policy on this repository has a gate over the pushed object set, so the push is \
             refused rather than passed on an answer the engine did not give. Serve this \
             repository from the znippy engine, or drop the gate."
        );
    }
    let started = std::time::Instant::now();
    // THE SAME handle the serving path holds. `put_pack` reloads the pack set
    // on the instance it is called on, so writing through a throwaway here is
    // what made every pushed object invisible until the next restart.
    let store = open_shared(git_dir, hash_kind)?;
    let pushed = super::read_pushed_pack_spooling(pack, hash_kind, Some(git_dir))?;
    // The transaction log first (`Engine::ingest_push`'s `before_store`).
    if let Some(hook) = before_store {
        hook(&pushed.bytes).context("framing the pushed pack in the transaction log")?;
    }
    match progress {
        Some(sink) => store.put_pack_observed(&pushed.bytes, sink),
        None => store.put_pack(&pushed.bytes),
    }
    .context("storing the pushed pack")?;
    let pack_id = super::pack_trailer(&pushed.bytes, hash_kind)?;
    Ok(PushReceipt {
        num_objects: pushed.num_objects,
        version: pushed.version,
        pack_id,
        pack_bytes: pushed.bytes.len() as u64,
        new_oids: None,
        elapsed_ms: started.elapsed().as_millis() as u64,
    })
}

/// `gc` alone. There is no seal on this arm — see the module docs.
pub fn maintain(git_dir: &Path, hash_kind: gix_hash::Kind) -> Result<MaintenanceReport> {
    // Shared for the same reason ingest is, and it is the same bug in the other
    // direction: `gc` UNLINKS packs, so a reader holding its own `PackSet`
    // would go on answering out of an index naming files that are gone.
    let store = open_shared(git_dir, hash_kind)?;
    let report = store.gc().context("sweeping dead packs")?;
    Ok(MaintenanceReport {
        strategy: report.strategy.to_owned(),
        bytes_before: report.bytes_before,
        bytes_after: report.bytes_after,
        rows: report.rows,
        retired_packs: report.retired_packs,
        verified: report.verified,
        sections_sealed: 0,
    })
}

#[cfg(test)]
mod tests {
    use std::process::Command;

    use super::*;

    /// A one-commit pack and the oid of its commit, built by stock `git`.
    ///
    /// Stock git rather than a hand-rolled writer: the bytes this arm has to
    /// accept are the bytes a real client pushes, and a pack this file wrote
    /// itself would prove that this file agrees with itself.
    fn tiny_pack(work: &Path) -> (Vec<u8>, String) {
        let run = |args: &[&str]| {
            let out = Command::new("git")
                .args(args)
                .current_dir(work)
                .output()
                .expect("git is on PATH (xtask preflight asserts it)");
            assert!(
                out.status.success(),
                "git {args:?}: {:?}",
                out.status.code()
            );
            out.stdout
        };
        run(&["init", "-q", "--initial-branch=main", "."]);
        run(&["config", "user.email", "t@t"]);
        run(&["config", "user.name", "t"]);
        std::fs::write(work.join("f"), b"hello").expect("fixture file");
        run(&["add", "f"]);
        run(&["commit", "-qm", "one"]);
        let oid = String::from_utf8(run(&["rev-parse", "HEAD"]))
            .expect("utf8")
            .trim()
            .to_owned();

        // `--revs --stdout` takes the rev on stdin and writes the pack to
        // stdout, which is the same shape `receive-pack` hands `ingest`.
        let mut child = Command::new("git")
            .args(["pack-objects", "--revs", "--stdout", "-q"])
            .current_dir(work)
            .stdin(std::process::Stdio::piped())
            .stdout(std::process::Stdio::piped())
            .spawn()
            .expect("git pack-objects");
        {
            use std::io::Write as _;
            let mut stdin = child.stdin.take().expect("stdin");
            writeln!(stdin, "{oid}").expect("write rev");
        }
        let out = child.wait_with_output().expect("pack-objects output");
        assert!(out.status.success(), "git pack-objects failed");
        assert!(!out.stdout.is_empty(), "git pack-objects produced no pack");
        (out.stdout, oid)
    }

    /// **A push must be visible to a reader that was opened BEFORE it.**
    ///
    /// The regression test for the defect in [`super::super::store_cache`], and
    /// the ordering is the whole test. `open` *after* `ingest` passes even on
    /// the broken code, because a fresh handle loads a fresh `PackSet` off disk
    /// — which is exactly why restarting the server "fixed" it and why this went
    /// unnoticed. The server does not work that way: `ServerStores` caches an
    /// `Arc<OpenRepo>` per repository, so the reader is almost always older than
    /// the push.
    ///
    /// Seen red on the pre-fix code, which answered
    /// `<oid> is a LOOSE object: it is in this repository but has no pack entry,
    /// so there are no stored bytes to copy` — with the object sitting in a pack
    /// that `git verify-pack` called `ok`.
    #[test]
    fn a_push_is_visible_to_a_reader_opened_before_it() {
        let dir = tempfile::tempdir().expect("tempdir");
        let work = dir.path().join("work");
        std::fs::create_dir_all(&work).expect("work dir");
        let (pack, oid_hex) = tiny_pack(&work);

        let git_dir = dir.path().join("repo.git");
        let kind = gix_hash::Kind::Sha1;

        // THE READER FIRST. This is the ordering the server has.
        let reader = open(&git_dir, kind).expect("opening the repository");
        let raw = gix_hash::ObjectId::from_hex(oid_hex.as_bytes()).expect("oid");
        assert!(
            !reader.serve().has(raw.as_slice()).expect("has() before"),
            "the repository already held the commit before it was pushed, so this test would \
             pass without the push and prove nothing"
        );

        let mut cursor: &[u8] = &pack;
        ingest(
            &git_dir,
            kind,
            &mut cursor,
            crate::storage::ObjectSet::NotNeeded,
            // `None`: this test is about the store handle a reader holds, not
            // about progress reporting. RED before this argument was added
            // here — `ingest` grew a fifth parameter and this call did not
            // follow it, so the whole `gix_engine` unit-test target stopped
            // compiling and took the crate's `--lib` tests with it.
            None,
            // `None`: no `before_store` tee either. `ingest` grew a sixth
            // parameter (786bc63, the twin frames a pushed pack) and this call
            // did not follow it, which again took the crate's `--lib` tests down.
            None,
        )
        .expect("ingesting the pushed pack");

        assert!(
            reader.serve().has(raw.as_slice()).expect("has() after"),
            "the reader opened before the push cannot see it: the ingest wrote through a \
             different store handle and refreshed that one instead"
        );
        let got = reader
            .serve()
            .get(raw.as_slice())
            .expect("get() must not error")
            .expect("the commit must have stored bytes to copy");
        assert!(
            !got.bytes.is_empty(),
            "the commit resolved to zero stored bytes, which is what `emit_pack` copies"
        );

        shutdown();
    }
}