Skip to main content

alkgit_substrate/
odb.rs

1//! The substrate object front (wave 7): alkblobs as the git-object
2//! store under the derivation family, and the sync Find the pack
3//! pipelines consume ([`gix_object::Find`] + [`gix_object::Exists`],
4//! [`gix_pack::Find`]) — substrate.md §object storage; ADR-019 §1; the
5//! POC-O `SubstrateOdb` pattern.
6//!
7//! # The derivation family (alkblobs ADR-018)
8//!
9//! Loose objects live under the git-object derivation family (registry
10//! byte `0x03`; `0x04` is the sha256 member, registered but untouched
11//! here — OQ-05 keeps every serving-path oid sha1). The stored preimage
12//! is git's loose-object form — `<type> <len>\0` + content, the header
13//! *inside* the value — so the digest is byte-identically git's oid of
14//! the object, every kind included, and external-oid addressing is
15//! map-free: the key is `[0x03] + oid bytes` ([`oid_key`]), built by
16//! `Key::from_bytes` over the physical encoding (family-explicit,
17//! rejects-never-guesses).
18//!
19//! The odb layer owns the header the family stores: it composes the
20//! loose form ([`loose_form`]) it puts through the known-length
21//! derivation seam (`put_as(Algorithm::GitObjectSha1, len, stream)` —
22//! alkblobs 0.2's post-ADR-020 shape) and parses the header back out
23//! through `gix_object::decode::loose_header` on read. Verification-on-
24//! put is the oid re-derivation (the store derives the digest from the
25//! stored bytes; a put whose returned key does not re-derive to the
26//! object's oid is rejected — corruption cannot enter), and the front
27//! re-derives on every read as well, so entries the store's kv tier
28//! does not cover (large-tier payloads are not re-derived at open) fail
29//! the read too. `stat` returns the *preimage* length — the stored
30//! value's length, git's header included, never the content length (the
31//! family semantic, alkblobs ADR-018 §Decision 3).
32//!
33//! # The Find front and the one-time construction
34//!
35//! [`ObjectFront`] is constructed once over the store (the
36//! handle-sharing shape — backend.md §crate and feature model; ADR-004's
37//! status clause: the retired disk-odb handle sharing is realized as
38//! this front, constructed once with per-session views).
39//! [`ObjectFront::session`] mints a per-session [`SessionFind`]: a view
40//! sharing the front's store + runtime handles that carries its own
41//! lookup-miss slot, so one session's failed walk cannot contaminate
42//! another's. Clones of a view share that session's slot (the pipeline
43//! clones the find — the core crate's `MemFind` double's sharing within
44//! one session).
45//!
46//! `MemFind` is the core crate's test-double form of this exact shape
47//! (`tests/common/mod.rs`). The location/entry/pack-offset
48//! ([`gix_pack::Find::location_by_oid`],
49//! [`gix_pack::Find::pack_offsets_and_oid`],
50//! [`gix_pack::Find::entry_by_location`]) legs answer "not in any
51//! pack": the substrate has no packed objects — the pipeline's `FindExt`
52//! blanket impls pick up the rest. No disk odb exists anywhere on the
53//! serving path (gix-odb is deliberately absent from the manifest).
54//!
55//! A lookup miss is recorded on the view ([`SessionFind::missed_object`])
56//! so a failed walk/count stage maps its not-found error back to the
57//! seam's `StorageError::ObjectMissing` abort (ADR-004), naming the
58//! missing object rather than the want.
59//!
60//! # The sync Find / async store bridge — owned once, in one place
61//!
62//! gitoxide's Find traits are sync by design; the substrate store is
63//! async. The bridge is this module's [`ObjectFront::block_on`]
64//! (`Handle::block_on` against the runtime handle captured at
65//! construction) — consumed from blocking pipeline tasks
66//! (`tokio::task::spawn_blocking`), never from async contexts, where it
67//! panics (the POC-O panic shape, learned live; poco-findings
68//! §trait posture). Async contexts that must reach the store bridge with
69//! [`ObjectFront::block_in_place`] (`block_in_place` there — it parks
70//! the worker and makes `block_on` legal). The eventual
71//! `GitPackGen`/`GitPackIngest` impls and the liveness source consume
72//! these two postures — no consumer constructs its own `Handle`.
73//!
74//! # Error model
75//!
76//! The async legs ([`ObjectFront::put`], [`ObjectFront::get`],
77//! [`ObjectFront::stat`]) surface the seam's `StorageError` family:
78//! backing-store failures are `StorageError::Io` (message-only, no
79//! engine source types — the same discipline as the other backend
80//! impls), corruption detected at the front (a header that does not
81//! parse back, a body that does not match its header, a re-derivation
82//! mismatch) is `StorageError::Invalid` ("structurally invalid backend
83//! state"). The sync trait layer maps those into `gix_error::Error`;
84//! misses stay `Ok(None)` — a miss is a recorded result, not an error.
85//! A non-sha1 oid ([`oid_key`]) is `StorageError::Invalid`: OQ-05's
86//! rejects-never-guesses posture at the front.
87
88use std::sync::Arc;
89
90use alkblobs::{Algorithm, EntryMeta, Pin, Store};
91use gix_hash::ObjectId;
92use gix_object::Kind;
93use tokio::runtime::Handle;
94
95use alkgit::StorageError;
96
97/// The derivation family every serving-path object rides: the sha1
98/// member of the git-object family (alkblobs ADR-018, registry byte
99/// `0x03`). The sha256 member (`0x04`) is registered but untouched —
100/// OQ-05 keeps every oid sha1 until ecosystem need re-opens it.
101pub(crate) const GIT_OBJECT_ALGORITHM: Algorithm = Algorithm::GitObjectSha1;
102
103/// The key the derivation family derives an object's oid to: `[0x03] +
104/// the oid bytes`, decoded through `Key::from_bytes` over the physical
105/// encoding — family-explicit, rejects-never-guesses (alkblobs ADR-018;
106/// ADR-020 §Decision 5's external-oid shape). A non-sha1 oid is
107/// `StorageError::Invalid` (OQ-05: never silently re-addressed).
108pub fn oid_key(oid: &gix_hash::oid) -> Result<alkblobs::Key, StorageError> {
109    if oid.kind() != gix_hash::Kind::Sha1 {
110        return Err(StorageError::Invalid(format!(
111            "oid {oid} is not a sha1 oid; the object front addresses only the sha1 family (OQ-05)"
112        )));
113    }
114    let mut bytes = Vec::with_capacity(1 + oid.as_bytes().len());
115    bytes.push(GIT_OBJECT_ALGORITHM.byte());
116    bytes.extend_from_slice(oid.as_bytes());
117    alkblobs::Key::from_bytes(&bytes)
118        .map_err(|e| StorageError::Invalid(format!("the oid-tagged key does not decode: {e}")))
119}
120
121/// The bounded prefix read behind [`ObjectFront::kind`]: the loose
122/// header (`<type> <len>\0`) is never longer than ~30 bytes, so a kind
123/// determination never touches the body — the is-commit half of the
124/// ack predicate (ADR-014 §5) does not read the object's content.
125pub(crate) const LOOSE_HEADER_PEEK_MAX: u64 = 128;
126
127/// The object id behind a family key — `Some` only for a
128/// `GitObjectSha1` key (the family the front serves; ADR-019 §1's
129/// loose-form keying, alkblobs ADR-018); any other
130/// algorithm is `None` (the `0x01`/`0x02` blob members and the `0x04`
131/// member are not oid-addressable from here).
132pub fn key_to_oid(key: &alkblobs::Key) -> Option<ObjectId> {
133    if key.algorithm() != GIT_OBJECT_ALGORITHM {
134        return None;
135    }
136    let raw = key.digest().bytes();
137    if raw.len() != gix_hash::Kind::Sha1.len_in_bytes() {
138        return None;
139    }
140    let mut bytes = [0u8; 20];
141    bytes.copy_from_slice(raw);
142    Some(ObjectId::from(bytes))
143}
144
145/// The stored preimage of a git object: git's loose-object form —
146/// `<type> <len>\0` + content, the header *inside* the value (alkblobs
147/// ADR-018; the digest over exactly these bytes is git's oid). The odb
148/// layer owns this composition — [`ObjectFront::put`] puts it, the read
149/// legs parse the header back out through
150/// `gix_object::decode::loose_header`.
151pub fn loose_form(kind: Kind, content: &[u8]) -> Vec<u8> {
152    let mut form = gix_object::encode::loose_header(kind, content.len() as u64).into_vec();
153    form.extend_from_slice(content);
154    form
155}
156
157/// The async store legs' error mapping (crate-shared — the packgen
158/// module's store legs compose through the same closure): backing-store
159/// failure vs corruption-classified, no engine source types on the
160/// surface. See the module's error-model section.
161pub(crate) fn store_leg_error(context: &str, error: alkblobs::Error) -> StorageError {
162    match error {
163        alkblobs::Error::Verification | alkblobs::Error::KeyInvalid => {
164            StorageError::Invalid(format!("{context}: {error}"))
165        }
166        alkblobs::Error::Missing
167        | alkblobs::Error::Io(_)
168        | alkblobs::Error::GcRefuse
169        | alkblobs::Error::RegistryFull => StorageError::Io(format!("{context}: {error}")),
170    }
171}
172
173/// The one-time object front over the substrate's alkblobs store: the
174/// handle-sharing shape's constructed-once half (backend.md §crate and
175/// feature model). The object pool is global, not per-repo-namespaced —
176/// repo scoping is the refs rows' and liveness sources' business.
177///
178/// Construct within the running runtime (the assembly does): the
179/// constructor captures `Handle::current()` for the sync/async bridge.
180#[derive(Clone)]
181pub struct ObjectFront {
182    store: Arc<Store>,
183    runtime: Handle,
184}
185
186impl ObjectFront {
187    /// The front over the substrate store. The store handle is shared,
188    /// not owned: pools are process-wide, the front is a view over one.
189    pub fn new(store: Arc<Store>) -> Self {
190        ObjectFront {
191            store,
192            runtime: Handle::current(),
193        }
194    }
195
196    /// A per-session view: carries its own lookup-miss slot, sharing the
197    /// front's store and runtime handles. Mint one per protocol session.
198    pub fn session(&self) -> SessionFind {
199        SessionFind {
200            front: self.clone(),
201            missed: Arc::default(),
202        }
203    }
204
205    /// The shared store handle — the substrate composition's object
206    /// pool. Batch puts (`put_in_batch`), range serving (`read_range`),
207    /// and the GC wiring ride this handle; the front's legs are the
208    /// loose-object seam over it.
209    pub fn store(&self) -> &Arc<Store> {
210        &self.store
211    }
212
213    /// The one sync Find / async store bridge, blocking-context half:
214    /// the call pack pipelines own (`Handle::block_on` against the
215    /// front's captured runtime handle — poco-findings §trait posture).
216    /// Call it only from a blocking context (`tokio::task::spawn_blocking`
217    /// pipeline tasks); in async contexts it panics (the POC-O panic
218    /// shape — regression-tested) and [`ObjectFront::block_in_place`] is
219    /// the bridge instead.
220    pub fn block_on<F: std::future::Future>(&self, fut: F) -> F::Output {
221        self.runtime.block_on(fut)
222    }
223
224    /// The bridge's async-context half (the POC-O liveness-source
225    /// posture): parks the worker and serves the store call through the
226    /// blocking pool, making the bridge legal on an async worker.
227    /// Requires a multi-thread runtime.
228    pub fn block_in_place<F>(&self, fut: F) -> F::Output
229    where
230        F: std::future::Future + Send,
231        F::Output: Send,
232    {
233        tokio::task::block_in_place(move || self.runtime.block_on(fut))
234    }
235
236    /// The write leg: put one loose object under the family, riding the
237    /// known-length derivation seam
238    /// (`put_as(Algorithm::GitObjectSha1, len, stream)`) with the
239    /// preimage length declared. Verification-on-put is the oid
240    /// re-derivation: the store derives the digest from the stored
241    /// bytes, and the front rejects a put whose returned key does not
242    /// re-derive to the object's oid — corruption cannot enter.
243    ///
244    /// Returns the family key and the put's pin guard (the wave-8 pin
245    /// bridge holds these across the ingest → CAS span; dropping the
246    /// guard releases the entry to the sweep window).
247    pub async fn put(
248        &self,
249        kind: Kind,
250        content: &[u8],
251    ) -> Result<(alkblobs::Key, Pin), StorageError> {
252        let form = loose_form(kind, content);
253        let (key, pin) = self
254            .store
255            .put_as(
256                GIT_OBJECT_ALGORITHM,
257                Some(form.len() as u64),
258                std::io::Cursor::new(&form),
259            )
260            .await
261            .map_err(|e| store_leg_error("alkblobs put_as", e))?;
262        let oid = key_to_oid(&key).ok_or_else(|| {
263            StorageError::Invalid(format!(
264                "the put returned key {:?} is not the sha1 git-object family",
265                key.digest().bytes()
266            ))
267        })?;
268        let derived = GIT_OBJECT_ALGORITHM.derive(&form, form.len() as u64);
269        if oid.as_bytes() != derived.bytes() {
270            return Err(StorageError::Invalid(format!(
271                "the put returned key {oid:?} does not re-derive to the object's oid"
272            )));
273        }
274        Ok((key, pin))
275    }
276
277    /// The read leg: the loose object behind `oid`, its kind decoded
278    /// from the header the odb layer owns. Verification-on-get is the
279    /// oid re-derivation run over the delivered bytes (the store's kv
280    /// tier verifies at open; the front re-verifies the full value so
281    /// large-tier deliveries — not re-derived there — fail the read
282    /// too): a corrupted entry is `StorageError::Invalid`.
283    pub async fn get(&self, oid: &gix_hash::oid) -> Result<Option<(Kind, Vec<u8>)>, StorageError> {
284        let key = oid_key(oid)?;
285        let Some((len, blob)) = self
286            .store
287            .get(&key.as_bytes())
288            .await
289            .map_err(|e| store_leg_error("alkblobs get", e))?
290        else {
291            return Ok(None);
292        };
293        let value = blob
294            .read_at(0, len)
295            .await
296            .map_err(|e| store_leg_error("alkblobs read_at", e))?;
297        let (kind, header_len) = parse_loose_form(&value)?;
298        let derived = GIT_OBJECT_ALGORITHM.derive(&value, value.len() as u64);
299        if oid.as_bytes() != derived.bytes() {
300            return Err(StorageError::Invalid(format!(
301                "corruption detected at {oid}: the delivered value does not re-derive to the oid"
302            )));
303        }
304        Ok(Some((kind, value[header_len..].to_vec())))
305    }
306
307    /// The object's kind behind `oid`, read from the loose header the
308    /// odb layer owns — a bounded prefix read (`gix_object`'s
309    /// `loose_header` parses
310    /// a scan-to-NUL header; the body is never touched). `None` is a
311    /// definitive miss. The kv tier's own get-time verification still
312    /// fails a corrupted entry (mapped `Invalid`); the large tier's
313    /// header parse is shape-level — body corruption fails the serving
314    /// reads that consume the body instead.
315    pub async fn kind(&self, oid: &gix_hash::oid) -> Result<Option<Kind>, StorageError> {
316        let key = oid_key(oid)?;
317        let Some((len, blob)) = self
318            .store
319            .get(&key.as_bytes())
320            .await
321            .map_err(|e| store_leg_error("alkblobs get", e))?
322        else {
323            return Ok(None);
324        };
325        let peeked = blob
326            .read_at(0, LOOSE_HEADER_PEEK_MAX.min(len))
327            .await
328            .map_err(|e| store_leg_error("alkblobs header peek", e))?;
329        let (kind, _size, _header_len) =
330            gix_object::decode::loose_header(&peeked).map_err(|e| {
331                StorageError::Invalid(format!(
332                    "the stored loose-object form's header does not parse: {e}"
333                ))
334            })?;
335        Ok(Some(kind))
336    }
337
338    /// The stat leg: the family entry's metadata — `len` is the
339    /// *preimage* length (the stored loose form, git's header included),
340    /// the family semantic (alkblobs ADR-018 §Decision 3), never the
341    /// content length. `None` is a definitive miss.
342    pub async fn stat(&self, oid: &gix_hash::oid) -> Result<Option<EntryMeta>, StorageError> {
343        let key = oid_key(oid)?;
344        let meta = self
345            .store
346            .stat(&key.as_bytes())
347            .await
348            .map_err(|e| store_leg_error("alkblobs stat", e))?;
349        match meta {
350            Some(meta) if meta.algorithm == GIT_OBJECT_ALGORITHM => Ok(Some(meta)),
351            Some(meta) => Err(StorageError::Invalid(format!(
352                "the family entry behind {oid} carries algorithm {:?}, not the sha1 git-object family",
353                meta.algorithm
354            ))),
355            None => Ok(None),
356        }
357    }
358}
359
360/// A per-session view of the [`ObjectFront`]: the object lookup Find
361/// the pack pipelines consume, carrying the session's lookup-miss slot
362/// (one session's failed walk cannot leak another's miss — mint one
363/// with [`ObjectFront::session`]). Clones share the slot, the shape the
364/// pipeline's `find.clone()` calls assume within one session.
365///
366/// A lookup miss is recorded — the last missed oid — so a failed
367/// walk/count stage maps its not-found error back to the seam's
368/// `StorageError::ObjectMissing` abort ([`SessionFind::missed_object`];
369/// the `MemFind` double's shape). Misses stay `Ok(None)`; corrupt
370/// entries and backing failures surface as `Err`.
371#[derive(Clone)]
372pub struct SessionFind {
373    front: ObjectFront,
374    missed: Arc<parking_lot::Mutex<Option<ObjectId>>>,
375}
376
377impl SessionFind {
378    /// The oid recorded by the latest lookup miss, if any (the recorded
379    /// id names the missing object rather than the want).
380    pub fn missed_object(&self) -> Option<ObjectId> {
381        self.missed.lock().take()
382    }
383}
384
385fn find_error(leg: StorageError) -> gix_error::Error {
386    match leg {
387        StorageError::ObjectMissing { oid } => {
388            gix_error::message!("substrate odb: object {oid} does not exist").not_found_error()
389        }
390        StorageError::Invalid(_) => gix_error::message!("substrate odb: {leg}").validation_error(),
391        StorageError::Io(_) => gix_error::message!("substrate odb: {leg}").retryable_error(),
392    }
393}
394
395impl gix_object::Exists for SessionFind {
396    fn exists(&self, id: &gix_hash::oid) -> bool {
397        <Self as gix_pack::Find>::contains(self, id)
398    }
399}
400
401impl gix_object::Find for SessionFind {
402    fn try_find<'a>(
403        &self,
404        id: &gix_hash::oid,
405        buffer: &'a mut Vec<u8>,
406    ) -> gix_error::Result<Option<gix_object::Data<'a>>> {
407        let find = <Self as gix_pack::Find>::try_find_cached(
408            self,
409            id,
410            buffer,
411            &mut gix_pack::cache::Never,
412        )?;
413        Ok(find.map(|(data, _location)| data))
414    }
415}
416
417impl gix_pack::Find for SessionFind {
418    fn contains(&self, id: &gix_hash::oid) -> bool {
419        let oid: ObjectId = (id as &gix_hash::oid).to_owned();
420        let Ok(key) = oid_key(&oid) else {
421            return false;
422        };
423        self.front
424            .block_on(self.front.store.has(&key.as_bytes()))
425            .unwrap_or(false)
426    }
427
428    fn try_find_cached<'a>(
429        &self,
430        id: &gix_hash::oid,
431        buffer: &'a mut Vec<u8>,
432        _pack_cache: &mut dyn gix_pack::cache::DecodeEntry,
433    ) -> gix_error::Result<
434        Option<(
435            gix_object::Data<'a>,
436            Option<gix_pack::data::entry::Location>,
437        )>,
438    > {
439        let oid: ObjectId = (id as &gix_hash::oid).to_owned();
440        let found = self.front.block_on(self.front.get(&oid));
441        match found {
442            Ok(None) => {
443                self.missed.lock().replace(oid);
444                Ok(None)
445            }
446            Ok(Some((kind, data))) => {
447                buffer.clear();
448                buffer.extend_from_slice(&data);
449                Ok(Some((
450                    gix_object::Data {
451                        kind,
452                        object_hash: id.kind(),
453                        data: buffer,
454                    },
455                    None,
456                )))
457            }
458            Err(leg) => Err(find_error(leg)),
459        }
460    }
461
462    fn location_by_oid(
463        &self,
464        _id: &gix_hash::oid,
465        _buf: &mut Vec<u8>,
466    ) -> gix_error::Result<Option<gix_pack::data::entry::Location>> {
467        Ok(None)
468    }
469
470    fn pack_offsets_and_oid(
471        &self,
472        _pack_id: u32,
473    ) -> gix_error::Result<Option<Vec<(gix_pack::data::Offset, ObjectId)>>> {
474        Ok(None)
475    }
476
477    fn entry_by_location(
478        &self,
479        _location: &gix_pack::data::entry::Location,
480    ) -> Option<gix_pack::find::Entry> {
481        None
482    }
483}
484
485/// The header parser: back out the kind and the body offset, and
486/// validate that the body's length matches the header's declared length
487/// — the odb layer's parse-back of the header it owns.
488fn parse_loose_form(value: &[u8]) -> Result<(Kind, usize), StorageError> {
489    let (kind, size, header_len) = gix_object::decode::loose_header(value).map_err(|e| {
490        StorageError::Invalid(format!("the stored loose-object form does not parse: {e}"))
491    })?;
492    let body = value.len().saturating_sub(header_len) as u64;
493    if body != size {
494        return Err(StorageError::Invalid(format!(
495            "the stored loose-object form's body is {body} bytes against a header declaring {size}"
496        )));
497    }
498    Ok((kind, header_len))
499}
500#[cfg(test)]
501mod tests {
502    use alkblobs::{Store, StoreMode};
503    use gix_hash::ObjectId;
504    use gix_object::Kind;
505    use std::sync::atomic::AtomicBool;
506    use std::sync::Arc;
507
508    use super::{key_to_oid, loose_form, oid_key, ObjectFront, SessionFind};
509    use crate::testutil::scratch_dir as scratch_root;
510    use alkgit::StorageError;
511
512    fn mem_front() -> ObjectFront {
513        ObjectFront::new(Arc::new(Store::mem_only()))
514    }
515
516    async fn put_object(front: &ObjectFront, kind: Kind, content: &[u8]) -> ObjectId {
517        let (key, _pin) = front.put(kind, content).await.unwrap();
518        key_to_oid(&key).expect("the put's key is the family key")
519    }
520
521    fn blob_content() -> &'static [u8] {
522        b"hello\n"
523    }
524
525    fn tree_bytes(blob: &ObjectId) -> Vec<u8> {
526        let mut tree = b"100644 hello\0".to_vec();
527        tree.extend_from_slice(blob.as_bytes());
528        tree
529    }
530
531    fn commit_bytes(tree: &ObjectId, parents: &[&ObjectId]) -> Vec<u8> {
532        let mut body = format!("tree {tree}\n");
533        for parent in parents {
534            body.push_str(&format!("parent {parent}\n"));
535        }
536        body.push_str(
537            "author A <a@example.invalid> 1750000000 +0000\n\
538             committer A <a@example.invalid> 1750000000 +0000\n\n",
539        );
540        body.into_bytes()
541    }
542
543    async fn fixture_front() -> (ObjectFront, ObjectId) {
544        let front = mem_front();
545        let blob = put_object(&front, Kind::Blob, blob_content()).await;
546        let tree = front
547            .put(Kind::Tree, &tree_bytes(&blob))
548            .await
549            .map(|(key, _)| key_to_oid(&key).unwrap())
550            .unwrap();
551        let base = front
552            .put(Kind::Commit, &commit_bytes(&tree, &[]))
553            .await
554            .map(|(key, _)| key_to_oid(&key).unwrap())
555            .unwrap();
556        let tip = front
557            .put(Kind::Commit, &commit_bytes(&tree, &[&base]))
558            .await
559            .map(|(key, _)| key_to_oid(&key).unwrap())
560            .unwrap();
561        (front, tip)
562    }
563
564    #[tokio::test]
565    async fn put_get_round_trips_through_the_family() {
566        let front = mem_front();
567        for (kind, content) in [
568            (Kind::Blob, b"hello\n".as_slice()),
569            (Kind::Blob, b"".as_slice()),
570            (Kind::Blob, vec![0u8, 255, 0, 10, 13, 255].as_slice()),
571        ] {
572            let (key, _pin) = front.put(kind, content).await.unwrap();
573            assert_eq!(
574                key.algorithm(),
575                alkblobs::Algorithm::GitObjectSha1,
576                "the family byte the front writes is the sha1 git-object member"
577            );
578            let (got_kind, got) = front
579                .get(&key_to_oid(&key).unwrap())
580                .await
581                .unwrap()
582                .unwrap();
583            assert_eq!(got_kind, kind);
584            assert_eq!(got, content);
585        }
586
587        let blob = put_object(&front, Kind::Blob, b"hello\n").await;
588        let tree = put_object(&front, Kind::Tree, &tree_bytes(&blob)).await;
589        let commit = put_object(&front, Kind::Commit, &commit_bytes(&tree, &[])).await;
590        let (got_kind, got) = front.get(&tree).await.unwrap().unwrap();
591        assert_eq!(got_kind, Kind::Tree);
592        assert_eq!(got, tree_bytes(&blob));
593        let (got_kind, got) = front.get(&commit).await.unwrap().unwrap();
594        assert_eq!(got_kind, Kind::Commit);
595        assert_eq!(got, commit_bytes(&tree, &[]));
596    }
597
598    /// The load-bearing interop shape (alkblobs' `git_interop.rs`
599    /// standard): the put's re-derived oid is byte-exact git's
600    /// hash-object oid — hardcoded vectors plus, when a `git` is
601    /// installed, a CLI cross-check in a detached scratch repo.
602    #[tokio::test]
603    async fn the_loose_preimage_hashes_to_git_s_oid() {
604        let front = mem_front();
605        let vectors: &[(&[u8], Kind, &str)] = &[
606            (b"", Kind::Blob, "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"),
607            (
608                b"hello world\n",
609                Kind::Blob,
610                "3b18e512dba79e4c8300dd08aeb37f8e728b8dad",
611            ),
612            (b"", Kind::Tree, "4b825dc642cb6eb9a060e54bf8d69288fbee4904"),
613        ];
614        for (content, kind, oid_hex) in vectors {
615            let oid = put_object(&front, *kind, content).await;
616            assert_eq!(
617                oid.to_string(),
618                *oid_hex,
619                "the derived oid is git's hash-object oid"
620            );
621        }
622    }
623
624    /// The CLI half of the interop shape (alkblobs' `git_interop.rs`
625    /// harness detail): a real `git hash-object --stdin` in a detached
626    /// scratch repo — skipped with a message when no `git` is installed.
627    #[tokio::test]
628    async fn the_put_oid_matches_the_git_cli() {
629        if std::process::Command::new("git")
630            .arg("--version")
631            .output()
632            .is_err()
633        {
634            eprintln!("skipping (no git binary installed)");
635            return;
636        }
637        let dir = scratch_root("odb", "cli");
638        let dir = dir.display().to_string();
639        let out = std::process::Command::new("git")
640            .args(["init", "-q"])
641            .current_dir(&dir)
642            .env("GIT_CONFIG_GLOBAL", "/dev/null")
643            .env("GIT_CONFIG_SYSTEM", "/dev/null")
644            .output()
645            .unwrap();
646        assert!(out.status.success(), "git init failed");
647        for content in [b"hello\n".as_slice(), b"hello world\n".as_slice()] {
648            let mut child = std::process::Command::new("git")
649                .current_dir(&dir)
650                .args(["hash-object", "--stdin"])
651                .env("GIT_CONFIG_GLOBAL", "/dev/null")
652                .env("GIT_CONFIG_SYSTEM", "/dev/null")
653                .stdin(std::process::Stdio::piped())
654                .stdout(std::process::Stdio::piped())
655                .spawn()
656                .unwrap();
657            use std::io::Write as _;
658            child.stdin.as_mut().unwrap().write_all(content).unwrap();
659            let out = child.wait_with_output().unwrap();
660            assert!(out.status.success(), "git hash-object failed");
661            let oid_hex = String::from_utf8(out.stdout).unwrap().trim().to_owned();
662
663            let front = mem_front();
664            let oid = put_object(&front, Kind::Blob, content).await;
665            assert_eq!(oid.to_string(), oid_hex, "the put's oid matches git's");
666        }
667        std::fs::remove_dir_all(&dir).unwrap_or_default();
668    }
669
670    /// `stat` returns the preimage length — the family semantic
671    /// (alkblobs ADR-018 §Decision 3): the stored value's length, git's
672    /// header included, never the content length.
673    #[tokio::test]
674    async fn stat_reports_the_preimage_length() {
675        let front = mem_front();
676        let content = b"hello".as_slice();
677        let oid = put_object(&front, Kind::Blob, content).await;
678        let form = loose_form(Kind::Blob, content);
679        assert_ne!(form.len(), content.len(), "the header is inside the value");
680        let meta = front.stat(&oid).await.unwrap().expect("the entry stats");
681        assert_eq!(meta.len, form.len() as u64, "stat is the preimage length");
682        assert_eq!(meta.len, 12, "header (7) + content (5) = the preimage");
683        assert_eq!(
684            meta.algorithm,
685            alkblobs::Algorithm::GitObjectSha1,
686            "stat's algorithm is the sha1 family member"
687        );
688
689        let missing = ObjectId::from([7u8; 20]);
690        assert!(
691            front.stat(&missing).await.unwrap().is_none(),
692            "stat is a definitive miss, never a guessed zero"
693        );
694    }
695
696    /// Verification-on-get over the kv tier: a corrupted stored value
697    /// fails the read (the store derives at open; the front's own
698    /// re-derivation backs it), riding the seeded sqlite store's `blobs`
699    /// rows tampered in place.
700    #[cfg(feature = "sqlite")]
701    #[tokio::test]
702    async fn corrupt_kvtier_entry_fails_the_read() {
703        let dir = scratch_root("odb", "kv-corrupt");
704        let db = dir.join("blobs.sqlite3");
705        let store = Arc::new(
706            Store::kv_only_sqlite(&db, 128 * 1024, alkblobs::DEFAULT_DELETE_WINDOW_DELAY)
707                .await
708                .unwrap(),
709        );
710        let front = ObjectFront::new(store);
711        let oid = put_object(&front, Kind::Blob, blob_content()).await;
712        assert!(front.get(&oid).await.is_ok(), "the pristine entry reads");
713
714        let key = oid_key(&oid).unwrap();
715        let conn = rusqlite::Connection::open(&db).unwrap();
716        conn.execute_batch("PRAGMA journal_mode = WAL;").unwrap();
717        let value = conn
718            .query_row(
719                "SELECT value FROM blobs WHERE key = ?",
720                [key.as_bytes()],
721                |row| row.get::<_, Vec<u8>>(0),
722            )
723            .unwrap();
724        assert_eq!(
725            value.len(),
726            loose_form(Kind::Blob, blob_content()).len(),
727            "the preimage is in the row"
728        );
729        let mut corrupted = value.clone();
730        corrupted[2] = b'9';
731        conn.execute(
732            "UPDATE blobs SET value = ? WHERE key = ?",
733            rusqlite::params![corrupted, key.as_bytes()],
734        )
735        .unwrap();
736
737        match front.get(&oid).await {
738            Err(StorageError::Invalid(reason)) => {
739                assert!(
740                    reason.contains("verification failed: content does not match key")
741                        || reason.contains("re-derive")
742                        || reason.contains("parse"),
743                    "the failure names the corruption, not a backing failure: {reason}"
744                );
745            }
746            Err(e) => panic!("corruption is a validation failure, not {e}"),
747            Ok(_) => panic!("a corrupted entry cannot read back"),
748        }
749        let find = front.session();
750        assert!(
751            tokio::task::spawn_blocking(move || {
752                gix_object::Find::try_find(&find, oid.as_ref(), &mut Vec::new()).is_err()
753            })
754            .await
755            .unwrap(),
756            "the Find leg fails the corrupted entry too"
757        );
758        std::fs::remove_dir_all(&dir).unwrap_or_default();
759    }
760
761    /// The front's own re-derivation: the large tier does not verify at
762    /// open, so a corrupted over-threshold entry is caught by the front's
763    /// oid re-derivation of the delivered bytes, not by the store.
764    #[tokio::test]
765    async fn corrupt_largetier_entry_fails_the_read() {
766        let dir = scratch_root("odb", "large-corrupt");
767        let store = Arc::new(Store::dual_tier_mem_local(&dir, 4).await.unwrap());
768        const THRESHOLD: u64 = 4;
769        assert_eq!(store.mode(), StoreMode::DualTier, "the large tier is live");
770        let form_threshold = loose_form(Kind::Blob, b"!").len() as u64;
771        assert!(form_threshold > THRESHOLD, "any loose form routes large");
772
773        let front = ObjectFront::new(store);
774        let oid = put_object(&front, Kind::Blob, blob_content()).await;
775        assert!(front.get(&oid).await.is_ok(), "the pristine entry reads");
776
777        let key = oid_key(&oid).unwrap();
778        let hex = hex::encode(key.as_bytes());
779        let entry_path = dir.join(&hex[..2]).join(&hex[2..4]).join(&hex);
780        assert!(entry_path.is_file(), "the large tier's committed file");
781        let mut value = std::fs::read(&entry_path).unwrap();
782        let last = value.len() - 1;
783        value[last] ^= 0xff;
784        std::fs::write(&entry_path, &value).unwrap();
785
786        match front.get(&oid).await {
787            Err(StorageError::Invalid(reason)) => {
788                assert!(
789                    reason.contains("re-derive"),
790                    "the failure names the corruption, not a backing failure: {reason}"
791                );
792            }
793            Err(e) => panic!("corruption is a validation failure, not {e}"),
794            Ok(_) => panic!("a corrupted entry cannot read back"),
795        }
796        std::fs::remove_dir_all(&dir).unwrap_or_default();
797    }
798
799    /// The kind leg: the header-derived kind without the body — every
800    /// kind round-trips, a miss is `None`, and the peek never reads the
801    /// content (a large-tier entry past the peek window still answers).
802    #[tokio::test(flavor = "multi_thread")]
803    async fn the_kind_leg_answer_from_the_header_not_the_body() {
804        let dir = scratch_root("odb", "kind-leg");
805        let store = Arc::new(Store::dual_tier_mem_local(&dir, 4).await.unwrap());
806        let front = ObjectFront::new(store);
807        for (kind, content) in [
808            (Kind::Blob, b"hello\n".as_slice()),
809            (Kind::Tree, tree_bytes(&ObjectId::from([3u8; 20])).as_slice()),
810            (Kind::Commit, commit_bytes(&ObjectId::from([3u8; 20]), &[]).as_slice()),
811            (Kind::Tag, b"object \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\ntype commit\ntag v\ntagger A <a@example.invalid> 1750000000 +0000\n\nm\n".as_slice()),
812        ] {
813            let oid = put_object(&front, kind, content).await;
814            assert_eq!(front.kind(&oid).await.unwrap(), Some(kind), "{kind:?}");
815        }
816
817        let missing = ObjectId::from([0xE2u8; 20]);
818        assert!(
819            front.kind(&missing).await.unwrap().is_none(),
820            "a miss is None"
821        );
822
823        // A kind answer does not depend on the body: the peek window is
824        // orders below a large-tier object's size and still answers.
825        let big: Vec<u8> = std::iter::repeat_n(b'x', 4096).collect();
826        let oid = put_object(&front, Kind::Blob, &big).await;
827        assert_eq!(front.kind(&oid).await.unwrap(), Some(Kind::Blob));
828        std::fs::remove_dir_all(&dir).unwrap_or_default();
829    }
830
831    /// OQ-05's posture at the front: every serving-path oid is sha1, the
832    /// `0x04` family byte stays untouched, and every address the front
833    /// builds decodes family-explicitly — never guessed.
834    #[tokio::test]
835    async fn the_front_serves_only_the_sha1_family() {
836        let front = mem_front();
837        put_object(&front, Kind::Blob, b"one\n").await;
838        put_object(&front, Kind::Blob, b"two\n").await;
839
840        for key in front.store().list().await.unwrap() {
841            assert_eq!(key.algorithm(), alkblobs::Algorithm::GitObjectSha1);
842            assert_eq!(key.as_bytes()[0], 0x03, "the sha1 family byte");
843        }
844        let (key, _pin) = front.put(Kind::Blob, b"three\n").await.unwrap();
845        assert_eq!(
846            key.as_bytes()[0],
847            0x03,
848            "the sha256 member (0x04) is never written"
849        );
850    }
851
852    /// The three trait impls over the front — `gix_object::Find` +
853    /// `Exists` + `gix_pack::Find` — with the location legs answering
854    /// "not in any pack" (the `MemFind` double's exact shape). The sync
855    /// Find legs run in a blocking task — the pipeline posture; from an
856    /// async context the bridge panics.
857    #[tokio::test]
858    async fn the_find_legs_answer_with_no_packed_objects() {
859        let (front, _tip) = fixture_front().await;
860        let blob = put_object(&front, Kind::Blob, blob_content()).await;
861        let find = front.session();
862
863        tokio::task::spawn_blocking(move || {
864            assert!(gix_object::Exists::exists(&find, blob.as_ref()));
865            let missing = ObjectId::from([9u8; 20]);
866            assert!(!gix_object::Exists::exists(&find, missing.as_ref()));
867
868            let mut buffer = Vec::new();
869            let data = gix_object::Find::try_find(&find, blob.as_ref(), &mut buffer)
870                .unwrap()
871                .expect("the blob is found");
872            assert_eq!(data.kind, Kind::Blob);
873            assert_eq!(data.data, blob_content());
874            assert_eq!(data.object_hash, gix_hash::Kind::Sha1);
875
876            use gix_object::FindExt as _;
877            let mut buffer = Vec::new();
878            let blob_ref = find.find_blob(blob.as_ref(), &mut buffer).unwrap();
879            assert_eq!(
880                blob_ref.data,
881                blob_content(),
882                "the FindExt blanket finds too"
883            );
884
885            use gix_pack::Find as _;
886            let id = blob.as_ref() as &gix_hash::oid;
887            assert!(
888                find.contains(id),
889                "the pack-level contains is the same lookup"
890            );
891            let locate = find.location_by_oid(id, &mut Vec::new()).unwrap();
892            assert!(locate.is_none(), "no pack holds the object");
893            let offsets = find.pack_offsets_and_oid(0).unwrap();
894            assert!(offsets.is_none(), "no pack exists");
895            let mut buffer = Vec::new();
896            let resolved = find.try_find(id, &mut buffer).unwrap().expect("found");
897            assert!(
898                resolved.1.is_none(),
899                "the location leg is none even on a resolved object"
900            );
901        })
902        .await
903        .unwrap();
904    }
905
906    /// `entry_by_location` answers none for every location shape — no
907    /// packed objects anywhere on the serving path (no disk odb, gix-odb
908    /// is deliberately absent from the manifest).
909    #[tokio::test]
910    async fn entry_by_location_is_unanswered() {
911        let (front, _tip) = fixture_front().await;
912        let find = front.session();
913        let location = gix_pack::data::entry::Location {
914            pack_id: 0,
915            entry_size: 1,
916            pack_offset: 0,
917        };
918        assert!(gix_pack::Find::entry_by_location(&find, &location).is_none());
919    }
920
921    /// Misses record per session: one session's failed walk cannot
922    /// contaminate another's, the recorded id names the missing object
923    /// (not the want), and the slot drains on take.
924    #[tokio::test]
925    async fn lookup_misses_record_per_session() {
926        let (front, _tip) = fixture_front().await;
927        let missing = ObjectId::from([11u8; 20]);
928        let blob = put_object(&front, Kind::Blob, b"again\n").await;
929
930        tokio::task::spawn_blocking(move || {
931            let missing_id = missing.as_ref() as &gix_hash::oid;
932
933            let one = front.session();
934            let two = front.session();
935            let mut buffer = Vec::new();
936            assert_eq!(
937                gix_object::Find::try_find(&one, missing_id, &mut buffer).unwrap(),
938                None,
939                "a miss is a recorded result, not an error"
940            );
941            assert_eq!(one.missed_object(), Some(missing));
942            assert!(two.missed_object().is_none(), "the miss is session-private");
943            assert!(one.missed_object().is_none(), "the slot drains on take");
944
945            let find = front.session();
946            assert!(
947                gix_object::Find::try_find(&find, missing.as_ref(), &mut Vec::new())
948                    .unwrap()
949                    .is_none()
950            );
951            assert_eq!(find.missed_object(), Some(missing));
952            assert!(
953                gix_object::Find::try_find(&find, blob.as_ref(), &mut Vec::new())
954                    .unwrap()
955                    .is_some(),
956                "a real lookup still succeeds after a recorded miss"
957            );
958        })
959        .await
960        .unwrap();
961    }
962
963    /// The POC-O panic shape, regression-tested: the blocking bridge
964    /// (`Handle::block_on`) panics in async contexts — never call it
965    /// there; `block_in_place` is the bridge instead.
966    #[tokio::test]
967    #[should_panic]
968    async fn the_blocking_bridge_panics_in_async_contexts() {
969        let (front, tip) = fixture_front().await;
970        let _ = front.block_on(front.get(&tip));
971    }
972
973    /// The bridge's async-context half: `block_in_place` parks the
974    /// worker and serves the store call through the blocking pool
975    /// (multi-thread runtimes only) — the liveness-source posture.
976    #[tokio::test(flavor = "multi_thread")]
977    async fn the_async_context_bridges_with_block_in_place() {
978        let (front, tip) = fixture_front().await;
979        assert!(front.block_in_place(front.get(&tip)).unwrap().is_some());
980    }
981
982    type PipelineResult = Result<(Vec<u8>, gix_hash::ObjectId, u32), StorageError>;
983
984    fn pipeline_error(find: &SessionFind, e: gix_error::Error, fallback: ObjectId) -> StorageError {
985        if e.is_not_found() {
986            StorageError::ObjectMissing {
987                oid: find.missed_object().unwrap_or(fallback),
988            }
989        } else {
990            StorageError::Io(e.to_string())
991        }
992    }
993
994    fn closure_of(find: &SessionFind, tip: ObjectId) -> Result<Vec<ObjectId>, StorageError> {
995        let mut closure = Vec::new();
996        let mut stack = vec![tip];
997        while let Some(current) = stack.pop() {
998            if closure.contains(&current) {
999                continue;
1000            }
1001            let mut buf = Vec::new();
1002            let commit = gix_object::FindExt::find_commit(find, &current, &mut buf)
1003                .map_err(|e| pipeline_error(find, e, tip))?;
1004            for parent in &commit.parents {
1005                let parent =
1006                    gix_hash::ObjectId::from_hex(parent).expect("the stored parent is hex");
1007                stack.push(parent);
1008            }
1009            closure.push(current);
1010        }
1011        Ok(closure)
1012    }
1013
1014    /// The generation-pipeline smoke over the front (ADR-004's Find-
1015    /// generic stages): the commit-ancestry closure rides `FindExt`,
1016    /// the count stage runs over the pack Find, and the entry/byte
1017    /// stages stream a V2 sha1 pack through the front's lookup legs.
1018    #[tokio::test]
1019    async fn the_generation_pipeline_smokes_over_the_front() {
1020        let (front, tip) = fixture_front().await;
1021        let find = front.session();
1022        let pack = tokio::task::spawn_blocking(move || smoke_pack(&find, tip))
1023            .await
1024            .unwrap();
1025        let (pack_bytes, digest, _objects) =
1026            pack.unwrap_or_else(|e| panic!("the pipeline packs over the front: {e}"));
1027        assert!(pack_bytes.starts_with(b"PACK"), "the pack header wrote");
1028        let trailing = &pack_bytes[pack_bytes.len() - 20..];
1029        assert_eq!(
1030            trailing,
1031            digest.as_slice(),
1032            "the trailer is the pipeline's digest"
1033        );
1034    }
1035
1036    fn smoke_pack(find: &SessionFind, want: ObjectId) -> PipelineResult {
1037        use gix_pack::data::output::bytes::FromEntriesIter;
1038        use gix_pack::data::output::count;
1039        use gix_pack::data::output::entry::iter_from_counts;
1040        use gix_pack::data::output::entry::iter_from_counts::{Mode, Options, Outcome};
1041        use gix_parallel::Finalize as _;
1042
1043        let commits = closure_of(find, want)?;
1044        let counter = gix_utils::progress::Discard;
1045        let interrupt = AtomicBool::new(false);
1046        let (counts, _outcome) = count::objects_unthreaded(
1047            find,
1048            &mut commits.into_iter().map(Ok::<ObjectId, gix_error::Error>),
1049            &counter,
1050            &interrupt,
1051            count::objects::ObjectExpansion::TreeContents,
1052        )
1053        .map_err(|e| pipeline_error(find, e, want))?;
1054        let counts_len = counts.len() as u32;
1055
1056        let sink = Vec::<u8>::new();
1057        let mut entries = gix_parallel::InOrderIter::from(
1058            iter_from_counts(
1059                counts,
1060                find.clone(),
1061                Box::new(gix_utils::progress::Discard),
1062                Options {
1063                    thread_limit: Some(1),
1064                    mode: Mode::PackCopyAndBaseObjects,
1065                    allow_thin_pack: false,
1066                    chunk_size: 10,
1067                    version: gix_pack::data::Version::V2,
1068                    compression: gix_zlib::Compression::DEFAULT,
1069                },
1070            )
1071            .map_err(|e| pipeline_error(find, e, want))?,
1072        );
1073        let mut from_entries = FromEntriesIter::new(
1074            entries.by_ref(),
1075            sink,
1076            counts_len,
1077            gix_pack::data::Version::V2,
1078            gix_hash::Kind::Sha1,
1079        );
1080        for step in from_entries.by_ref() {
1081            step.map_err(|e| pipeline_error(find, e, want))?;
1082        }
1083        let digest = from_entries
1084            .digest()
1085            .ok_or_else(|| StorageError::Io("the trailer was never written".to_string()))?;
1086        let pack_bytes = from_entries.into_write();
1087        let statistics: Outcome = entries
1088            .inner
1089            .finalize()
1090            .map_err(|e| pipeline_error(find, e, want))?;
1091        assert_eq!(
1092            statistics.missing_objects, 0,
1093            "ADR-004's abort posture: no missing objects in the smoke"
1094        );
1095        Ok((pack_bytes, digest, counts_len))
1096    }
1097}