kcode-pending-object-store 0.1.0

Durable checksummed pending-object sidecar storage
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83

# kcode-pending-object-store Specification

## Scope

The crate owns durable installation, validation, reconciliation, and deletion
of checksummed pending-object sidecars. The caller owns directory creation,
position assignment, lifecycle policy, reference tracking, and concurrency
control.

## Configuration

A store is configured with a directory, namespace, and format ID. Construction
does not access or modify the filesystem.

The namespace is one to 255 ASCII bytes consisting only of alphanumeric
characters, `-`, and `_`. The format ID is nonempty and its UTF-8 encoding must
fit an unsigned 16-bit length.

## Files

Position `n` uses these same-directory paths:

```text
<namespace>-<n>.pending-object
<namespace>-<n>.pending-object.tmp
```

Recognition accepts only canonical decimal `u64` positions. Leading zeroes are
invalid except for `0`. Malformed names and names belonging to similarly
prefixed namespaces are not owned by the store.

A final file contains, in order:

1. magic bytes `KSPENDING01\n`;
2. little-endian `u16` format-ID byte length;
3. little-endian `u32` filename byte length;
4. little-endian `u32` media-type byte length;
5. little-endian `u64` object byte length;
6. 32-byte SHA-256 digest of the object bytes;
7. format-ID bytes;
8. UTF-8 filename bytes;
9. UTF-8 media-type bytes;
10. object bytes.

This encoding is byte-for-byte compatible with current
`kcode-session-log` pending-object sidecars when the namespace and format ID
match its session ID and format version.

## Installation and durability

Installation requires nonempty filename and media-type text and an existing
directory. It refuses to overwrite the canonical final path.

Before writing, installation removes only the exact temporary path for the
requested position. It creates a new temporary file, writes the complete
encoding, synchronizes the file, atomically renames it to the final path, and
synchronizes the directory before success.

## Reading and verification

Reading opens only the canonical final path for the requested position. It
rejects incorrect magic, a format ID different from the configured value,
length overflow, invalid UTF-8 metadata, empty metadata, any file length other
than the exactly declared length, and an object checksum mismatch.

`verify_all` reads every supplied position and performs no cleanup.

## Reconciliation and deletion

`reconcile` removes every recognized temporary file and every recognized final
file whose position is not referenced. It leaves malformed and similarly
prefixed names unchanged, synchronizes the directory if a removal occurred,
and then verifies every referenced position.

`delete_all` removes all recognized temporary and final files for the
namespace, leaves other names unchanged, and synchronizes the directory before
success.

## Concurrency boundary

The crate does not coordinate concurrent writers or protect against external
filesystem modification. Applications must serialize conflicting operations
and prevent unsupported cross-process mutation.