See manifest.sql for the manifest DB schema.
DB instance are identified by their dir path.
DB instance clients have:
* Lazily created exclusive files for writing new values in parallel without locking the manifest.
* A sqlite3 connection to the manifest file.
* A cache of exclusive files ready for writing
* A cache of file clones, invalidated as necessary
A snapshot has:
* One or more cloned value files.
write values for keys:
* append to exclusive file
* hash parts? (not implemented)
* take exclusive write lock on manifest
* add entries to manifest
* unlock manifest
* punching blocks from new writes that are duplicates
* evict and punch holes until size below max
for a read:
* open manifest for deferred write (in case we abort the read, or the values we want are missing)
* for each read key, update the last_used and copy out the value location
* clone any files that contain regions to be streamed out
* unlock manifest
* return snapshots
when a snapshot is closed:
drop ref to cloned value files, deleting on the last one (or maybe they're anonymous to begin with, unlinked after the clone completes)
when evicting:
open deferred write transaction on manifest
while manifest page_count*page_size + sum(value_length) from keys >= capacity:
* delete least recently used item
how to do a singleflight fetch for a missing item:
* open manifest
* see a key is in an undesired state
* write a tag with an timestamp of when a fetch was started
* unlock manifest
* perform the fetch
* do a write, and clear the fetch state
advantages:
* you can bring your own files to the cache in their entirety and then have them punched out by the implementation as they decay in usefulness.