Crate loonfs_core

Expand description

Core LoonFS namespace operations.

loonfs-core is the low-level API for building directly on the LoonFS metadata protocol. Most callers should start with NamespaceEngine.

A namespace is one durable filesystem history. File bytes are written to object storage first, then metadata is published as a committed namespace mutation. Reads rebuild or reuse a verified view of the namespace before walking paths.

§Example

Commits are published as candidate batches through publish::NamespaceCommitEngine; day-to-day reads and writes should go through the loonfs crate’s FsReader/FsWriter handles, which wrap this crate with caching and batching.

use loonfs_api::{AbsolutePath, CommitId, NamespaceId};
use loonfs_core::publish::{
    FilesystemOperation, CommitRequest, NamespaceCommitEngine, CommitCandidate,
    PublishTailOptions,
};
use loonfs_core::{BootstrapOptions, MutationContext, NamespaceEngine};
use loonfs_objectstore::local_fs_store::LocalFsStore;

let store = LocalFsStore::new(std::env::temp_dir()).expect("store");
let namespace = NamespaceId::parse("docs").expect("valid namespace id");

let engine = NamespaceEngine::builder(store)
    .namespace_id(namespace.clone())
    .writer_id("example-writer")
    .build()
    .expect("engine");
let _ = engine.bootstrap_namespace(BootstrapOptions::default());

let publish_store = LocalFsStore::new(std::env::temp_dir()).expect("store");
let context = MutationContext {
    writer_id: "example-writer".to_owned(),
    now_ms: 0,
};
let mut publisher = NamespaceCommitEngine::new(namespace);
let _ = publisher.publish_batch(
    &publish_store,
    vec![CommitCandidate::new(CommitRequest::single(
        CommitId::generate(),
        None,
        FilesystemOperation::CreateDirectory {
            path: AbsolutePath::parse("/plans").expect("path"),
            parents: false,
        },
    ))],
    &context,
    &PublishTailOptions::default(),
);

Modules§

cache: Cache types and configuration for runtime read paths. Consumed by loonfs, which owns the runtime’s cache configuration and stats, and which re-exports cache::Recency for the grep index’s own block cache.
commit: Commit planning, validation, and materialization. Consumed by the loonfs publisher and by this crate’s commit-validation integration tests. Planned commits, from planner output to durable WAL frame.
content: Content staging and preparation-token minting. Consumed by loonfs’s write path and its server-integration content_tokens seam. Public facade over the durable content storage helpers.
control: Typed namespace control-object loaders and verified catalog state. Consumed by loonfs’s cache and write paths, and re-exported as loonfs::control for the white-box layout assertions the server and this crate’s own tests make.
limits: Protocol and resource ceilings. Consumed by loonfs (re-exported to the server for request validation) and by layout tests. One source of truth for publication, verification, provider, and garbage-collection timing bounds.
metadata: Durable metadata state and its row codecs. Public for this crate’s white-box integration tests, which compare projected state against the reference model; loonfs reaches metadata only through the seams above. Materialized namespace metadata: append-only row families plus derived indexes that answer reads over them.
path: Path parsing and current-state resolution. Consumed by loonfs’s write path (parse_mutation_path). Path parsing, reading, and mutation planning over namespace metadata.
publish: Commit publication types for runtime integrations. Consumed by loonfs’s publisher, and re-exported as loonfs::publish for the server’s filesystem handlers.
time: The wall-clock boundary durable timestamps are stamped at. Consumed by loonfs, whose mutation contexts and maintenance clock stamp from the same boundary this crate’s own commits do. The one wall-clock boundary durable timestamps are stamped at.

Structs§

BeginDirectMultipartUploadTargetResponse: Internal response for preparing a direct_multipart session.
BeginDirectPutUploadTargetResponse: Internal response for preparing a direct_put session before URL signing.
BootstrapOptions: Options for namespace bootstrap.
CheckpointFile: One file visible in the checkpointed state, with the content the checkpoint pinned for it.
CheckpointFilesPage: One page of the files a checkpoint pins.
CheckpointFilesPageCursor: Where a file enumeration resumes: strictly after this inode id.
CurrentFileState: What one inode looks like in the namespace’s current state.
DeleteNamespaceOptions: Options for namespace deletion.
DirectDownloadTarget: Where one file’s bytes actually live, for a host that is about to authorize a client to read them without passing them through itself.
DirectMultipartUploadTarget: Internal target used by server integrations before they mint part URLs.
DirectPutUploadTarget: Internal target used by server integrations before they mint a presigned URL.
FileContentStream: One file’s current content, read as fixed-size ranged chunks.
GcConfig: The grace window for the sweep (format spec, “Garbage collection”). It is wall-clock cleanup policy, never a validity input, and the default is conservative: every object gets one hour of unconditional protection. Abandoned fork records are not under it — a fork attempt carries its own lease, and letting that pass is the whole proof (gc/fork_checkpoints.rs) — and neither are upload sessions or the content they leave behind: a session carries its own lease, and the window a completed session’s content is protected for is derived in limits, not configured.
MetadataReorganizeReport
MultipartPartTarget: One part a server integration is about to sign.
MultipartPartTargets: Everything a server integration needs to sign one wave of part uploads.
MutationContext
NamespaceEngine: A namespace-scoped core API.
NamespaceEngineBuilder: Builder for NamespaceEngine.
PassBudget: The work one garbage-collection pass is allowed to do.
RuntimeReadContext: The pinned inputs the runtime resolves once per read: the head anchored to its manifest, the shared caches, and (when the runtime has it) the namespace’s immutable catalog pair.
WriterFence: The fencing event a writer session observed: the epoch the session held, the epoch that displaced it, and — when the head recorded a writer block — the winner’s label and when it acquired.

Enums§

AgedSweep: What aging one unreferenced candidate decided.
BootstrapNamespaceError
Error: Public error type returned by loonfs-core.
ErrorCode: Stable machine-readable error reason.
ErrorKind: Broad error category for caller or operator action.
MetadataProjectionLoadError: Failures while loading a bounded manifest-plus-tail metadata projection.
MetadataReorganizeOutcome: What one reorganization step did.
MetadataViewError: Failures specific to manifest-plus-tail metadata views.
NamespaceEngineBuildError: Error returned when a NamespaceEngine cannot be built.
StoreFailureClass: What a failed provider operation says about who can fix it, preserved across the message-flattening seams so the wire code can distinguish “fix the storage credentials” from “unclassified internal failure”.

Constants§

CONTENT_READ_CHUNK_BYTES: Bytes one ranged read of a content object fetches, and therefore the most of that object a streaming read holds at once.
MAX_RESOLVE_CURRENT_FILES: The most inode ids [resolve_current_files] answers in one call.

Functions§

delete_if_aged: Deletes one unreferenced candidate if the grace window has passed over it, and says what it decided otherwise.
gc_namespace