Expand description
S3-compatible protocol adapter for the Shardline server ecosystem.
This crate provides the S3 frontend’s protocol primitives:
- bucket (
{owner}.{name}) encoding and decoding and the storage object-key layout (protocols/s3/{scope_namespace}/{key}) —S3ObjectKey; - the S3 XML wire models and error envelope —
ListBucketResultandS3Error; - the SigV4 → bearer-token auth bridge and bucket-scope binding —
AuthSchemeandCredentialScope; - the query-param sub-resource dispatch set, ETag formatting, and byte-range
parsing —
S3SubResourceandparse_s3_range.
§Quick start
use shardline_s3_adapter::{decode_bucket, encode_bucket, s3_object_key};
// A bucket is the single dotted segment `{owner}.{name}` of the token scope.
let bucket = encode_bucket("acme", "models");
assert_eq!(
decode_bucket(&bucket)?,
("acme".to_owned(), "models".to_owned())
);
// Object keys live under the `protocols/s3/{scope_namespace}/` prefix.
let key = s3_object_key("global", "data/model.pt")?;
assert_eq!(key.as_str(), "protocols/s3/global/data/model.pt");
// Errors serialize to the S3 XML envelope with the correct status.
use axum::response::IntoResponse;
let response = shardline_s3_adapter::S3Error::no_such_key("data/model.pt")
.into_response();
assert_eq!(response.status(), 404);Structs§
- Complete
Multipart Upload Result - The
CompleteMultipartUploadresponse envelope. - Complete
Parts - The parsed
CompleteMultipartUploadrequest body. - Contents
- One listed object entry.
- Copy
Object Result - The
CopyObjectresponse envelope. - Copy
Source - A parsed
x-amz-copy-sourcevalue. - Credential
Scope - The five
/-separated segments of a SigV4Credentialscope. - Delimiter
- A validated
ListObjectsV2grouping delimiter (exactly one character). - Entity
Tag - An entity tag with its comparison strength preserved.
- Head
Object Headers - Response headers for
HeadObject. - Initiate
Multipart Upload Result - The
InitiateMultipartUploadresponse envelope. - Invalid
S3Header Value - A malformed S3 conditional-header (
If-Match/If-None-Match) orx-amz-copy-sourcevalue. - List
Bucket Result - The
ListObjectsV2response envelope. - List
Bucket Result V1 - The
ListObjects(v1) response envelope. - List
Buckets Result - The
ListAllMyBucketsResult(service-levelGET /) response envelope. - List
Objects V1Params - Parsed
ListObjects(v1) request parameters. - List
Objects V2Params - Parsed
ListObjectsV2request parameters. - List
Objects V2Response Context - Request values echoed by a ListObjectsV2 response. Filtering and cursors use raw values; encoding applies only when serializing the response.
- List
Page - One grouped page of a
ListObjectsV2listing. - Multipart
Part - One stored part of a multipart upload session.
- Multipart
Upload Session - A disk-persisted S3 multipart upload session.
- Part
Quota Limits - Aggregate and per-session limits for one staged part validation.
- PutObject
Response Headers - Response headers for a successful
PutObject. - Query
Credential - The presigned-URL
X-Amz-Credentialquery parameter. - S3Error
- A ready-to-send S3 error envelope.
- S3Error
Body - The S3 error envelope body (
<Error><Code>…</Code>…). - S3Object
Key - A validated client-facing S3 object key.
- S3Session
Part File Lock - Cross-process advisory lock protecting one multipart session’s part files.
- S3Upload
Session Lock - A held session-store advisory file lock.
- SigV4
Chunk Verifier - Checks the selected streaming mode against verified request headers. Buffers at most one 1 KiB framing line, independent of chunk or upload size. Signed trailer mode additionally checks declared checksums and their signature. Unsigned trailer mode checks checksums without a body MAC and requires TLS.
- SigV4
Credential - Signing material resolved by the server, never from untrusted request data.
- SigV4
Post Policy Auth - Authentication fields supplied by an AWS Signature Version 4 POST form.
- Verified
SigV4 - A verified request commitment, not proof that its body was received intact.
- Verified
SigV4 Post Policy - A verified SigV4 browser POST policy and its permitted form conditions.
- Wire
Conditional Header - A validated S3 precondition retaining opaque entity-tag bytes.
Enums§
- Auth
Scheme - The typed
Authorizationheader scheme. - Bucket
Decode Error - S3 bucket decode failure.
- Conditional
Header - A parsed S3 conditional request header.
- Entity
TagSet - The entity-tag set of an S3 conditional header.
- S3Error
Class - Coarse server-error classification used by the S3 frontend to translate a server-layer failure into an S3 error envelope.
- S3Key
Error - S3 object key construction failure.
- S3Session
Error - S3 multipart upload session persistence failure.
- S3Sub
Resource - A recognized S3 sub-resource (the
?subresourcequery dispatch set).
Constants§
- MAX_
LIST_ KEYS - The S3
ListObjectsV2maximum page size. - MAX_
S3_ DELETE_ KEYS - The maximum number of keys in a single
DeleteObjectsrequest. - MAX_
S3_ PART_ NUMBER - The S3 protocol maximum part number.
Traits§
- S3Error
Classify - Classifies a server-layer error into an
S3ErrorClass.
Functions§
- acquire_
session_ part_ lock - Returns the legacy per-session part-write lock for an upload id, creating it on first use.
- acquire_
session_ part_ lock_ for_ root - Returns the per-session part-write lock scoped to one local deployment root, creating it on first use.
- classify
- Classifies the query parameters of an S3 request into the recognized sub-resources, in query order.
- count_
active_ sessions - Counts the currently active (unexpired) sessions.
- create_
session - Creates a new multipart upload session and returns its upload id.
- decode_
bucket - Decodes a
{owner}.{name}S3 bucket into its parts. - decode_
continuation_ token - Decodes an opaque base64 continuation token back to the raw cursor key.
- delete_
session - Deletes a session and all of its part files.
- delete_
session_ locked - The lock-free counterpart of
delete_session: the caller must hold the session lock (lock_upload_sessions) for the duration of the mutation. - encode_
bucket - Encodes a repository owner and name into the
{owner}.{name}S3 bucket. - encode_
continuation_ token - Encodes a raw cursor key as an opaque base64 continuation token.
- etag_
header - Formats a content hash as the S3-style quoted ETag header value.
- extract_
access_ key - Extracts the S3 access key — the Shardline bearer token — from request headers.
- extract_
access_ key_ from_ query - Extracts the S3 access key from the query string (presigned-URL form).
- format_
iso8601 - Formats unix seconds as an S3
LastModifiedISO-8601 UTC string (2026-08-13T09:51:00Z). - group_
page - Groups a sorted, prefix-filtered batch of index rows into one page.
- is_
expired - Returns whether a session has expired against the TTL.
- list_
bucket_ sessions - Lists live sessions for a bucket and namespace without opening part payloads.
- lock_
session_ parts - Acquires the cross-process advisory lock for one session’s part files.
- lock_
upload_ sessions - Acquires the upload-root advisory file lock.
- new_
upload_ id - Generates a new random upload id (32 lowercase hex chars).
- parse_
complete_ multipart_ parts - Parses ordered parts and their echoed ETags from completion XML.
- parse_
content_ md5 - Parses an optional
Content-MD5header without allocating request-sized data. - parse_
copy_ source - Parses an S3
x-amz-copy-sourceheader value (/{bucket}/{key}, with the leading slash optional) into its typed bucket + key parts. - parse_
delete_ object_ keys - Parses the
<Key>values from aDeleteObjectsrequest body. - parse_
list_ objects_ url_ encoding - Returns whether ListObjects response key-like fields should be URL encoded. The existing v1/v2 parameter structs keep their public literal shape.
- parse_
list_ objects_ v1_ params - Parses the
ListObjects(v1) query parameters from a decoded query map. - parse_
list_ objects_ v2_ params - Parses the
ListObjectsV2query parameters from a decoded query map. - parse_
s3_ range - Parses an S3
Rangeheader against a resource length. - parse_
subresource - Parses one query pair into its typed sub-resource, if the name is a recognized sub-resource.
- part_
file_ path - The
part-{n}file path for one part of a session. - read_
conditional_ headers - Reads the S3 conditional headers (
If-Match/If-None-Match) from a request’s header map. - read_
session - Reads a session, treating missing or expired sessions as
NotFound. - read_
session_ locked - Reads a session while the caller owns
lock_upload_sessions. - read_
wire_ conditional_ headers - Parses all repeated conditional fields, preserving If-Match precedence.
- repair_
session_ evidence - Explicitly establishes missing reliability baselines for one legacy session after operator approval.
- require_
s3_ bucket_ binding - Requires the bucket to bind exactly to the token’s repository scope.
- s3_
object_ key - Builds the storage object key for an S3 object.
- session_
dir - The per-session directory (
<root>/s3-uploads/<upload_id>). - session_
metadata_ path - The session metadata path (
session.jsoninside the session directory). - store_
part - Records a stored part’s size in the session metadata.
- store_
part_ locked - The lock-free counterpart of
store_part: the caller must hold the session lock (lock_upload_sessions) for the duration of the mutation so a concurrent sweep cannot remove the session directory mid-write. - store_
versioned_ part_ locked - Publishes an already persisted immutable content-addressed part.
- store_
versioned_ part_ with_ checksum_ locked - Publishes an immutable part with its negotiated checksum in the same integrity-protected session snapshot.
- stored_
part_ file_ path - Resolves a recorded part without permitting path traversal or cross-part references.
- sweep_
expired_ sessions - Removes every expired (or unreadable) session directory, returning the number of sessions removed.
- upload_
dir - The upload root directory (
<root>/s3-uploads). - validate_
part_ number - Validates an S3 part number against the protocol bounds.
- validate_
part_ quota_ for_ session_ locked - Validates quotas using a session already loaded under the caller’s session lock. This avoids re-reading the same session metadata on request paths that already needed it for ownership and body-size validation.
- validate_
part_ quota_ locked - Validates a part against the per-session and aggregate byte quotas and the global active-part-file cap WITHOUT persisting anything.
- validate_
upload_ id - Validates that an upload id is a safe path component (hex or
-). - verify_
sigv4 - Verifies header or query authentication with a configured S3 region.
- verify_
sigv4_ post_ policy - Verifies an AWS Signature Version 4 browser POST policy.
- versioned_
part_ file_ name - Builds the immutable filename for a BLAKE3-addressed part.
- visit_
bucket_ sessions - Visits verified live session metadata one session at a time, without part I/O. The synchronous visitor runs under the upload-session lock and must not do blocking work or acquire that lock again.
- xml_
escape - Escapes XML text, omitting characters XML 1.0 cannot represent.
Type Aliases§
- Query
Map - An ordered query-parameter list.