s3-endpoint
Rust S3-compatible endpoint focused on upload and object-access workflows used by AWS SDKs and the AWS CLI.
The implementation targets Rust 1.95, edition 2024, and forbids unsafe code in application modules.
Supported Scope
Implemented operations:
PutObject
CreateMultipartUpload
UploadPart
ListParts
CompleteMultipartUpload
AbortMultipartUpload
HeadBucket
HeadObject
GetObject
DeleteObject
Implemented compatibility features:
Path-style and virtual-hosted-style addressing
SigV4 Authorization header authentication
SigV4 presigned URLs
UNSIGNED-PAYLOAD and fixed SHA256 payload hashes
HTTP chunked request bodies
Content-Encoding: aws-chunked
Signed aws-chunked payloads and trailer signatures
Content-MD5
x-amz-checksum-crc32, crc32c, sha1, sha256, sha512
Checksum trailers
Multipart ownership checks
Conditional GET/HEAD
Single-range GET
GetObject response header overrides
Filesystem-backed object and multipart persistence
Explicit non-goals for this version include bucket listing, bucket creation, ACLs, IAM policy syntax, object lock, versioning, server-side encryption, CopyObject, UploadPartCopy, and full S3 account semantics.
Run
Useful environment variables mirror the CLI flags:
S3_ENDPOINT_ADDR
S3_ENDPOINT_STORAGE_ROOT
S3_ENDPOINT_ALLOW_ANONYMOUS
S3_ENDPOINT_ACCESS_KEY_ID
S3_ENDPOINT_SECRET_KEY
S3_ENDPOINT_SESSION_TOKEN
S3_ENDPOINT_CREDENTIALS_FILE
S3_ENDPOINT_REGION
S3_ENDPOINT_ALLOWED_BUCKETS
S3_ENDPOINT_ALLOWED_ACTIONS
S3_ENDPOINT_VIRTUAL_HOST_BASE_DOMAIN
S3_ENDPOINT_MAX_OBJECT_SIZE
S3_ENDPOINT_MAX_PART_SIZE
S3_ENDPOINT_MIN_NON_FINAL_PART_SIZE
S3_ENDPOINT_MAX_CONCURRENT_S3_REQUESTS
S3_ENDPOINT_MAX_ACTIVE_OBJECT_WRITERS
S3_ENDPOINT_MAX_ACTIVE_MULTIPART_PART_WRITERS
S3_ENDPOINT_MAX_ACTIVE_AWS_CHUNKED_DECODERS
The health endpoint is:
Credentials File
--credentials-file accepts a JSON array of additional credentials:
Empty allowed_buckets or allowed_actions means the credential is unrestricted for that dimension.
Storage Durability
The filesystem store publishes object bytes, metadata, multipart parts, and upload session files with temporary files plus atomic renames where the platform supports them. Failed commits use best-effort rollback and startup cleanup removes stale temporary files and orphaned object/metadata pairs.
The current store does not fsync object files or parent directories before acknowledging writes. It is intended for local-compatible S3 workflows where process crashes and ordinary I/O failures are handled, but crash-consistent durability across sudden power loss is delegated to the underlying filesystem and host configuration.
AWS CLI Example
AWS_ACCESS_KEY_ID=test \
AWS_SECRET_ACCESS_KEY=testsecret \
AWS_DEFAULT_REGION=us-east-1 \
boto3 Example
=
Verify
Run the normal project gates:
RUSTDOCFLAGS="-D missing_docs"
Run the SDK smoke tests locally:
In another shell:
AWS_ACCESS_KEY_ID=test \
AWS_SECRET_ACCESS_KEY=testsecret \
AWS_DEFAULT_REGION=us-east-1 \
( && )
( && )
( && && )
The GitHub Actions workflow runs the Rust gates plus AWS CLI, boto3, presigned curl, Go SDK v2, Java SDK v2, and JavaScript SDK v3 smoke flows against the built binary.