s3s-http3 0.16.0-alpha.1

Experimental HTTP/3 transport for s3s
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75

# s3s

[![Apache 2.0 licensed][license-badge]][license-url]
[![Unsafe Forbidden][unsafe-forbidden-badge]][unsafe-forbidden-url]

[license-badge]: https://img.shields.io/badge/license-Apache--2.0-blue.svg
[license-url]: ./LICENSE
[unsafe-forbidden-badge]: https://img.shields.io/badge/unsafe-forbidden-success.svg
[unsafe-forbidden-url]: https://github.com/rust-secure-code/safety-dance/

S3 Service Adapter


| crate                      |                                           version                                           |                                 docs                                 |
| :------------------------- | :-----------------------------------------------------------------------------------------: | :------------------------------------------------------------------: |
| [s3s](./crates/s3s/)       |    [![Crates.io](https://img.shields.io/crates/v/s3s.svg)](https://crates.io/crates/s3s)    |    [![Docs](https://docs.rs/s3s/badge.svg)](https://docs.rs/s3s/)    |
| [s3s-aws](./crates/s3s-aws/)       |    [![Crates.io](https://img.shields.io/crates/v/s3s-aws.svg)](https://crates.io/crates/s3s-aws)    |    [![Docs](https://docs.rs/s3s-aws/badge.svg)](https://docs.rs/s3s-aws/)    |
| [s3s-sigv2](./crates/s3s-sigv2/)    |    [![Crates.io](https://img.shields.io/crates/v/s3s-sigv2.svg)](https://crates.io/crates/s3s-sigv2)    |    [![Docs](https://docs.rs/s3s-sigv2/badge.svg)](https://docs.rs/s3s-sigv2/)    |
| [s3s-sigv4](./crates/s3s-sigv4/)    |    [![Crates.io](https://img.shields.io/crates/v/s3s-sigv4.svg)](https://crates.io/crates/s3s-sigv4)    |    [![Docs](https://docs.rs/s3s-sigv4/badge.svg)](https://docs.rs/s3s-sigv4/)    |
| [s3s-rfc2047](./crates/s3s-rfc2047/) |    [![Crates.io](https://img.shields.io/crates/v/s3s-rfc2047.svg)](https://crates.io/crates/s3s-rfc2047)    |    [![Docs](https://docs.rs/s3s-rfc2047/badge.svg)](https://docs.rs/s3s-rfc2047/)    |
| [s3s-fs](./crates/s3s-fs/) | [![Crates.io](https://img.shields.io/crates/v/s3s-fs.svg)](https://crates.io/crates/s3s-fs) | [![Docs](https://docs.rs/s3s-fs/badge.svg)](https://docs.rs/s3s-fs/) |

📚 **[Development documentation](https://s3s-project.github.io/s3s/)** for the `main` branch is available on GitHub Pages.

This experimental project intends to offer an ergonomic adapter for building S3-compatible services.

`s3s` implements Amazon S3 REST API in the form of a generic [hyper](https://github.com/hyperium/hyper) service. S3-compatible services can focus on the S3 API itself and don't have to care about the HTTP layer.

`s3s-aws` provides useful types and integration with [`aws-sdk-s3`](https://crates.io/crates/aws-sdk-s3).

`s3s-rfc2047` provides RFC 2047 MIME encoded-word encoding and decoding for non-ASCII header values.

`s3s-fs` implements the S3 API based on file system, as a sample implementation. It is designed for integration testing, which can be used to [mock an S3 client](https://github.com/Nugine/s3s/blob/main/crates/s3s-fs/tests/it_aws.rs). It also provides a binary for debugging. [Play it!](./CONTRIBUTING.md#play-the-test-server)

## How it works

![architecture diagram](docs/arch/arch.svg)

The diagram above shows how `s3s` works. 

`s3s` converts HTTP requests to operation inputs before calling the user-defined service. 

`s3s` converts operation outputs or errors to HTTP responses after calling the user-defined service.

The data types, serialization and deserialization are generated from the smithy model in [aws-sdk-rust](https://github.com/awslabs/aws-sdk-rust) repository. We apply manual hacks to fix some problems in [smithy server codegen](https://smithy-lang.github.io/smithy-rs/design/server/overview.html) and make `s3s` ready to use now.

## Security

`S3Service` and other adapters in this project are not a complete security boundary. If they are exposed to the Internet directly, they may be **attacked**.

It is up to the user to implement security enhancements such as **HTTP body length limits**, object-size limits, rate limits and back pressure.

**Authentication is required for production deployments.** Without calling `set_auth`, the service accepts anonymous (unsigned) requests and skips authorization entirely: every S3 operation is open to any client that can reach the service, and signed requests fail with `NotImplemented` because no authentication provider is configured. A forgotten `set_auth` turns the service into a publicly readable and writable endpoint.

For streaming uploads (`PUT Object`, `UploadPart`), `s3s` applies a default 5 GiB object-size limit matching the AWS single-PUT limit; set `S3Config::put_object_max_size` to `None` to disable it and enforce deployment-specific caps in the `S3` implementation. For production, set it explicitly even though the default is already 5 GiB. `POST Object` keeps using `S3Config::post_object_max_file_size`.

List-type responses (`ListObjects`, `ListBuckets`, ...) are serialized in full by `s3s`: their memory usage grows with the number of entries the `S3` implementation returns. Implementations should paginate (`max-keys` / continuation tokens) and deployments should bound response sizes.

## Docker

Docker images are available at [GitHub Container Registry (GHCR)](https://github.com/s3s-project/s3s/pkgs/container/s3s).

See [Docker documentation](./docs/docker.md) for usage details.

## Contributing

+ [Development Guide](./CONTRIBUTING.md)

## Sponsor

We have a reward funds pool for contributors: <https://github.com/Nugine/s3s/issues/174>

If my open-source work has been helpful to you, please [sponsor me](https://github.com/Nugine#sponsor).

Every little bit helps. Thank you!