Crate http_serve

Helpers for serving HTTP GET and HEAD responses asynchronously with the http crate and tokio. Works well with hyper 0.14.x.

This crate supplies two ways to respond to HTTP GET and HEAD requests:

• the serve function can be used to serve an Entity, a trait representing reusable, byte-rangeable HTTP entities. Entity must be able to produce exactly the same data on every call, know its size in advance, and be able to produce portions of the data on demand.
• the streaming_body function can be used to add a body to an otherwise-complete response. If a body is needed (on GET rather than HEAD requests), it returns a BodyWriter (which implements std::io::Writer). The caller should produce the complete body or call BodyWriter::abort, causing the HTTP stream to terminate abruptly.

It supplies a static file Entity implementation and a (currently Unix-only) helper for serving a full directory tree from the local filesystem, including automatically looking for .gz-suffixed files when the client advertises Accept-Encoding: gzip.

Why two ways?

They have pros and cons. This table shows some of them:

	serve	streaming_body
automatic byte range serving	yes	no [1]
backpressure	yes	no [2]
conditional GET	yes	no [3]
sends first byte before length known	no	yes
automatic gzip content encoding	no [4]	yes

[1]: streaming_body always sends the full body. Byte range serving wouldn’t make much sense with its interface. The application will generate all the bytes every time anyway, and http-serve’s buffering logic would have to be complex to handle multiple ranges well.

[2]: streaming_body is often appended to while holding a lock or open database transaction, where backpressure is undesired. It’d be possible to add support for “wait points” where the caller explicitly wants backpressure. This would make it more suitable for large streams, even infinite streams like Server-sent events.

[3]: streaming_body doesn’t yet support generating etags or honoring conditional GET requests. PRs welcome!

[4]: serve doesn’t automatically apply Content-Encoding: gzip because the content encoding is a property of the entity you supply. The entity’s etag, length, and byte range boundaries must match the encoding. You can use the http_serve::should_gzip helper to decide between supplying a plain or gzipped entity. serve could automatically apply the related Transfer-Encoding: gzip where the browser requests it via TE: gzip, but common browsers have chosen to avoid requesting or handling Transfer-Encoding.

Use serve when:

• metadata (length, etag, etc) and byte ranges can be regenerated cheaply and consistently via a lazy Entity, or
• data can be fully buffered in memory or on disk and reused many times. You may want to create a pair of buffers for gzipped (for user-agents which specify Accept-Encoding: gzip) vs raw.

Use streaming_body when regenerating the entire body each time a response is sent.

Once you return a hyper::server::Response to hyper, your only way to signal error to the client is to abruptly close the HTTP connection while sending the body. If you want the ability to return a well-formatted error to the client while producing body bytes, you must buffer the entire body in-memory before returning anything to hyper.

If you are buffering a response in memory, serve requires copying the bytes (when using Data = Vec<u8> or similar) or atomic reference-counting (with Data = Arc<Vec<u8>> or similar). streaming_body doesn’t need to keep its own copy for potential future use; it may be cheaper because it can simply hand ownership of the existing Vec<u8>s to hyper.

Why the weird type bounds? Why not use hyper::Body and bytes::Bytes for everything?

These bounds are compatible with hyper::Body and bytes::Bytes, and most callers will use those types. Note: if you see an error like the one below, ensure you are using hyper’s stream feature:

error[E0277]: the trait bound `Body: From<Box<(dyn futures::Stream<Item = Result<_, _>> +
std::marker::Send + 'static)>>` is not satisfied

Cargo.toml should look similar to the following:

hyper = { version = "0.14.7", features = ["stream"] }

There are times when it’s desirable to have more flexible ownership provided by a type such as reffers::ARefs<'static, [u8]>. One is mmap-based file serving: bytes::Bytes would require copying the data in each chunk. An implementation with ARefs could instead mmap and mlock the data on another thread and provide chunks which munmap when dropped. In these cases, the caller can supply an alternate implementation of the http_body::Body trait which uses a different Data type than bytes::Bytes.

Modules

dirdir

Directory traversal on local filesystems. Currently Unix-only. Gated behind the dir feature.

Structs

BodyWriter

A std::io::Write implementation that makes a chunked hyper response body stream. Automatically applies gzip content encoding if requested by the client.

ChunkedReadFile

HTTP entity created from a std::fs::File which reads the file chunk-by-chunk within a tokio::task::block_in_place closure.

StreamingBodyBuilder

A builder returned by streaming_body.

Traits

Entity

A reusable, read-only, byte-rangeable HTTP entity for GET and HEAD serving. Must return exactly the same data on every call.

Functions

serve

Serves GET and HEAD requests for a given byte-ranged entity. Handles conditional & subrange requests. The caller is expected to have already determined the correct entity and appended Expires, Cache-Control, and Vary headers if desired.

should_gzip

Returns iff it’s preferable to use Content-Encoding: gzip when responding to the given request, rather than no content coding.

streaming_body

Creates a response and streaming body writer for the given request.