multipart-stream 0.1.1

parse and serialize async `multipart/x-mixed-replace` streams.
Documentation

crates.io CI

Rust library to parse and serialize async multipart/x-mixed-replace streams. Note this is different than multipart/form-data; you might be interested in the multipart crate for that.

What's a multipart stream for?

A multipart stream is a sequence of parts in one HTTP response, each part having its own headers and body. A stream might last forever, serving parts that didn't exist at the start of the request. This is a type of "hanging GET" or Comet request.

It's a simple HTTP/1.1 way of accomplishing what otherwise might require fancier server- and client-side technologies, such as:

Never-ending multipart streams seem popular in the IP camera space:

  • Dahua IP cameras provide a multipart/x-mixed-replace stream of events such as motion detection changes. (spec)
  • Hikvision IP cameras provide a multipart/mixed stream of events, as described here.
  • wikipedia mentions that IP cameras use this format for MJPEG streams.

There's a big limitation, however, which is that browsers have fairly low limits on the number of concurrent connections. In Chrome's case, six per host.

What is a multipart stream exactly?

A multipart response might look like this:

Content-Type: multipart/x-mixed-replace: boundary=B

--B
Content-Type: text/plain
Content-Length: 3

foo

--B
Content-Type: text/plain
Content-Length: 3

bar

and is typically paired with Transfer-Encoding: chunked or Connection: close to allow sending a response whose size is infinite or not known until the end.

I can't find a good specification. This WHATWG document describes multipart/x-mixed-replace loosely. It refers to RFC 2046 which defines multipart encodings originally used for rich emails. I don't think these HTTP multipart streams quite follow that RFC. My library currently requires:

  • Content type multipart/...; boundary=.... In MIME media type terminology, the type is multipart; the subtype may be anything. There should be exactly one parameter, boundary. No preamble. That is, no arbitrary bytes to discard before the first part's boundary.
  • Zero or more newlines (to be precise: \r\n sequences) between each part and the next part's boundary.
  • A Content-Length line for each part. This is a much cleaner approach than producers attempting to choose a boundary that doesn't appear in any part and consumers having to search through the part body.
  • No extra -- suffix on the final part's boundary. In practice, all the streams I've seen only end due to error, so this point has never come up.

Please open a github issue if you encounter a multipart stream which doesn't match these requirements.

What does this library do?

It takes a stream of Bytes (such as those returned by reqwest or hyper) and returns a stream of multipart_stream::Parts, or vice versa. Each Part packages together headers and a body.

Author

Scott Lamb <slamb@slamb.org>

License

SPDX-License-Identifier: MIT OR Apache-2.0

See LICENSE-MIT.txt or LICENSE-APACHE, respectively.