Crate otf_pixels

Expand description

A streaming, demand-driven image processing engine.

Pixels is a libvips-class pipeline engine: images are lazy operation graphs, pixels are pulled through the graph on demand, and memory stays bounded regardless of image size. This crate is the facade — the chainable Image API and the Image::output terminal — over otf_pixels_core’s engine, otf_pixels_ops’ kernels and the codec crates.

use otf_pixels::{Format, Image, ImageDescriptor, PixelFormat};

let descriptor = ImageDescriptor::new(4, 4, PixelFormat::Gray8)?;
let pixels: Vec<u8> = (0..16).collect();

// Construction and chaining do no pixel work.
let bytes = Image::from_raw(descriptor, pixels)?
    .crop(1, 1, 2, 2)
    .flip()
    .output(Format::Raw, Default::default())
    .bytes()?;

assert_eq!(bytes, [9, 10, 5, 6]);

§Errors are deferred, not swallowed

Chaining methods take and return Self rather than Result, so a pipeline reads as one expression. An error raised mid-chain — a crop window outside the image, say — is captured and carried to the terminal, where it surfaces from Output::write or Output::bytes. Nothing is silently ignored, and no operation runs after a failed one.

§Evaluation

Terminals run the pipeline on the demand-driven tile scheduler: output tiles are evaluated in parallel and delivered to the sink in order, with peak memory bounded by tiles in flight rather than image size.

Every output runs on Scheduler::global unless told otherwise: one pool of worker threads, one per core, and one tile cache, shared by every pipeline in the process. That is the right setup for a server or a runtime handling many images at once, and it needs no code: call output(...).bytes() from as many threads as you like and the runs share the workers. Do not build a Scheduler per request, and do not set Output::threads or Output::scheduler_options to tune a busy host — both give that run a private pool, spawned and joined each time, which under concurrency means a pool per request competing for the same cores. Output::with_scheduler runs on a scheduler you built, for a host that wants image work confined to a fixed number of threads.

Output::bytes_via_reference runs the same pipeline through the M1 whole-image evaluator instead. That path is slow and holds every intermediate in full, but it is obviously correct, so it is the oracle the scheduler is verified against.

§Serving images

What an image endpoint or a runtime’s image API does: bytes in, a thumbnail out. Opening identifies the format from its bytes, turns the image upright, converts its colours to sRGB and enforces input limits; nothing is decoded until the output is pulled, and a JPEG source decodes at a reduced scale when the thumbnail allows it.

use otf_pixels::{
    EncodeOptions, Fit, Format, Image, Limits, OpenOptions, ResizeOptions,
};

// Per request: bound what an untrusted upload may allocate.
let options = OpenOptions::default()
    .with_limits(Limits::default().with_max_pixels(50_000_000));
let image = Image::from_stream_with(std::io::Cursor::new(upload), options)?;

let meta = image.metadata()?; // free: no pixels decoded
assert_eq!((meta.width, meta.height), (64, 48));
if let Some(animation) = image.animation() {
    // v1 processes the first frame; the caller decides whether that is
    // acceptable for this animation.
    let _ = animation.frame_count;
}

let webp = image
    .resize_with(32, 32, ResizeOptions::default().with_fit(Fit::Cover))
    .output(Format::WebP, EncodeOptions::with_quality(80)?)
    .bytes()?;
assert_eq!(&webp[8..12], b"WEBP");

§Scope (v1)

Formats, all implemented in this workspace and checked against their reference implementations: PNG, GIF, JPEG (baseline, and progressive decode), TIFF, WebP (lossy and lossless) and AVIF, read and written, plus raw pixels.
Ops: crop, flip/flop, quarter-turn rotation and orientation, resize with sharp’s five fit modes, modulate, convolve/blur/sharpen, composite, flatten, channel extraction, pixel-format and sRGB conversion.
Metadata: everything sharp’s metadata() reports, from the headers alone (Image::source_metadata), and its stats() (Image::stats); EXIF/HEIF orientation applied on open; ICC profiles converted to sRGB or carried to the output; animation reported, with the first frame processed. EXIF and XMP are written only when an output keeps them (EncodeOptions::keep_exif, EncodeOptions::keep_xmp), so location data is stripped from uploads by default.

Not yet: multi-frame (animated) pipelines and arbitrary-angle rotation. Each will arrive as an addition, not a change: OpenOptions::animated is already reserved for the first.

Structs§

Animation: How an animated image plays: its frames and their timing.
AvifCodec: The AVIF entry in a sniffing registry.
AvifDecoder: Decodes an AVIF stream.
AvifEncoder: Encodes an AVIF still.
ChannelStats: One channel’s statistics.
Comment: A text entry a file carries, such as PNG’s tEXt (sharp’s comments).
Composite: Draw an overlay over a base.
ConvertFormat: Convert to a different PixelFormat.
Convolve: Apply a convolution kernel.
Crop: Extract a rectangular window of an image.
Density: How densely the image is meant to be printed or shown (sharp’s density, resolutionUnit).
EncodeOptions: Encoder tuning shared across formats.
EnsureAlpha: Add an alpha channel at a fixed level if there is none (sharp’s ensureAlpha); an image that has one is unchanged.
Extend: Add top, right, bottom and left pixels around an image.
ExtractChannel: Extract one channel as a greyscale image (SPEC §Core ops).
Flatten: Composite an image onto an opaque background, discarding alpha.
Flip: Mirror an image vertically: the top row becomes the bottom row.
Flop: Mirror an image horizontally: the left column becomes the right column.
Gamma: Gamma: each colour channel becomes value^(1 / exponent), as libvips’ gamma defines it. An exponent above 1 brightens, below 1 darkens.
GifCodec: The GIF entry in a sniffing registry.
GifDecoder: Decodes a GIF stream.
GifEncoder: Encodes a single-frame GIF.
Greyscale: Greyscale (sharp’s greyscale): every colour channel becomes the pixel’s BT.601 luma. The channel count is kept, as in sharp, whose greyscale output is three identical channels; alpha is untouched.
Image: A lazily evaluated image pipeline.
ImageDescriptor: The shape of an image at a point in the graph.
JpegCodec: The JPEG entry in a sniffing registry.
JpegDecoder: Decodes a JPEG stream.
JpegEncoder: Encodes a baseline JPEG stream.
JxlCodec: The JPEG XL entry in a sniffing registry.
JxlDecoder: Decodes a JPEG XL stream.
JxlEncoder: Encodes a JPEG XL codestream.
Kernel: A convolution kernel: odd-sized, square or rectangular, small.
Limits: Safety limits applied before any pixel memory is allocated.
Linear: A per-channel linear map, value * a + b (sharp’s linear). b is in 8-bit units, as in sharp: an offset of 255 is the full range, at any depth. Alpha is untouched.
Metadata: Header-only facts about an image.
Modulate: Brightness, saturation and hue adjustment (SPEC §Core ops).
Negate: Invert colours: each channel becomes 1 - value (sharp’s negate).
OpenOptions: How Image::open_with and Image::from_stream_with read an image.
Output: A pipeline with its output format chosen, ready to be pulled.
Overlay: One image for Image::composite_overlays, with sharp’s per-image options: blend, gravity, left/top and tile.
PlanOptions: Knobs for Plan::build.
PngCodec: The PNG entry in a sniffing registry.
PngDecoder: Decodes a PNG stream.
PngEncoder: Encodes a PNG stream.
RawCodec: Format sniffing for raw streams.
RawDecoder: Decodes a raw pixel stream, one row per call.
RawEncoder: Encodes rows of pixels as a raw stream.
RawFormat: The layout of a raw pixel stream.
Recomb: Recombine channels through a matrix (sharp’s recomb): sepia, channel swaps and the like.
Region: An axis-aligned rectangle of pixels, in image coordinates.
Resize: Resample an image to a new size.
ResizeOptions: Options for Resize.
Rotate: Rotate an image by a multiple of 90 degrees.
RunStats: Counters describing one run, for tests and diagnostics.
Scheduler: A demand-driven, parallel tile evaluator.
SchedulerOptions: Tuning for a Scheduler.
SourceInfo: What a decoder learned about its file from the header, beyond the pixel layout: the facts sharp’s metadata() reports per format. Every field is empty for a format with no notion of it.
SourceMetadata: What a source file says about itself, read from its headers (sharp’s metadata()). Nothing here costs a pixel decode.
Stats: Pixel statistics of an image (sharp’s stats()), in its own sample scale: 0–255 for 8-bit, 0–65535 for 16-bit, 0–1 for float.
Threshold: Two-level threshold (sharp’s threshold): a value at or above the threshold becomes full, anything below becomes zero.
TiffCodec: The TIFF entry in a sniffing registry.
TiffDecoder: Decodes a TIFF stream.
TiffEncoder: Encodes a baseline TIFF.
Tint: Tint with a colour (sharp’s tint): each pixel keeps its luma and takes its chroma from the tint.
ToSrgb: Converts pixels in an ICC profile’s colour space to sRGB.
Trim: Options for finding a trim box.
WebPCodec: The WebP entry in a sniffing registry.
WebPDecoder: Decodes a WebP stream.
WebPEncoder: Encodes a WebP stream.

Enums§

AccessPattern: The tile shape an op wants its input delivered in.
Background: The colour a file names as its background, in its own samples.
Blend: How the overlay is combined with the base: sharp’s blend.
Channel: A channel named the way sharp’s extractChannel names it.
ChannelLayout: The channel layout of a pixel, independent of sample type.
ChromaSubsampling: How an encoder samples chroma (sharp’s chromaSubsampling).
ColorModel: The color model a descriptor’s samples are interpreted in.
Conversion: What ToSrgb::from_profile makes of a profile.
ErrorCode: A stable, machine-readable error classification.
ExtendWith: What fills the margin Extend adds.
Filter: A resampling filter kernel (SPEC §Core ops).
Fit: How a resize reconciles the requested box with the source aspect ratio, with sharp’s meanings.
Format: A container format.
Limit: The safety limit that a PixelsError::LimitExceeded refers to.
Orientation: One of the eight orientations the EXIF/TIFF Orientation tag (274) can name.
PixelFormat: An interleaved pixel format: a ChannelLayout over a SampleKind.
PixelsError: The error type returned by every fallible engine operation.
Placement: Where the overlay goes: sharp’s left/top, gravity and tile.
Position: Which part of the image Fit::Cover keeps, and where Fit::Contain places it: sharp’s position gravities.
Quarter: A quarter-turn rotation, clockwise.
ResolutionUnit: A resolution’s unit, as the file states it.
SampleKind: The numeric type of one channel sample.
Scale: How much of an image a decode produces, as eighths of full size.
Subsampling: How much the chroma channels are subsampled relative to luma.
TiffLayout: How an encoder arranges pixels in the file.
TileShape: The shape tiles take through a segment of the graph.
Unconvertible: Why a profile is not converted.

Traits§

Codec: Format sniffing.
Decoder: Decodes a byte stream into rows of pixels.
Encoder: Encodes rows of pixels into a byte stream.
Sink: A sink for encoded bytes.
Source: A forward-only source of bytes.

Functions§

evaluate_reference: Evaluate image to a whole-image buffer.

Type Aliases§

Result: The engine’s result alias.

Crate otf_pixels

Crate otf_pixels Copy item path

§Errors are deferred, not swallowed

§Evaluation

§Serving images

§Scope (v1)

Structs§

Enums§

Traits§

Functions§

Type Aliases§

Crate otf_pixels