Skip to main content

Crate moqros

Crate moqros 

Source
Expand description

§moqros

Stream ROS 2 topics over Media over QUIC.

Images are converted to I420, encoded (VP8 via libvpx by default, AV1 via libaom with the av1 feature, or H.264 via Cisco’s OpenH264 binary with the h264 feature; on a GPU through GStreamer with the hw feature) and published as hang broadcasts, the format played by the @moq/watch web component. The reverse direction (MoQ -> decoded images) is also provided.

Messages of any other type travel as JSON or CDR payloads in a hang data track (Publisher::create_data_broadcast, Subscriber::subscribe_data).

The core API is ROS-independent:

let config = moqros::ClientConfig::new("http://localhost:4443/anon".parse().unwrap());

let publisher = moqros::Publisher::connect(&config)?;
let mut camera = publisher.create_image_broadcast("camera", moqros::EncoderSettings::default())?;
camera.publish(&moqros::Image::new(640, 480, moqros::PixelFormat::Rgb8, rgb_bytes, capture_time)?)?;

let subscriber = moqros::Subscriber::connect(&config)?;
let mut images = subscriber.subscribe_images("camera").await?;
while let Some(decoded) = images.next_image().await? {
    // decoded.image is rgb8, captured at decoded.capture_time
}

Data broadcasts carry opaque payloads:

let settings = moqros::DataSettings::new(moqros::DataFormat::Json);
let mut status = publisher.create_data_broadcast("status", settings)?;
status.publish(r#"{"battery": 0.82}"#)?;

let mut messages = subscriber.subscribe_data("status").await?;
while let Some(message) = messages.next_message().await? {
    // message.payload is one JSON document
}

Subscriber::subscribe returns either kind, depending on what the broadcast’s catalog offers. Enable the ros feature for ros bridges built on r2r.

§Features

FeatureWhat
vp8 (default)VP8 through the system libvpx, real-time CBR. Needs libvpx, pkg-config, and libclang at build time.
av1AV1 (Main profile, 8-bit) through the system libaom in real-time mode. Needs libaom, pkg-config, and libclang at build time.
h264H.264 (Constrained Baseline, avc3) through Cisco’s prebuilt OpenH264 binary, loaded at runtime.
hwHardware encoders (VA-API, NVENC) through GStreamer, falling back to software. Needs the GStreamer development files at build time.
compressed (default)Decode JPEG and PNG images (Image::from_compressed, sensor_msgs/CompressedImage) with pure-Rust decoders.
rosROS 2 bridges built on r2r. Needs a sourced ROS 2 environment and libclang at build time.

Pick the codec with EncoderSettings::codec. A subscriber picks the best rendition in the catalog that this build can decode.

With the hw feature, EncoderSettings::hardware (Hardware::Auto by default) encodes on the first hardware encoder GStreamer can start for the codec: VA-API (va plugin, or the older vaapi plugin) on Intel/AMD GPUs, or NVENC (nvcodec plugin) on NVIDIA GPUs. When none works, it logs a warning and encodes in software, so the codec’s own feature is still required. Decoding always runs in software.

The h264 feature never compiles OpenH264: Cisco only covers the MPEG LA royalties for binaries downloaded from Cisco. Give the binary’s path in OPENH264_LIBRARY or with set_openh264_library; its SHA-256 must match a known Cisco release. scripts/fetch_openh264.sh in the repository downloads the right one for the platform.

§Wire format

  • Video broadcasts: the catalog (catalog.json) lists one video track, VIDEO_TRACK, and is republished whenever the resolution changes. Each frame goes in the hang “legacy” container (a timestamp plus the codec payload). Each keyframe starts a new MoQ group, so EncoderSettings::keyframe_interval bounds how long a new viewer waits for the first picture. Subscribers always jump to the newest group. With ImageBroadcastSettings::renditions, smaller renditions go into tracks of their own (video_<height>p), all listed in the same catalog; SubscribeOptions picks one.
  • Pixel formats: see PixelFormat. An odd trailing row or column is cropped, because 4:2:0 video needs even dimensions.
  • Timestamps: capture times become a µs timeline that starts at 0 for each broadcast.
  • Data broadcasts: the catalog lists one track, DATA_TRACK, in its json section (DataFormat::Json) or its binary section (DataFormat::Cdr, MIME type CDR_MIME), with the DataMode and the ROS message type (rosType). Each MoQ frame is one message payload, without a container.
  • Capture times: a root moqros section of the catalog can carry captureOrigin, the capture time in µs that frame timestamp 0 maps to, in the publisher’s clock (e.g. ROS time, which may be sim time). It is republished whenever the timeline re-anchors. When the capture times are wall-clock times since 2020, hang’s clock carries the same origin. Data broadcasts published with DataPublisher::publish_at use it.
  • Version: the moqros section carries version (1 now; a section without one is version 1). It only goes up when a field changes meaning, and a subscriber ignores a section newer than it understands.

Modules§

convert
Color conversion between Image pixel formats and planar I420 (YUV 4:2:0).
rosros
ROS 2 integration built on r2r.

Structs§

Announcements
Broadcasts announced under a prefix, from Subscriber::announced.
ClientConfig
Where and how to connect to a MoQ relay.
DataMessage
One message received from a data broadcast.
DataPublisher
Writes messages into one MoQ data broadcast.
DataSettings
Describes a data track: given to Publisher::create_data_broadcast, and read back from the catalog by DataSubscriber::settings.
DataSubscriber
Reads the messages of one data track.
DecodedImage
One decoded picture received from a broadcast.
Decoder
Decoder producing rgb8 Images.
EncodedFrame
One encoded frame.
Encoder
Stateful encoder that accepts Images of any supported pixel format.
EncoderSettings
Tuning knobs for Encoder.
Image
An uncompressed image, equivalent to the payload of a sensor_msgs/Image.
ImageBroadcastSettings
How Publisher::create_image_broadcast encodes a video broadcast: its full-size rendition, and optionally smaller ones (simulcast).
ImagePublisher
Encodes images and writes them into one MoQ broadcast.
ImageSubscriber
Reads frames of one video track, always jumping to the newest group so a slow reader falls back to the live edge instead of accumulating latency.
Publisher
A connection to a relay that can carry any number of image and data broadcasts.
RenditionSettings
A smaller copy of a video broadcast, for viewers with less bandwidth or smaller screens.
Session
The connection to a relay behind a Publisher or Subscriber.
SubscribeOptions
How Subscriber::subscribe_with and Subscriber::subscribe_images_with subscribe.
Subscriber
A connection to a relay used to consume image and data broadcasts.
Url
A parsed URL record.
VideoFrame
One encoded video frame received from a broadcast.

Enums§

Codec
Video codecs moqros can produce and consume.
DataFormat
How each message is serialized on the wire.
DataMode
How messages map onto MoQ groups.
Error
Errors produced by moqros.
Hardware
Whether Encoder encodes on a hardware encoder (feature hw).
PixelFormat
Pixel layouts understood by moqros.
Published
What ImagePublisher::publish or ImagePublisher::publish_encoded did with a picture.
Subscription
What Subscriber::subscribe found in a broadcast.

Constants§

CDR_MIME
Media type of DataFormat::Cdr payloads in the catalog’s binary section.
DATA_TRACK
Name of the single data track inside every moqros data broadcast.
DEFAULT_MAX_PIXELS
Default for Decoder::set_max_pixels: 8K UHD (7680x4320).
VIDEO_TRACK
Name of the single video track inside every moqros broadcast.

Functions§

broadcast_is_under
Whether the broadcast name is under prefix, by whole path segments: robots covers robots/a/camera and robots, but not robots2/camera. Slashes at either end of both are ignored, and an empty prefix covers every name.
broadcast_name_for_topic
Turn a ROS topic into a broadcast name: /camera/image_raw -> camera/image_raw.
connect_both
Start one connection to the relay that both publishes and subscribes, e.g. for a robot that streams its camera and takes commands. Must be called inside a tokio runtime.
normalize_broadcast_name
A broadcast name without slashes at either end, the form to compare names in: /robots/a/ -> robots/a.
set_openh264_libraryh264
Use Cisco’s OpenH264 binary at path for H.264. Overrides OPENH264_LIBRARY.

Type Aliases§

Result
std::result::Result defaulting to moqros’s Error.