Expand description
§moqros
Stream ROS 2 topics over Media over QUIC.
Images are converted to I420, encoded (VP8 via libvpx by default, AV1 via libaom with the
av1 feature, or H.264 via Cisco’s OpenH264 binary with the h264 feature; on a GPU through
GStreamer with the hw feature) and published as
hang broadcasts, the format played by the @moq/watch
web component. The reverse direction (MoQ -> decoded images) is also provided.
Messages of any other type travel as JSON or CDR payloads in a hang data track
(Publisher::create_data_broadcast, Subscriber::subscribe_data).
The core API is ROS-independent:
let config = moqros::ClientConfig::new("http://localhost:4443/anon".parse().unwrap());
let publisher = moqros::Publisher::connect(&config)?;
let mut camera = publisher.create_image_broadcast("camera", moqros::EncoderSettings::default())?;
camera.publish(&moqros::Image::new(640, 480, moqros::PixelFormat::Rgb8, rgb_bytes, capture_time)?)?;
let subscriber = moqros::Subscriber::connect(&config)?;
let mut images = subscriber.subscribe_images("camera").await?;
while let Some(decoded) = images.next_image().await? {
// decoded.image is rgb8, captured at decoded.capture_time
}Data broadcasts carry opaque payloads:
let settings = moqros::DataSettings::new(moqros::DataFormat::Json);
let mut status = publisher.create_data_broadcast("status", settings)?;
status.publish(r#"{"battery": 0.82}"#)?;
let mut messages = subscriber.subscribe_data("status").await?;
while let Some(message) = messages.next_message().await? {
// message.payload is one JSON document
}Subscriber::subscribe returns either kind, depending on what the broadcast’s catalog
offers. Enable the ros feature for ros bridges built on r2r.
§Features
| Feature | What |
|---|---|
vp8 (default) | VP8 through the system libvpx, real-time CBR. Needs libvpx, pkg-config, and libclang at build time. |
av1 | AV1 (Main profile, 8-bit) through the system libaom in real-time mode. Needs libaom, pkg-config, and libclang at build time. |
h264 | H.264 (Constrained Baseline, avc3) through Cisco’s prebuilt OpenH264 binary, loaded at runtime. |
hw | Hardware encoders (VA-API, NVENC) through GStreamer, falling back to software. Needs the GStreamer development files at build time. |
compressed (default) | Decode JPEG and PNG images (Image::from_compressed, sensor_msgs/CompressedImage) with pure-Rust decoders. |
ros | ROS 2 bridges built on r2r. Needs a sourced ROS 2 environment and libclang at build time. |
Pick the codec with EncoderSettings::codec. A subscriber picks the best rendition in
the catalog that this build can decode.
With the hw feature, EncoderSettings::hardware (Hardware::Auto by default) encodes
on the first hardware encoder GStreamer can start for the codec: VA-API (va plugin, or the
older vaapi plugin) on Intel/AMD GPUs, or NVENC (nvcodec plugin) on NVIDIA GPUs. When
none works, it logs a warning and encodes in software, so the codec’s own feature is still
required. Decoding always runs in software.
The h264 feature never compiles OpenH264: Cisco only covers the MPEG LA royalties for
binaries downloaded from Cisco. Give the binary’s path in OPENH264_LIBRARY or with
set_openh264_library; its SHA-256 must match a known Cisco release.
scripts/fetch_openh264.sh in the repository downloads the right one for the platform.
§Wire format
- Video broadcasts: the catalog (
catalog.json) lists one video track,VIDEO_TRACK, and is republished whenever the resolution changes. Each frame goes in the hang “legacy” container (a timestamp plus the codec payload). Each keyframe starts a new MoQ group, soEncoderSettings::keyframe_intervalbounds how long a new viewer waits for the first picture. Subscribers always jump to the newest group. WithImageBroadcastSettings::renditions, smaller renditions go into tracks of their own (video_<height>p), all listed in the same catalog;SubscribeOptionspicks one. - Pixel formats: see
PixelFormat. An odd trailing row or column is cropped, because 4:2:0 video needs even dimensions. - Timestamps: capture times become a µs timeline that starts at 0 for each broadcast.
- Data broadcasts: the catalog lists one track,
DATA_TRACK, in itsjsonsection (DataFormat::Json) or itsbinarysection (DataFormat::Cdr, MIME typeCDR_MIME), with theDataModeand the ROS message type (rosType). Each MoQ frame is one message payload, without a container. - Capture times: a root
moqrossection of the catalog can carrycaptureOrigin, the capture time in µs that frame timestamp 0 maps to, in the publisher’s clock (e.g. ROS time, which may be sim time). It is republished whenever the timeline re-anchors. When the capture times are wall-clock times since 2020, hang’sclockcarries the same origin. Data broadcasts published withDataPublisher::publish_atuse it. - Version: the
moqrossection carriesversion(1 now; a section without one is version 1). It only goes up when a field changes meaning, and a subscriber ignores a section newer than it understands.
Modules§
- convert
- Color conversion between
Imagepixel formats and planar I420 (YUV 4:2:0). - ros
ros - ROS 2 integration built on r2r.
Structs§
- Announcements
- Broadcasts announced under a prefix, from
Subscriber::announced. - Client
Config - Where and how to connect to a MoQ relay.
- Data
Message - One message received from a data broadcast.
- Data
Publisher - Writes messages into one MoQ data broadcast.
- Data
Settings - Describes a data track: given to
Publisher::create_data_broadcast, and read back from the catalog byDataSubscriber::settings. - Data
Subscriber - Reads the messages of one data track.
- Decoded
Image - One decoded picture received from a broadcast.
- Decoder
- Decoder producing
rgb8Images. - Encoded
Frame - One encoded frame.
- Encoder
- Stateful encoder that accepts
Images of any supported pixel format. - Encoder
Settings - Tuning knobs for
Encoder. - Image
- An uncompressed image, equivalent to the payload of a
sensor_msgs/Image. - Image
Broadcast Settings - How
Publisher::create_image_broadcastencodes a video broadcast: its full-size rendition, and optionally smaller ones (simulcast). - Image
Publisher - Encodes images and writes them into one MoQ broadcast.
- Image
Subscriber - Reads frames of one video track, always jumping to the newest group so a slow reader falls back to the live edge instead of accumulating latency.
- Publisher
- A connection to a relay that can carry any number of image and data broadcasts.
- Rendition
Settings - A smaller copy of a video broadcast, for viewers with less bandwidth or smaller screens.
- Session
- The connection to a relay behind a
PublisherorSubscriber. - Subscribe
Options - How
Subscriber::subscribe_withandSubscriber::subscribe_images_withsubscribe. - Subscriber
- A connection to a relay used to consume image and data broadcasts.
- Url
- A parsed URL record.
- Video
Frame - One encoded video frame received from a broadcast.
Enums§
- Codec
- Video codecs moqros can produce and consume.
- Data
Format - How each message is serialized on the wire.
- Data
Mode - How messages map onto MoQ groups.
- Error
- Errors produced by moqros.
- Hardware
- Whether
Encoderencodes on a hardware encoder (featurehw). - Pixel
Format - Pixel layouts understood by moqros.
- Published
- What
ImagePublisher::publishorImagePublisher::publish_encodeddid with a picture. - Subscription
- What
Subscriber::subscribefound in a broadcast.
Constants§
- CDR_
MIME - Media type of
DataFormat::Cdrpayloads in the catalog’sbinarysection. - DATA_
TRACK - Name of the single data track inside every moqros data broadcast.
- DEFAULT_
MAX_ PIXELS - Default for
Decoder::set_max_pixels: 8K UHD (7680x4320). - VIDEO_
TRACK - Name of the single video track inside every moqros broadcast.
Functions§
- broadcast_
is_ under - Whether the broadcast
nameis underprefix, by whole path segments:robotscoversrobots/a/cameraandrobots, but notrobots2/camera. Slashes at either end of both are ignored, and an empty prefix covers every name. - broadcast_
name_ for_ topic - Turn a ROS topic into a broadcast name:
/camera/image_raw->camera/image_raw. - connect_
both - Start one connection to the relay that both publishes and subscribes, e.g. for a robot that streams its camera and takes commands. Must be called inside a tokio runtime.
- normalize_
broadcast_ name - A broadcast name without slashes at either end, the form to compare names in:
/robots/a/->robots/a. - set_
openh264_ library h264 - Use Cisco’s OpenH264 binary at
pathfor H.264. OverridesOPENH264_LIBRARY.
Type Aliases§
- Result
std::result::Resultdefaulting to moqros’sError.