#[non_exhaustive]pub enum Caps {
CompressedVideo {
codec: VideoCodec,
width: Dim,
height: Dim,
framerate: Rate,
},
RawVideo {
format: RawVideoFormat,
width: Dim,
height: Dim,
framerate: Rate,
interlace: Interlace,
},
Audio {
format: AudioFormat,
channels: u8,
sample_rate: u32,
},
Tensor {
dtype: TensorDType,
shape: TensorShape,
layout: TensorLayout,
},
ByteStream {
encoding: ByteStreamEncoding,
},
Text {
format: TextFormat,
},
Klv,
ClosedCaption {
format: ClosedCaptionFormat,
},
SubPicture {
format: SubPictureFormat,
},
}Expand description
Caps describes one fixated (or partially-narrowed) link.
Video is split into Caps::CompressedVideo and Caps::RawVideo
because a codec bitstream and a raw pixel buffer are different
kinds of media, not different values of one “format” slot. A raw
sink (waylandsink, kmssink) intercepting a CompressedVideo caps is
a category error; the type system now expresses that as a variant
mismatch rather than a runtime enum compare. (Mirrors GStreamer’s
video/x-h264 vs video/x-raw distinction; M17 split.)
Both video variants carry geometry today. That’s pragmatic, not
honest: GStreamer’s video/x-h264 doesn’t have width/height because
they live in the SPS. Our solver, the RtspSrc placeholder Range, and
our Range-as-placeholder convention all hang off geometry on
compressed caps. Dropping it is a deeper rework that overlaps
workaround #1’s redesign; out of scope here.
Variants (Non-exhaustive)§
This enum is marked as non-exhaustive
CompressedVideo
Compressed video bitstream (codec). Width/height/framerate are nominal until the bitstream parser confirms them via SPS/equivalent.
RawVideo
Raw pixel buffer in format. Geometry is authoritative.
Audio
Tensor
A tensor stream (ML). Its shape (TensorShape) is a fixed-rank
inline array (M636), so the variant is part of the no-alloc MCU subset
like every other caps kind.
ByteStream
An opaque container / elementary byte stream, not yet demuxed or parsed
into a typed media stream. The link type between a byte source (a file or
network source carrying e.g. an MPEG-TS) and a demuxer that splits it into
elementary streams. encoding names the wire format so a demuxer only
accepts a stream it understands.
Fields
encoding: ByteStreamEncodingText
A text stream (subtitles, captions, transcription, OCR, overlay strings).
format names the syntax (TextFormat); the payload is UTF-8 bytes in
the frame’s system buffer, and “subtitle” is just timed Text (cue PTS +
duration on FrameTiming). One kind, not a
per-use-case variant, so an overlay, a caption sink, and a text analytics
element all negotiate the same caps.
Fields
format: TextFormatKlv
A KLV (SMPTE ST 336 key-length-value) metadata stream, each frame one KLV
packet (for STANAG 4609 UAS streams, a MISB ST 0601 local set). The
elementary metadata stream a transport demuxer splits out alongside video,
timed by the frame’s PTS (GStreamer meta/x-klv).
ClosedCaption
A raw closed-caption stream: a container track carrying caption data as its
own elementary stream rather than inside a video bitstream’s SEI (the MP4
c608 / c708 raw-caption tracks). Each frame is one sample’s cc_data
byte triples, (marker | cc_valid | cc_type, cc_data_1, cc_data_2), the
same ATSC A/53 layout an SEI caption block carries, so one caption decoder
serves both paths. format names which carriage the track declared, and
therefore which services its triples can hold: 608 line-21 fields, or 708
DTVCC packets. Captions embedded in a coded video stream stay
Caps::CompressedVideo; this is the separate-track case only.
Fields
format: ClosedCaptionFormatSubPicture
A coded bitmap-subtitle (subpicture) stream: each frame one cue’s coded
bitmap, not pixels. The bitmap subtitle counterpart of Caps::Text,
which stays the timed-text kind. format names the coding
(SubPictureFormat); a subpicture decoder turns it into full-frame
transparent Caps::RawVideo RGBA canvases a compositor can paint over
video, so nothing downstream needs a bitmap-cue concept.
Fields
format: SubPictureFormatImplementations§
Source§impl Caps
impl Caps
Sourcepub fn is_raw_media(&self) -> bool
pub fn is_raw_media(&self) -> bool
Whether this caps carries a raw, uncompressed heavy media buffer: raw
pixels, PCM audio samples, or a tensor. These are the payloads whose bytes
a memory-domain crossing actually copies, so the copy/allocation plan
counts a domain transfer as a real frame copy only between two raw caps
(a codec boundary like CompressedVideo -> RawVideo is a decode, not a
raw-frame copy). Compressed streams, opaque byte streams, and text are not
raw media.
Sourcepub fn intersect(&self, other: &Caps) -> Result<Caps, G2gError>
pub fn intersect(&self, other: &Caps) -> Result<Caps, G2gError>
Phase 1 intersection (DESIGN.md §4.2). Narrow self against other,
returning the overlap. Both must be the same variant; ranged fields
(Dim/Rate) intersect field-wise, scalar fields (codec /
format, channels, sample_rate, tensor dtype/shape/layout) must
be equal. Any empty field overlap, variant mismatch, or scalar
mismatch yields CapsMismatch.
CompressedVideo and RawVideo are distinct variants — a raw
sink offered compressed input gets CapsMismatch structurally,
not a runtime format compare.
Sourcepub fn is_fixed(&self) -> bool
pub fn is_fixed(&self) -> bool
True when every ranged field is Fixed. Scalar-only variants are
always fixed.
Sourcepub fn fixate(&self) -> Result<Caps, G2gError>
pub fn fixate(&self) -> Result<Caps, G2gError>
Phase 2 fixation (DESIGN.md §4.2): collapse every ranged field to a
single Fixed value. Range fixates to its minimum, reflecting
the latency-first design (less data is lower latency); an element
preferring a different value counter-proposes via
ConfigureOutcome::ReFixate.
Any carries no information to fixate against and yields
CapsMismatch.
Sourcepub fn dims(&self) -> Option<(&Dim, &Dim, &Rate)>
pub fn dims(&self) -> Option<(&Dim, &Dim, &Rate)>
Borrow the geometry triple if this caps carries one. Both video
variants (compressed + raw) currently do; Audio and Tensor
return None. Used by element code that needs width/height/fps
without caring whether the link is pre- or post-decode.
Sourcepub fn to_gst_string(&self) -> String
pub fn to_gst_string(&self) -> String
Render these caps as a GStreamer caps string, the inverse of
CapsSet::from_gst_string. For -v pipeline dumps, logs, and porting
diagnostics. The fixed media types round-trip through the parser;
Tensor has no GStreamer media type and is rendered as a g2g-specific
tensor/x-raw descriptor.