av-mumu — Audio/Video plugin for Lava / MuMu

av-mumu is a native audio plugin that adds high‑quality output and MP3 decoding to the Lava / MuMu runtime. It is built for cooperative, non‑blocking streaming: decoders never sleep, drains work in micro‑bursts, and the interpreter stays responsive.

Repository: https://gitlab.com/tofo/av-mumu
Crate: av-mumu (library + cdylib)

What’s inside

• Output (CPAL‑backed) with ring‑buffered producer & safe up/down‑mixing to device channels.
• MP3 decoding (Symphonia) surfaced as a zero‑arg transform you can drain.
• Explicit control surface — open/start/drain and pause/resume/restart/seek/status; no hidden “player” magic.
• Deterministic micro‑bursts — all drains are budgeted to keep the single‑threaded interpreter snappy.

We deliberately removed the old av:play_mp3 helper to keep the API format‑agnostic. Compose with mp3_decode (or future decoders) and out_* primitives.

API overview

Output lifecycle

• av:out_open([device?:string, sample_rate?:int, channels?:int, buffer_ms?:int]) -> Ref(KeyedArray)
  Returns a handler (as a Ref(KeyedArray)) containing public fields, e.g. id, device, channels, etc.

• av:out_start(handler) -> 0 — starts the device (idempotent).

• av:out_pause(handler) -> 0 — pauses the device; handler state becomes "paused".

• av:out_resume(handler) -> 0 — resumes the device; state becomes "started".

• av:out_stop(handler) -> 0 — stops (pauses) the device.

• av:out_close(handler) -> 0 — closes the device; handler state is updated.

Decoding (sources)

• av:mp3_decode([path:string, chunk_frames?:int, channels?:int, start_ms?:long]) -> () => IntArray(i32) | "AGAIN" | "NO_MORE_DATA"
  A zero‑arg transform that yields interleaved PCM chunks widened to i32.
  start_ms allows starting decoding at a millisecond offset (implemented as a conservative decode‑then‑skip).

Draining (sink)

• av:out_drain(handler, source[, on_done:Function(long)]) -> 0
  Non‑blocking micro‑burst drain from source into the output ring. The optional on_done is called exactly once with total frames when EOF is reached.
  The drain also mirrors progress into the handler map (best‑effort):
  • frames_written (device‑side)
  • position_frames (drain‑side count)
  • position_ms (derived if sample_rate known)
  • draining / ended flags

Seek, restart & status

• av:out_seek(handler, ms[, source_or_opts]) -> 0
  Seeks to a new offset (ms) by restarting with a fresh source:

  • 3rd arg Function → treated as the new transform (already positioned).
  • 3rd arg KeyedArray → treated as decoder options; we set/override start_ms := ms and call av:mp3_decode(...) to build the transform.
  • If the 3rd arg is omitted, a clear error is returned (no hidden state).

• av:out_restart(handler, source_or_opts) -> 0
  Restarts playback from the beginning or a new offset by providing either a new transform or decoder options (same semantics as above, without forcing start_ms).

• av:out_status(handler) -> KeyedArray — richer status snapshot:

{
  "state":            "created" | "started" | "paused" | "stopped" | "closed" | "ended",
  "draining":         bool,
  "paused":           bool,
  "ended":            bool,
  "frames_written":   long,   // device‑side total frames rendered
  "position_frames":  long,   // drain‑side progress (best effort)
  "position_ms":      long,   // derived from sample_rate
  "sample_rate":      int,
  "channels":         int,
  "buffer_ms":        int
}

Devices

• av:devices([suppress?:bool=true]) -> StrArray — lists output device names (host only).

Quickstart

Minimal playback

extend("av")

handler = av:out_open([buffer_ms: 60, channels: 2])
decode  = av:mp3_decode([path: "examples/mp3/01.mp3", chunk_frames: 2048, channels: 2])

av:out_start(handler)
av:out_drain(handler, decode)

Pause after 2s; resume after another 2s

extend("av")
extend("event")

handler = av:out_open([buffer_ms: 60, channels: 2])
decode  = av:mp3_decode([path: "examples/mp3/01.mp3", chunk_frames: 2048, channels: 2])

av:out_start(handler)
av:out_drain(handler, decode)

event:timeout(2000, () => av:out_pause(handler))   // pause at ~2s
event:timeout(4000, () => av:out_resume(handler))  // resume at ~4s

Seek to 30s and continue

extend("av")

h = av:out_open([buffer_ms: 60, channels: 2])
src = av:mp3_decode([path: "examples/mp3/01.mp3", chunk_frames: 2048, channels: 2])

av:out_start(h)
av:out_drain(h, src)

// Later, seek to 30s by providing decoder opts (we set start_ms under the hood)
av:out_seek(h, 30_000, [path:"examples/mp3/01.mp3", chunk_frames:2048, channels:2])

Restart from the beginning with a fresh transform

extend("av")

h = av:out_open([buffer_ms: 60, channels: 2])
src = av:mp3_decode([path: "examples/mp3/01.mp3", chunk_frames: 2048, channels: 2])

av:out_start(h)
av:out_drain(h, src)

fresh = av:mp3_decode([path: "examples/mp3/01.mp3", chunk_frames: 2048, channels: 2])
av:out_restart(h, fresh)

Status snapshot

extend("av")

// ...after starting a drain
info = av:out_status(h)
slog(info)

Design notes

• Cooperative scheduling – drains never sleep. If temporarily not ready (e.g., ring full or throttle window), they return "AGAIN" and the interpreter keeps ticking.
• Ring & mixing – the ring stores interleaved i16 samples at the source channel count. The device callback safely up/down‑mixes to the actual device layout.
• Typed results – decoders yield IntArray(i32) for ergonomics with the type system; drains clamp to i16 when pushing to the ring.
• Seeking – start_ms uses a conservative decode‑then‑skip approach; true container seeking will be wired as Symphonia seeking is exposed and proven across formats. The public API (av:out_seek) is stable and forwards to restart with a positioned transform.

Build / Install

Host (native) playback is provided under the host feature.

• Build (debug):

  make
  

• Build (release):

  make release
  

• Install library (copies libmumuav.so/.dylib into /usr/local/lib and runs ldconfig when applicable):

  make install
  

Cross‑compiles can use TARGET_TRIPLE=<triple> make release (e.g. x86_64-unknown-linux-gnu).
Web/WASM builds are stubbed — functions report “unavailable” in that mode.

Configuration & environment

• Drain micro‑bursts (defaults in parentheses)
  LAVA_AV_BURST (64) — max chunks per poll tick
  LAVA_AV_BUDGET_US (500) — per‑tick time budget in microseconds

• Verbose bridge logs
  LAVA_VERBOSE=1 — additional logs during registration & operations

Changelog (API highlights)

• Removed av:play_mp3 (use mp3_decode + out_* primitives instead).
• Added av:out_pause, av:out_resume, av:out_restart, av:out_seek, av:out_status.
• av:mp3_decode now accepts start_ms for offset starts.

License

Dual‑licensed under MIT and Apache‑2.0. See LICENSE.

Acknowledgements

• Author: Tom Fotheringham (@tofo)
• Contributors: the Lava / MuMu community
• Upstream: cpal, symphonia