Skip to content

feat(moq-transcode): decode once per source, GPU resize fanout, and a moq transcode verb#2158

Merged
kixelated merged 3 commits into
devfrom
claude/gpu-shared-decode
Jul 12, 2026
Merged

feat(moq-transcode): decode once per source, GPU resize fanout, and a moq transcode verb#2158
kixelated merged 3 commits into
devfrom
claude/gpu-shared-decode

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Follow-up to #2145. Three pieces, in dependency order:

1. decode::Frame::resize in moq-video (GPU box filter + CPU fallback)

Fans one decoded stream out to several sizes. A CUDA frame (NVDEC output) resizes on the GPU with a box-average NV12 kernel and stays in device memory, so resize -> encode still never touches the CPU; a CPU frame resizes with a SIMD bilinear convolution (fast_image_resize moved here from moq-transcode).

The kernel is written in CUDA C (frame/nv12_resize.cu) and vendored as PTX, generated once with a CUDA 12.2 nvcc (matching cudarc's cuda-12020 pin, so the PTX ISA 8.2 loads on any supported driver) and JIT-compiled by the driver at runtime. Building the crate still requires no CUDA toolkit, same philosophy as the vendored bindgen output. cudarc's dependency-free nvrtc feature is enabled only for the Ptx type and load_module; libnvrtc itself is never loaded. If the driver rejects the PTX, resize degrades to download + CPU with a one-time warning instead of killing the stream.

A box filter (not 4-tap bilinear) because rung downscales run 2-6x, where naive bilinear aliases; the GPU output is asserted against the CPU reference on hardware.

2. Shared live decode in moq-transcode (decode once per source)

Rungs used to subscribe and decode independently, so N active rungs decoded the same source N times and NVDEC throughput scaled with ladder depth rather than source count. The new feed module owns one source subscription and one decoder per broadcast; active rungs attach via a broadcast channel of Arc'd decoded frames plus group boundaries. Each rung resizes its copy (GPU-resident when hardware decoded) and encodes it in place.

  • The feed runs only while at least one rung has live demand: first listener subscribes and opens the decoder, last one out tears both down. Idle broadcasts still cost nothing.
  • A rung that lags the channel (capacity 16) skips to the next group boundary instead of stalling other rungs; a rung whose encoder fails aborts alone (observed live: the failing rung cancelled while the healthy one kept streaming).
  • Group fetches keep their one-shot pipeline with decoder-side scaling (optimal for a single rung); scale.rs is deleted, subsumed by Frame::resize.
  • Timestamps ride the decoded frames, and every output group still opens with a forced IDR mirroring source sequences 1:1.

3. moq transcode verb in moq-cli (feature-gated)

moq --client-connect <url> --broadcast <src> transcode [--output <path>] [--rung h:bps ...] [--encoder ...] [--decoder ...] publishes <src>/transcode.hang with relative source references (depth-aware) and the ladder. Gated behind a transcode feature, off by default for the same reason as capture: it pulls moq-video, whose default vaapi feature links libva. Documented in doc/bin/cli.md.

Also fixes a startup race present in the shipped example: request_broadcast fails Unroutable until the first session registers its origin handler, so both now wait for Status::Connected.

Public API changes

  • moq_video::decode::Frame::resize(&self, w, h) -> Result<Frame> (additive).
  • moq-video gains a fast_image_resize dependency and enables cudarc's nvrtc feature (no runtime library impact, see above).
  • moq-transcode: no public signature changes (run/Config/Rung untouched); internals rewired.
  • moq-cli: new transcode verb + feature; nvenc/vaapi/nvdec features now also forward to moq-transcode.
  • New workspace dep entry for moq-transcode (default-features off, consistent with moq-video).

Testing

  • Hardware (RTX 3070 Ti): cuda_resize_matches_cpu (kernel vs CPU reference, odd-ish source size so pitch != width), the full NVDEC suite from feat(moq-video): NVDEC hardware decode, zero-copy NVDEC -> NVENC transcode #2145, live_multi_rung_hardware (two rungs riding one NVDEC session into two NVENC sessions), and the existing hardware end-to-end.
  • Deterministic multi-rung test (live_multi_rung, software codecs) using tokio::time::pause so both rungs attach to the feed before the first group exists.
  • Live end-to-end over a real relay: ffmpeg testsrc -> moq import avc3 -> moq transcode (NVDEC/NVENC confirmed in logs) -> two concurrent moq export h264 rung subscribers; outputs verified with ffprobe (correct sizes, 4:2:0). The second rung attached to the already-running shared decode without opening a new decoder or source subscription.
  • Driverless runs (no LD_LIBRARY_PATH, hardware probes fail) pass with clean fallbacks; clippy/fmt clean across nvenc/nvdec/both/neither and moq-cli --features transcode.

Two observations from the live run, both pre-existing and out of scope here: a 4:4:4 H.264 source fails decode with a clean per-session error and rung cancellation (NVDEC and openh264 are 4:2:0-only; backend fallback happens at open time, before the chroma format is known), and the relay's cache holds a rung subscription after the last viewer leaves, so demand.unused() does not fire and the rung keeps encoding (same behavior with the old per-rung decode).

(Written by Claude Fable 5)

🤖 Generated with Claude Code

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @kixelated, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@kixelated

Copy link
Copy Markdown
Collaborator Author

Self-review pass (Sourcery rate-limited again; CodeRabbit passed with no findings). Three findings, addressed in the follow-up commit:

  1. CI failure: cargo-sort rejected the root manifest (the new moq-transcode workspace entry landed out of order). Sorted, and the default-features comment that drifted onto the wrong dependency during the re-sort was reattached.
  2. Per-frame Resizer allocation (perf): I420::resize created a fresh fast_image_resize::Resizer per call, discarding its cached convolution state on every frame of a software live transcode (the old Scaler kept one per pipeline). Now cached per thread.
  3. Feed teardown race (robustness): a rung attaching just as the shared decode session finished (but before the session cleared itself from the feed state) would subscribe to a closed channel and abort its track. listen() now treats a finished task as idle and starts a fresh session.

Also verified during review: Reconnect::status() reports the current state on first poll (not just transitions), so the new wait-for-Connected in the CLI/example cannot hang on a fast connect; the chroma-plane kernel bounds stay inside the allocation for all even sizes; and the box-filter accumulator cannot overflow u32 at NVDEC's 8K decode ceiling.

Re-ran on hardware after the fixes: moq-video 27 (GPU resize + NVDEC suites), moq-transcode 11 (multi-rung hardware), plus driverless fallback runs; fmt/clippy/sort/shear/doc clean.

(Written by Claude Fable 5)

kixelated and others added 2 commits July 11, 2026 09:38
…ranscode verb

Decoding used to be per rung, so N active rungs decoded the same source N
times and NVDEC throughput scaled with ladder depth instead of source count.

- moq-video grows decode::Frame::resize(w, h): a scaled copy that keeps a
  CUDA frame on the GPU (a box-filter kernel, vendored as PTX and
  JIT-compiled by the driver, so builds still need no CUDA toolkit) and
  resizes CPU frames with a SIMD bilinear convolution. Kernel output is
  validated against the CPU reference on hardware.
- moq-transcode gains a shared live feed (feed.rs): one source subscription
  and one decoder per broadcast, fanned out to every rung with live demand
  over a broadcast channel of Arc'd frames. Rungs resize and encode their
  own copy; a lagging rung skips to the next group instead of stalling the
  others. Fetches keep their one-shot pipeline (decoder-side scaling), and
  the CPU scale.rs is gone, subsumed by Frame::resize.
- moq-cli gains a feature-gated `transcode` verb wrapping moq-transcode:
  `moq --client-connect <url> --broadcast <src> transcode [--rung h:bps ...]`.
  Also fixes a startup race (in the example too): request_broadcast is
  unroutable until the first session connects, so wait for Connected.

Validated on an RTX 3070 Ti: unit suites (multi-rung hardware test rides one
NVDEC session into two NVENC sessions) plus a live end-to-end against a
local relay (ffmpeg -> moq import -> moq transcode -> two concurrent rung
exports, outputs verified with ffprobe; the second rung attached to the
already-running shared decode without opening a new decoder).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Sort the root Cargo.toml dependencies (cargo-sort, the CI failure) and
  reattach the default-features comment that drifted onto the wrong entry.
- Cache the CPU resizer per thread: I420::resize built a fresh
  fast_image_resize::Resizer per call, discarding its convolution state on
  every frame of a software live transcode.
- Feed::listen treats a finished-but-not-yet-cleared decode session as idle
  and starts a fresh one, instead of subscribing to a channel that can only
  yield Closed (a small teardown race that would abort the rung track).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@kixelated kixelated force-pushed the claude/gpu-shared-decode branch from 8af4efd to 234dbff Compare July 11, 2026 16:40
…refactor

Rebase fallout from #2146: decode::Frame::resize preserves `timestamp` (the
field `timestamp_us` no longer exists), and the shared feed passes container
timestamps through as-is instead of converting to microseconds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@kixelated kixelated merged commit 39d7296 into dev Jul 12, 2026
3 checks passed
@kixelated kixelated deleted the claude/gpu-shared-decode branch July 12, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant