feat: relative broadcast paths for cross-broadcast catalog renditions (incl. WHEP egress)#1371
Merged
Conversation
Contributor
WalkthroughThis change adds an optional per-rendition broadcast reference to audio and video configs and implements path utilities to normalize and resolve relative broadcast strings. Runtime code (Broadcast.trackBroadcast and updated decoder/MSE codepaths) now resolves and caches override broadcasts and falls back to the catalog broadcast when absent or self-referential. Importers populate the new field as None by default. Rust and JS tests exercise normalization, resolution, serialization, and subscription behavior. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
A hang catalog rendition can now set an optional `broadcast` field, a path relative to the broadcast that served the catalog (e.g. "../source"), pointing at the broadcast that actually publishes the track. A transcoder can then publish a sidecar catalog that adds new renditions while pointing unchanged ones at the original broadcast instead of re-publishing the bytes. - moq-net: new PathRelative type (normalized, ".."-capable, serde-only) and Path::resolve. Mirrored in @moq/net as Path.normalizeRelative/Path.resolve. - hang / @moq/hang: optional `broadcast` field on VideoConfig and AudioConfig. - moq-mux: new Source type bundling the catalog BroadcastConsumer with optional (OriginConsumer, path) context. Exporters take `impl Into<Source>` (bare BroadcastConsumer still works via From) and the shared ExportSource resolves per-rendition references via OriginConsumer::request_broadcast. - moq-cli subscribe, moq-srt egress, and moq-hls pass their origin through. - @moq/watch: Broadcast.trackBroadcast() resolves the reference against the broadcast name and consumes the resolved path on the same connection, caching one subscription per referenced broadcast; wired into the audio and video decoder and MSE pipelines. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
kixelated
force-pushed
the
catalog-broadcast-override
branch
from
be48c4c to
87b2f9b
Compare
kixelated
changed the title
…-917897 # Conflicts: # js/watch/src/broadcast.ts # rs/moq-cli/src/main.rs # rs/moq-cli/src/subscribe.rs # rs/moq-hls/src/export/mod.rs # rs/moq-hls/src/export/rendition.rs # rs/moq-mux/src/codec/h264/export.rs # rs/moq-mux/src/container/fmp4/export.rs # rs/moq-mux/src/container/mkv/export.rs # rs/moq-mux/src/lib.rs # rs/moq-srt/src/ts.rs
25 tasks
# Conflicts: # js/watch/src/video/decoder.ts # rs/moq-cli/src/subscribe.rs # rs/moq-hls/src/export/mod.rs # rs/moq-hls/src/export/rendition.rs # rs/moq-hls/src/server/mod.rs # rs/moq-mux/src/codec/h264/export.rs # rs/moq-mux/src/codec/h265/export.rs # rs/moq-mux/src/container/flv/export.rs # rs/moq-mux/src/container/fmp4/export.rs # rs/moq-mux/src/container/mkv/export.rs # rs/moq-mux/src/container/source.rs # rs/moq-mux/src/container/ts/export.rs
The WHEP egress subscribed each rendition track directly on the catalog broadcast, ignoring a rendition's catalog `broadcast` field. A sidecar catalog that points a rendition at a sibling broadcast (e.g. `../source`) therefore failed to egress that track over WebRTC. Add `moq_mux::Source::subscribe_track`, the async counterpart to the poll-driven container exporters: it resolves an optional cross-broadcast reference against the origin and subscribes, returning a raw `track::Subscriber` for consumers that wrap it themselves. `EgressSource::new` now takes `impl Into<moq_mux::Source>` and `pick_track` routes each rendition through `subscribe_track`, so the WHEP server (which already holds the subscribe origin) resolves references the same way as moq-cli/moq-srt/moq-hls. A bare `broadcast::Consumer` still works via `From` (the WHIP client path), degrading to `Error::MissingOrigin` only if such a catalog actually references another broadcast. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kixelated
changed the title
cargo deny check advisories flags crossbeam-epoch 0.9.18 (invalid pointer dereference in the fmt::Pointer impl for Atomic/Shared). Transitive dep; lockfile-only bump to the patched 0.9.20. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # rs/moq-mux/src/container/flv/export.rs
kixelated
commented
Jul 7, 2026
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @kixelated, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
Reshape moq_mux::Source per review: require an origin and the catalog broadcast's path instead of a pre-resolved BroadcastConsumer + optional origin context. A Source without an origin could never follow a cross-broadcast rendition reference, so making origin mandatory removes that footgun and the Error::MissingOrigin it needed. Source::new(origin, path) now resolves both the catalog broadcast and any sibling broadcast a rendition's catalog `broadcast` field references through origin.request_broadcast, which deduplicates announced paths. This drops From<BroadcastConsumer>, with_origin, and the Into<Source> sugar on every exporter, so fmp4/mkv/ts/flv/h264/h265 exporters, moq_hls::Broadcaster, and moq-cli's Subscribe now take a Source directly. broadcast() is async (it resolves through the origin), and Broadcaster::new is async + fallible. Callers thread origin + path: moq-cli subscribe, moq-srt, moq-hls, the WHEP server, and now also the WHIP and RTMP client egress paths and the RTMP server play path, so cross-broadcast renditions are honored everywhere the exporters run. moq_rtc::Client::publish and moq_rtmp Client::publish take (origin, path) to match. JS (js/watch): rename Broadcast.trackBroadcast to relativeBroadcast and drop the per-path override cache. conn.consume mints an independent consumer and each rendition subscribes a distinct track, so the cache added no real dedup; the reference now resolves inline, scoped to the caller's effect like the catalog broadcast. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The concept doc still showed the removed Source::new(broadcast).with_origin(origin, path) API and the bare-BroadcastConsumer error path. Describe the current shape. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
request_broadcast only deduplicated announced broadcasts and concurrently-queued requests: a dynamically-served (OriginDynamic) broadcast is never announced, and the handler removes the queue entry when it serves, so each later request for the same path re-invoked the handler and opened a duplicate upstream subscription (or, with a one-shot handler, failed with Unroutable). Cache served broadcasts weakly in OriginDynamicState, keyed by path, mirroring the existing weak-dedup that broadcast::Consumer::track already does for tracks. A repeat request for a still-live served path now resolves to a shared clone; a closed entry is evicted lazily and re-served. Weak so the broadcast still closes once its real consumers drop. This makes moq_mux::Source's "resolve everything through request_broadcast" correct for the dynamic path (WHEP egress, JIT transcode workers): the catalog and every same-broadcast rendition share one upstream subscription instead of spinning up N+1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
relativeBroadcast calls #isPathAnnounced in a per-rendition reactive path, so the Cloudflare 'no discovery' warning fired on every re-evaluation. Gate it behind a per-connection WeakSet so it logs at most once. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # js/watch/src/video/decoder.ts
This was referenced Jul 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rewritten from scratch against
dev(the original main-based implementation predated the moq-mux catalog machinery and the js/watch rework) and now hooked up end to end on the consumer side, including WHIP/WHEP and RTMP egress.A rendition in a hang catalog can set an optional
broadcastfield: a path relative to the broadcast that served the catalog (e.g."../source"), pointing at the broadcast that actually publishes the track. A transcoder can then publish a sidecar catalog that adds new renditions while pointing unchanged ones at the original broadcast, instead of re-publishing those bytes.Schema / path math
moq-net: newPathRelativetype (trims slashes, drops.segments, preserves..; serde only, never on the announce/subscribe wire) andPath::resolvewith excess-..clamping. Mirrored in@moq/netasPath.normalizeRelative/Path.resolveso both sides agree byte-for-byte.hang/@moq/hang: optionalbroadcastfield onVideoConfigandAudioConfig(skipped when unset, so existing catalogs are byte-identical).Consumer wiring
moq-net:origin::Consumer::request_broadcastnow weakly caches dynamically-served (OriginDynamic) broadcasts by path, so the catalog and every same-broadcast rendition share one upstream subscription instead of re-invoking the handler N+1 times (which previously duplicated subscriptions, or failed withUnroutableagainst a one-shot handler). Announced broadcasts were already deduplicated.moq-mux: newSourcetype built viaSource::new(origin, path)(anOriginConsumerplus the catalog broadcast's path). Origin is required: a source without one could never follow a cross-broadcast reference, so there's no bare-BroadcastConsumerpath and noMissingOriginerror. Both the catalog broadcast and any referenced sibling resolve throughOriginConsumer::request_broadcast, which deduplicates announced paths (a self-reference or excess-..falls back to the catalog broadcast). All exporters (fmp4, mkv, ts, flv, h264, h265) now take aSource;Source::broadcast()is async.moq-cli subscribe,moq-srtegress,moq-hls, themoq-rtcWHEP server and WHIP client egress, and themoq-rtmpserver play path and client push all pass their origin + path through, so every place the exporters run honors cross-broadcast renditions.moq_rtc::Client::publishandmoq_rtmp::Client::publishnow take(origin, path);moq_hls::Broadcaster::newis async + fallible.@moq/watch:Broadcast.relativeBroadcast(effect, rel)resolves the reference against the broadcast name and consumes the resolved path on the same connection, scoped to the caller's effect (no manual cache:conn.consumemints an independent consumer and each rendition subscribes a distinct track, so the previous per-path cache added no real dedup). Wired into the audio + video decoder and MSE pipelines.Public API changes
moq_net::{PathRelative, PathRelativeOwned},moq_net::Path::resolve(plus an internalrequest_broadcastdedup, no signature change),moq_mux::Source(new(origin, path)/ asyncbroadcast()/subscribe_track()),hang::catalog::{VideoConfig,AudioConfig}::broadcast,@moq/netPath.resolve/Path.normalizeRelative,@moq/hangRelativeBroadcastSchema,@moq/watchBroadcast.relativeBroadcast.dev): moq-mux exporter constructors,moq_hls::Broadcaster::new(nowasync+Result),moq-cli'sSubscribe::new,moq_rtc::EgressSource::new,moq_rtc::Client::publish, andmoq_rtmp::Client::publishall take aSource/(origin, path)instead of aBroadcastConsumer. Catalog format gains an optional field.Not covered (follow-ups)
moq-ffi(and the py/swift/kt/go wrappers) andmoq-gstconsume tracks from a bareBroadcastConsumerand don't build aSource, so they don't resolve references yet; they need origin context plumbed the same way.conn.consume(path)in JS.@moq/netconn.consume(path)does not yet deduplicate the way Rustrequest_broadcastnow does (JSclose()closes shared state, so it needs a consume-ownership rework). Tracked in js/net: dedupe conn.consume() so repeat subscribes to a path share one broadcast #2121; the currentjs/watchpath works for announced broadcasts (all the browser hits) but churns on effect rerun.Test plan
PathRelativenormalization +Path::resolveunit tests (Rust and JS mirrors)Nonestays off the wire, empty string normalizes (Rusthang+@moq/hangzod)moq_mux::Sourceunit tests: no-override, self-reference reuse, excess-..fallback, referenced-broadcast resolution through a live origincargo test+clippy(default features) across moq-mux, moq-hls, moq-rtc, moq-srt, moq-rtmp, moq-cli; JS:tsc+ biome on the watch changesroom/source, publish a sidecar catalog withbroadcast: "../source", verify@moq/watchandmoq subscribepull the track from the source broadcast(Written by Claude Opus 4.8)
🤖 Generated with Claude Code