Skip to content

Add muxer support for E-AC-3 and Dolby Atmos (E-AC-3 JOC)#3317

Open
rexjin-meta wants to merge 1 commit into
androidx:mainfrom
rexjin-meta:muxer-atmos-v2
Open

Add muxer support for E-AC-3 and Dolby Atmos (E-AC-3 JOC)#3317
rexjin-meta wants to merge 1 commit into
androidx:mainfrom
rexjin-meta:muxer-atmos-v2

Conversation

@rexjin-meta

@rexjin-meta rexjin-meta commented Jul 2, 2026

Copy link
Copy Markdown

Problem

Mp4Muxer and FragmentedMp4Muxer had no support for E-AC-3 (audio/eac3) or E-AC-3 JOC (audio/eac3-joc, the container format for Dolby Atmos). Attempting to mux an E-AC-3 or Atmos audio track into an MP4 container would fail at the supported-format check, making stream-copy of these tracks impossible.

Changes

  • Boxes.java: Added dec3Box() which wraps the raw EC3SpecificBox payload from format.initializationData verbatim into a dec3 box, following the same pattern as av1CBox(). Added AUDIO_E_AC3 / AUDIO_E_AC3_JOC cases in codecSpecificBox() (routes to dec3Box) and codecSpecificFourcc() (returns "ec-3").
  • Mp4Muxer.java: Added AUDIO_E_AC3 and AUDIO_E_AC3_JOC to SUPPORTED_AUDIO_SAMPLE_MIME_TYPES.
  • FragmentedMp4Muxer.java: Same addition to SUPPORTED_AUDIO_SAMPLE_MIME_TYPES.
  • BoxesTest.java: Added 3 unit tests covering dec3 payload wrapping for AUDIO_E_AC3, dec3 payload wrapping for AUDIO_E_AC3_JOC, and IllegalArgumentException when initializationData is absent.

Testing

  • ./gradlew :lib-muxer:compileDebugJavaWithJavac — clean compile
  • ./gradlew :lib-muxer:test — all existing tests pass, 3 new unit tests pass

Regarding an end-to-end parameterized test — sample_eac3joc.mp4 is already in the test asset library. The blocker is that BoxParser currently parses the dec3 box into Format fields (channel count, sample rate, mime type) but does not carry the raw bytes forward into

Format.initializationData. This is a pre-existing behavior in the extractor, not introduced by this PR. Since Mp4Muxer needs those raw bytes to write the dec3 box, the full round-trip pipeline doesn't connect yet. Happy to add the end-to-end test once that extractor behavior is addressed, or if the team has a preferred approach for bridging that gap.

@snow2405 snow2405 self-assigned this Jul 3, 2026
@FongMi

FongMi commented Jul 3, 2026

Copy link
Copy Markdown

Findings

  1. P1 Mp4Muxer.java:417 / FragmentedMp4Muxer.java:184
    Adding AUDIO_E_AC3 / AUDIO_E_AC3_JOC to the supported list causes the default Transformer to directly transmux E-AC-3. However, dec3Box() requires the raw dec3 payload in Format.initializationData, while the existing MP4 extractor parses dec3 into Format fields without preserving initialization data. For common E-AC-3/Atmos MP4 or TS inputs, this may change behavior from transcoding to AAC into direct copy, then fail later while muxing moov because csd-0 is missing. I’d recommend either preserving raw dec3 in BoxParser in the same PR, or not adding these MIME types to the factory supported list yet.

  2. P3 Boxes.java:1658
    dec3Box() only checks that initializationData is non-empty, but does not check that csd-0 itself has bytes. An empty payload would produce an empty dec3 box, which is not a valid EC3SpecificBox. This should mirror esdsBox() / hvcCBox() with checkArgument(csd0.length > 0, ...), plus a test for an empty byte array.

Implement the dec3 box (EC3SpecificBox) and ec-3 fourcc in Boxes.java so
Mp4Muxer and FragmentedMp4Muxer can write E-AC-3 and Dolby Atmos audio
tracks. The dec3 payload is carried verbatim from
format.initializationData, following the same pattern as av1C. Direct
callers can mux Atmos by supplying the raw dec3 payload as csd-0.

E-AC-3 is intentionally NOT added to SUPPORTED_AUDIO_SAMPLE_MIME_TYPES
yet: the MP4 extractor does not currently preserve the raw dec3 payload
in Format.initializationData, so advertising support would make the
default Transformer attempt a direct stream-copy that then fails while
writing moov. E-AC-3 will be added to the supported list in a follow-up
once the extractor preserves the dec3 payload.

Unit tests added to BoxesTest cover:
- dec3 box wraps csd-0 payload verbatim for AUDIO_E_AC3
- dec3 box wraps csd-0 payload verbatim for AUDIO_E_AC3_JOC
- IllegalArgumentException is thrown when initializationData is absent
@rexjin-meta

rexjin-meta commented Jul 3, 2026

Copy link
Copy Markdown
Author

Hey @FongMi and @snow2405 , Thanks for the reviews — addressed both comments:

  • P1 (Transformer regression): Dropped AUDIO_E_AC3 / AUDIO_E_AC3_JOC from SUPPORTED_AUDIO_SAMPLE_MIME_TYPES in both Mp4Muxer and FragmentedMp4Muxer, so the default Transformer keeps transcoding E-AC-3 to AAC — no behavior change. The muxer retains the dec3/ec-3 write capability for direct callers that supply csd-0. Added a comment on dec3Box() explaining the deferral.

  • P3 (empty csd-0): Added checkArgument(csd0.length > 0, "csd-0 is empty for dec3 box."), mirroring esdsBox() / hvcCBox(), plus a test for the empty-byte[] case.

Also added a unit test asserting the ec-3 fourcc on the audio sample entry.

This PR will laser focus on enabling muxer to write atmos.
Follow-up (separate PR): preserve the raw dec3 payload in BoxParser, then add E-AC-3 to the supported list and add an end-to-end mux test (with the audio_sample_entry_box_eac3 golden dump). This staging keeps the current PR free of the extractor change and any Transformer impact.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants