Motivation
While working on HeartMuLa support (#1227), I noticed that its 48 kHz stereo output cannot currently be represented unambiguously by the public audio result API. Could we add a small, mono-compatible multichannel output contract before wiring up the full codec output?
Current TRTMC interface
At a50cf5dc, AudioResult contains samples, num_samples, and sample_rate, but no channel count or layout. AudioChunkCallback likewise has no channel information. The CLI write_wav hardcodes num_channels = 1.
The sample rate is already configurable; the gap is preserving channels and defining sample-count semantics. Flattening stereo into the existing mono contract would not preserve the intended audio.
Proposed scope
- Add channel metadata to audio results, defaulting to mono, and document a canonical layout (for example, interleaved float samples).
- Explicitly distinguish total scalar samples from frames per channel, retaining existing mono behavior.
- Update WAV output to derive channel count, block alignment, byte rate, and data size consistently; validate malformed buffers. Audit reader behavior and document any intentional mono downmix separately.
- Decide how streaming can carry the same information without silently breaking existing callback users. A new callback type or overload may be preferable to changing the existing signature.
- Add CPU tests for unchanged mono output and stereo round trips with distinct left/right signals, including duration, sample rate, channel order, and invalid buffer lengths.
Initial implementation
PR #1256 proposes the complete-result and WAV-output portion, including channel-aware benchmark duration and CPU regression tests. It does not implement multichannel streaming, additional encoded formats, or an HTTP audio API.
Follow-up capabilities for discussion
- Multichannel streaming output: carry channel count, sample rate, and layout with chunks; define frame boundaries and end-of-stream behavior while preserving existing mono callback users. Test that concatenated chunks preserve channel order, sample count, and duration.
- Additional audio output formats: consider FLAC, MP3, and Opus alongside WAV, with explicit format selection, channel preservation, and clear errors when an encoder is unavailable. Agree on optional dependencies and codec limitations before implementation; lossy formats should not use byte-exact waveform comparisons as acceptance criteria.
- HTTP audio serving: discuss whether this belongs in TRTMC's serving layer and how callers should request audio generation and receive encoded audio with the correct content type, including streaming where supported. Endpoint design, request validation, cancellation, and error propagation need a separate scoped proposal.
These are proposed follow-ups, not implemented features or requirements to expand #1256. Would maintainers prefer separate tracking issues/PRs for them? I am happy to work on the agreed scope, starting with the output contract needed by #1227.
Motivation
While working on HeartMuLa support (#1227), I noticed that its 48 kHz stereo output cannot currently be represented unambiguously by the public audio result API. Could we add a small, mono-compatible multichannel output contract before wiring up the full codec output?
Current TRTMC interface
At
a50cf5dc,AudioResultcontainssamples,num_samples, andsample_rate, but no channel count or layout.AudioChunkCallbacklikewise has no channel information. The CLIwrite_wavhardcodesnum_channels = 1.The sample rate is already configurable; the gap is preserving channels and defining sample-count semantics. Flattening stereo into the existing mono contract would not preserve the intended audio.
Proposed scope
Initial implementation
PR #1256 proposes the complete-result and WAV-output portion, including channel-aware benchmark duration and CPU regression tests. It does not implement multichannel streaming, additional encoded formats, or an HTTP audio API.
Follow-up capabilities for discussion
These are proposed follow-ups, not implemented features or requirements to expand #1256. Would maintainers prefer separate tracking issues/PRs for them? I am happy to work on the agreed scope, starting with the output contract needed by #1227.