Skip to content

Feature: preserve multichannel audio in public results and WAV output #1254

Description

@ruiling-smartbear

Motivation

While working on HeartMuLa support (#1227), I noticed that its 48 kHz stereo output cannot currently be represented unambiguously by the public audio result API. Could we add a small, mono-compatible multichannel output contract before wiring up the full codec output?

Current TRTMC interface

At a50cf5dc, AudioResult contains samples, num_samples, and sample_rate, but no channel count or layout. AudioChunkCallback likewise has no channel information. The CLI write_wav hardcodes num_channels = 1.

The sample rate is already configurable; the gap is preserving channels and defining sample-count semantics. Flattening stereo into the existing mono contract would not preserve the intended audio.

Proposed scope

  1. Add channel metadata to audio results, defaulting to mono, and document a canonical layout (for example, interleaved float samples).
  2. Explicitly distinguish total scalar samples from frames per channel, retaining existing mono behavior.
  3. Update WAV output to derive channel count, block alignment, byte rate, and data size consistently; validate malformed buffers. Audit reader behavior and document any intentional mono downmix separately.
  4. Decide how streaming can carry the same information without silently breaking existing callback users. A new callback type or overload may be preferable to changing the existing signature.
  5. Add CPU tests for unchanged mono output and stereo round trips with distinct left/right signals, including duration, sample rate, channel order, and invalid buffer lengths.

Initial implementation

PR #1256 proposes the complete-result and WAV-output portion, including channel-aware benchmark duration and CPU regression tests. It does not implement multichannel streaming, additional encoded formats, or an HTTP audio API.

Follow-up capabilities for discussion

  • Multichannel streaming output: carry channel count, sample rate, and layout with chunks; define frame boundaries and end-of-stream behavior while preserving existing mono callback users. Test that concatenated chunks preserve channel order, sample count, and duration.
  • Additional audio output formats: consider FLAC, MP3, and Opus alongside WAV, with explicit format selection, channel preservation, and clear errors when an encoder is unavailable. Agree on optional dependencies and codec limitations before implementation; lossy formats should not use byte-exact waveform comparisons as acceptance criteria.
  • HTTP audio serving: discuss whether this belongs in TRTMC's serving layer and how callers should request audio generation and receive encoded audio with the correct content type, including streaming where supported. Endpoint design, request validation, cancellation, and error propagation need a separate scoped proposal.

These are proposed follow-ups, not implemented features or requirements to expand #1256. Would maintainers prefer separate tracking issues/PRs for them? I am happy to work on the agreed scope, starting with the output contract needed by #1227.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions