Skip to content

feat(tts): add VoxCPM as a TTS provider - #801

Open
Caldalis wants to merge 4 commits into
codedogQBY:mainfrom
Caldalis:feat/voxcpm-tts
Open

Caldalis wants to merge 4 commits into
codedogQBY:mainfrom
Caldalis:feat/voxcpm-tts

Conversation

@Caldalis

Copy link
Copy Markdown

What

Adds VoxCPM (OpenBMB, Apache-2.0) as a TTS provider, pointed at a self-hosted OpenAI-compatible endpoint.

VoxCPM served through vLLM-Omni speaks /v1/audio/speech, the same shape the existing openai-compatible provider already talks. It gets its own engine id for three reasons:

  • fetchOpenAITTSAudio throws on an empty API key, but a self-hosted server has no auth. Authorization is now sent only when a key is configured.
  • Defaults point at a local server: http://localhost:8000/v1, openbmb/VoxCPM2, wav.
  • Voice Design, a natural-language voice description VoxCPM2 takes as a parenthesized prefix on the text. No other provider has an equivalent.

Playback reuses BufferedAudioTTSPlayer (now exported) since VoxCPM returns a whole file per chunk. Mobile goes through the existing TrackPlayerCloudTTSPlayer.

Bug fixes

Three pre-existing bugs, each of which blocks the provider outright, so they are here rather than in a separate PR. Each is its own commit.

  • A suspended AudioContext was never resumed in BufferedAudioTTSPlayer, so playback was silent with no UI feedback at all.
  • The initial store state was never normalized. normalizeTTSConfig is what fills in the built-in profiles, and the persist layer only calls it when a config file already exists, so with no tts.json on disk the settings profile list was empty for every provider.
  • Buffered synthesis failures were logged and then reported as a normal end of playback, and stop() never cancelled the in-flight request because the abort signal was never passed to fetchAudio.

DashScopeTTSPlayer and PCMStreamingTTSPlayer have the same AudioContext gap and are deliberately left alone: verifying either needs provider credentials I do not have. Happy to open a separate issue.

Testing

  • pnpm test: 619 pass (598 on main), 21 new
  • pnpm lint: no new findings
  • tsc --noEmit on core, app and app-expo: matches the main baseline
  • Desktop app by hand: provider shows up in settings and in the reader engine pickers, and preview plays

Voice Design verified on VoxCPM2 (RTX 4090, bfloat16, RTF 0.32, 48kHz). Same sentence, four voice descriptions: ASR transcripts contain the sentence and none of the description wording, so the prefix is consumed as a control signal rather than spoken. Median F0 follows the description, 267.8 Hz for "young woman" against 110.7 Hz for "older man", with non-overlapping P10–P90 ranges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant