Repository navigation
Add VoxMel TTS and voice cloning - #1353
Merged
Merged
Conversation
0xShug0
force-pushed
the
feature/voxmel
branch
from
November 20, 2026 18:05
adf3a34 to
b340071
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the
voxmel_ttsfamily for VoxMel 0.6B English/Mandarin TTS and reference-audio voice cloning. Generation is offline, with framework text chunking for long text. F16 and Q8 packages include the tokenizer, frontend constants and F32 Vocos; F16 is the default and the spec status issupported.Reuses the existing causal decoder, Transformer/FFN modules, device denoiser, Vocos, RNG, text chunker and spec-v1 loader. Graphs are session-owned with bounded retention. No shared framework or GGML implementation changes.
Performance
RTX 5090 / Ryzen 9 7900X, eight threads, Debug build. Official English and Mandarin cloning examples, seed 2026, default 32 sampling steps. Results are medians of three measured requests after one warmup in the same session. All runs were sequential. Generation time excludes model loading and HTTP transport; process peak memory includes loading, warmup and measured requests.
Matched-FP32 CUDA comparison: model and vocoder weights are F32, C++ attention is eager, and Python uses its official default uncompiled inference path. Wall-time speedup is calculated from generation latency; RTF uses each output's actual duration.
Native release packages use F16/Q8 model weights, F32 Vocos and default auto attention. These results are separate from the matched-FP32 comparison above. Memory columns show the maximum process peak across the two examples; RAM is peak RSS.
RTF below one is faster than real time. CPU remains slower than real time on this machine. The model is offline-only; no streaming TTFA is reported.
Validation
voxmel_tts.Includes conversion and independent replay tests, usage documentation and license attribution. Weight files and generated fixtures are not committed. The F16/Q8 packages target
audio-cpp/audio.cpp-gguf.