Skip to content

Slowed playback: no stutter, no doubled kicks, no echo - #727

Merged
thcp merged 3 commits into
next-releasefrom
fix/slowed-playback-dropout
Sep 30, 2026
Merged

thcp merged 3 commits into
next-releasefrom
fix/slowed-playback-dropout

Conversation

@thcp

@thcp thcp commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #722
Closes #728
Closes #729

Reported and diagnosed by @goermezer in #701. This takes the engine half of their patch, with two adjustments. The slider half stays open in #701 as a separate design question.

What changes

The chunk scheduler's lookahead gate now measures against ctxTimeToSourceTime(ctx.currentTime) instead of the output playhead.

  • SoundTouch path: sources play at 1x, so the gate now counts at 1x. Below 1x it no longer overstates its margin, and the sources stay fed.
  • Tape-effect fallback: ctxTimeToSourceTime follows _srcRate(), so the gate still counts at the slowed rate there, the same as before.
  • 1x: unchanged when nothing is transposed. With a transpose it schedules the pipeline latency (a fraction of a second) earlier, which is harmless.

Differences from the patch in #701:

  1. It reuses the existing ctxTimeToSourceTime instead of adding a second clock that assumed 1x, which was wrong for the tape-effect fallback.
  2. It does not subtract the pipeline latency, since that delay sits after the sources.

Tests

tests/js/slowed-scheduling.test.mjs plays a two minute synthetic track against a fake clock and checks, every 100 ms, that the scheduled sources still reach past "now". It covers SoundTouch at 1x, SoundTouch at 0.75x and tape effect at 0.75x.

  • Without the fix, SoundTouch at 0.75x runs dry at 50.1 s.
  • With it, all 9 checks pass.

The e2e fixture is 6 seconds long, so no Playwright test can reach this.

Doubled kicks at slowed speeds (#728)

Found while testing the fix above. Below 1x the stretcher's grains overlap, and an attack in the overlap is played twice, which turns a kick into a ~50 ms flam.

  • The shared tempo stage marks attacks and, when the next grain would replay one, carries straight on instead. The splice joins identical samples, so it is seamless.
  • The time that costs is borrowed, at most 100 ms, and paid back over the next few grains. The whole mix and the click share this stage, so they stay together. Only the playhead can trail the audio, by up to 100 ms, briefly.
  • The pitch stages do not do this. They are aligned sample for sample with the unpitched drums.

Measured at 0.75x on 30 s of a full mix: 93 attacks in, 104 out before, 93 after.

tests/js/tempo-attacks.test.mjs stretches 40 synthetic kicks over a held chord. The old stretcher gives 47 attacks, the new one exactly 40, and no kick drifts past the borrow limit.

Signalsmith Stretch as the tempo stage (#729)

Below 1x, WSOLA repeats overlapping 82 ms fragments, which is heard as an echo on everything sustained, not only drums. The shared tempo stage now uses the Signalsmith Stretch WASM core (MIT), at 40 ms blocks. That size was chosen by ear against 30, 60 and the library's 120 ms default. The 120 ms default softened about a quarter of the drum attacks.

  • Vendored as static/vendor/signalsmith-stretch.js: the loader from signalsmith-stretch@1.3.2, unchanged, with its MIT license. The package's own AudioWorkletNode is not used, because it ignores the rate on live input.
  • Falls back to WSOLA if the core does not load. The swap happens only at a flush, never under audio already playing.
  • Latency is reported by the processor, and both engines compute it from one shared module, static/js/tempoStage.js. Measured end to end, it matches within 1 ms at 44.1 and 48 kHz.
  • CSP: script-src gains 'wasm-unsafe-eval', in both the server and Tauri CSPs. It allows WebAssembly compilation only; JS eval stays blocked. test_csp.py now pins script-src to exactly those two sources.
  • CPU: about 2.5% of one core at 44.1 kHz, offline.

On 20 s of a full mix at 0.75x (55 attacks in): WSOLA 60 attacks out with 8 close pairs, Signalsmith 61 with 2.

Tests:

  • tests/js/signalsmith-stage.test.mjs: the swap, the fallback, latency within 1 ms, and 40 kicks out as 40 with no drift.
  • tests/e2e/tempo-stage.spec.mjs: in real Chromium, slowed playback reports the Signalsmith stage. With 'wasm-unsafe-eval' removed it fails and reports wsola, which also shows the fallback working in a browser.

Found along the way, not changed here: the WSOLA path's latency as the engines count it (122 ms) is well under what I measure end to end (about 300 ms at 0.75x). Only the fallback still uses that path.

Gate

  • node --check on all of static/js, every tests/js test, i18n coverage clean, uv.lock untouched.
  • Playwright: 411 passed on the final run. retention-setting and lyrics-align each failed once in earlier runs and passed on rerun with and without these changes.
  • pitch-shift.test.mjs still passes, so the pitch stages are unaffected.
  • pytest: the 8 known machine-local ffmpeg failures only. ruff and bandit clean.
  • Rust: cargo fmt --check, clippy -D warnings and 116 tests pass (tauri.conf.json changed).

Thales added 2 commits September 30, 2026 07:23
The lookahead gate compared _scheduledTo, in source seconds, against the
output playhead. With SoundTouch stretching, sources play at 1x and the
worklet buffers the surplus, so below 1x the output playhead lags what
the sources have consumed. The gate overstated its margin by (1 - rate)
seconds per second until the sources ran dry and the next chunk was
scheduled late and skipped. At 0.75x that happened about fifty seconds in.

The gate now measures against ctxTimeToSourceTime, which already follows
the source rate, so it also holds in the tape-effect fallback.

Reported and diagnosed by @goermezer in #701.

Closes #722
Below 1x each WSOLA sequence plays 70 ms of input but advances only
70 * tempo, so consecutive sequences overlap and an attack in the
overlap is heard twice. At 0.75x kicks came out as a ~50 ms flam.

The shared tempo stage now marks attacks on the sample-to-sample
difference, and when the next sequence would replay one it carries
straight on from where the last sequence ended instead. That splices
identical samples, so it is seamless. The extra input is borrowed, at
most 100 ms, and paid back by later sequences stretching slightly more.

Only the shared tempo stage does this. The pitch stages are aligned
sample for sample with the unpitched drums, and borrowing there would
pull the band off the kit.

Measured on a full mix at 0.75x: 93 attacks in, 104 out before, 93 out
after.

Closes #728
@thcp thcp changed the title Keep slowed playback from stuttering and skipping Keep slowed playback from stuttering, skipping and doubling kicks Sep 30, 2026
WSOLA slows audio by repeating overlapping 82 ms fragments, heard as an
echo on everything sustained. The shared tempo stage now uses the
Signalsmith Stretch WASM core (MIT, vendored from signalsmith-stretch
1.3.2) at 40 ms blocks, chosen by ear against 30, 60 and 120 ms.

- The core is its own worklet module, added before the processor. The
  processor instantiates it asynchronously and swaps it in at the next
  flush, never under audio already playing. If it does not load, WSOLA
  stays the tempo stage.
- The processor reports which stage runs and its latency in two parts.
  Latency is the priming plus the core's input side, divided by the
  tempo, plus its output side; measured end to end it lands within 1 ms.
  Both engines take it from one shared module, tempoStage.js, instead of
  each hard-coding WSOLA's.
- script-src gains 'wasm-unsafe-eval', in the server CSP and the Tauri
  CSP. It allows WebAssembly compilation only, not JS eval. The CSP test
  now pins script-src to exactly 'self' and 'wasm-unsafe-eval'.

On 20 s of a full mix at 0.75x (55 attacks in): WSOLA 60 attacks out
with 8 close pairs, Signalsmith 61 with 2.

Closes #729
@thcp thcp changed the title Keep slowed playback from stuttering, skipping and doubling kicks Slowed playback: no stutter, no doubled kicks, no echo Sep 30, 2026
@thcp
thcp changed the base branch from main to next-release September 30, 2026 10:55
@thcp
thcp merged commit 8eed12b into next-release Sep 30, 2026
15 checks passed
@thcp thcp mentioned this pull request Oct 1, 2026
thcp added a commit that referenced this pull request Oct 2, 2026
Release 0.19.1: everything on `next-release` since v0.19.0.

- #727: slowed playback without stutter, doubled kicks or echo
(Signalsmith Stretch as the tempo stage).
- #737: the bugs reported against 0.19.0 (CUDA setup, torch that cannot
load, favourites in a narrow window, minor keys, the key label).
- #740: the remaining open issues (top bar and Extract stems, the card
at every width, favourites on the phone, the lyrics lookup on the
server, an adjustable slow speed, Rust advisories).
- #741: trivy ignores the two urllib3 CVEs until the lock next moves.
- #743: a 0.5x button beside the slow speed.

Verified on local Windows NVIDIA builds: 0.19.1.dev0 passed 12 of 12
(#727, #737), and 0.19.1.dev1 passed 18 of 18 automated checks including
#740, in an isolated profile. `uv.lock` is unchanged since v0.19.0, so
the in-app update works.

Closes #722
Closes #728
Closes #729
Closes #723
Closes #724
Closes #726
Closes #730
Closes #731
Closes #732
Closes #733
Closes #736
Closes #701
Closes #738
Closes #725
Closes #735
Closes #734
Closes #719
Closes #742
@thcp
thcp deleted the fix/slowed-playback-dropout branch October 5, 2026 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant