From 58b335cae49c8d645d52130ab697907ae4811f9a Mon Sep 17 00:00:00 2001 From: gilpanal Date: Thu, 11 Jun 2026 17:13:12 +0200 Subject: [PATCH] feat: add MediaRecorder 1ch experiment and demo panel, doc consistency fixes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - src/experiments/mr1ch.{html,js}: standalone 1ch MediaRecorder research experiment — direct mic capture, exposes the start-timing bias that the 2ch experiment eliminates; linked from src/index.html - demo: new "MediaRecorder 1ch" panel showcasing recording-mode="mediarecorder-1ch" (multi-run, aggregates, Chrome 78 baseline), mirroring the AudioWorklet panel - src/experiments/mr2ch.js: AudioContext-before-getUserMedia ordering, MediaRecorder.stop() safety in catch path - README: docs site link, npm-safe absolute image URLs, Node version wording (18+ minimum, 22 via .nvmrc, 24 in docs CI) - docs: index.md snippet now creates AudioContext before getUserMedia; install.md Node requirement aligned with package engines field - agents docs: prepublishOnly claim corrected, draft-notice guidance marked resolved, session handoff updated Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 12 +- README.md | 11 +- agents/CLAUDE_REVIEW.md | 2 +- agents/SESSION_HANDOFF.md | 22 ++- agents/SESSION_MODEL_FIX.md | 8 +- demo/index.html | 50 ++++++ demo/js/mr1ch.js | 64 +++++++ docs/index.md | 2 +- docs/install.md | 2 +- src/experiments/mr1ch.html | 145 ++++++++++++++++ src/experiments/mr1ch.js | 322 ++++++++++++++++++++++++++++++++++++ src/experiments/mr2ch.js | 6 +- src/index.html | 1 + 13 files changed, 624 insertions(+), 23 deletions(-) create mode 100644 demo/js/mr1ch.js create mode 100644 src/experiments/mr1ch.html create mode 100644 src/experiments/mr1ch.js diff --git a/CLAUDE.md b/CLAUDE.md index c37dd23..9588217 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,7 +10,7 @@ **weblatencytest** is a proof-of-concept web application that measures browser round-trip audio latency using an MLS (Maximum Length Sequence) signal and cross-correlation. It is a research tool associated with a WAC 2025 paper (see README.md for citation). -**Future goal (in progress):** Convert this app into a reusable Web Component that can be embedded in other Web Audio projects. v1 ships with `MediaRecorder` as the default recording backend. v2 will switch the default to `AudioWorklet` for sample-accurate raw PCM capture. +**Status:** The `` Web Component is live and published as `@adasp/latency-test` (v1.x). v1 ships with `MediaRecorder` (2-channel) as the default recording backend. v2 will switch the default to `AudioWorklet` for sample-accurate raw PCM capture. --- @@ -62,6 +62,8 @@ src/ experiments/ — Research-only experiment pages (served by npm run dev, also published to GitHub Pages under /dev/) mr2ch.html — MediaRecorder 2ch stereo capture feasibility experiment mr2ch.js — Experiment logic (standalone, no component dependency) + mr1ch.html — MediaRecorder 1ch experiment: direct mic capture, exposes start-timing bias + mr1ch.js — Experiment logic (standalone, no component dependency) assets/ ERC_logo.png docs/ @@ -93,7 +95,7 @@ tests/ ``` demo/ - index.html — Public integration showcase: loads dist/latency-test.legacy.iife.js; grid of 8 demo panels + index.html — Public integration showcase: loads dist/latency-test.legacy.iife.js; grid of 9 demo panels style.css — Demo-only styles: card grid, panels, result boxes, event log, audio info table js/ common.js — Shared setup: getUserMedia + AudioContext (created once), card-grid toggle, audio info @@ -104,6 +106,7 @@ demo/ context-share.js — Panel: Context Share — demonstrates host-managed AudioContext & MediaStream pattern mode-toggle.js — Panel: Mode Toggle — runs MediaRecorder then AudioWorklet sequentially for A/B comparison audioworklet.js — Panel: AudioWorklet — recording-mode="audioworklet", multi-run, aggregate stats + mr1ch.js — Panel: MediaRecorder 1ch — recording-mode="mediarecorder-1ch" fallback, multi-run, aggregate stats lifecycle.js — Panel: Lifecycle — logs all six latency-* events with timestamps debug.js — Panel: Debug Mode — intercepts console.debug to surface [latency-test] lines on-page host-gain.js — Panel: Host Gain — ChannelSplitter + GainNode chain for low-level mics (e.g. Safari) @@ -208,7 +211,7 @@ Results are dispatched as `CustomEvent` from the element. The demo page renders ## Current Implementation Notes -The web component refactor (Phases 1–3a) is complete. Previous design issues are resolved. Remaining known limitations: +Phases 1–3b are complete. Previous design issues are resolved. Remaining known limitations: 1. **`input-gain` not yet wired** — The attribute is observed and the property is settable, but no `GainNode` is created. Setting `input-gain` has no effect in the current code. Use the host-gain pattern instead (see `docs/examples/host-gain.md`). Deferred to v2. @@ -224,7 +227,7 @@ The web component refactor (Phases 1–3a) is complete. Previous design issues a ## Web Component Status -Phases 1–3a are complete. The `` Custom Element is implemented with: +Phases 1–3b are complete. The `` Custom Element is implemented with: - Shadow DOM (open mode, empty — headless-first) - `start()` / `stop()` public methods @@ -233,7 +236,6 @@ Phases 1–3a are complete. The `` Custom Element is implemented w - `worker.js` cross-correlates two buffers: in the audioworklet path these are `{ mic, ref }` Float32 PCM; in the mediarecorder (2ch) path these are `ch0` (mic) and `ch1` (reference) from the decoded stereo recording; in the mediarecorder-1ch path these are the decoded mono recording vs the pre-generated MLS AudioBuffer **Still in progress:** -- Phase 3b: complete ✅ - Phase 4: histogram, browser verification matrix across all three modes **Planned configurable attributes (beyond `number-of-tests`, `mls-bits`, `max-lag-ms`):** diff --git a/README.md b/README.md index 6fb077b..589cb69 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ A Web Component for measuring browser round-trip audio latency in Web Audio appl ## What it does -MLS round-trip latency measurement diagram +MLS round-trip latency measurement diagram - Measures round-trip browser audio latency using an [MLS (Maximum Length Sequence)](https://en.wikipedia.org/wiki/Maximum_length_sequence) signal and cross-correlation - Designed for integration into Web Audio and DAW-like web applications @@ -71,11 +71,12 @@ Multiple consecutive tests with aggregate statistics: Full integration docs are published via VitePress (see `docs/`): +- **Docs site:** https://idsinge.github.io/latency-test/ - **API reference** — attributes, methods, events, algorithm constants: `docs/api.md` - **Framework examples** — Vanilla JS, React, Vue, Svelte, Angular, Next.js: `docs/examples/` - **Installation** — npm, CDN, AudioContext sharing: `docs/install.md` - **Live demo:** https://idsinge.github.io/latency-test/demo/ -- **Dev & research pages:** https://idsinge.github.io/latency-test/dev/ (test pages and experiments, including the MediaRecorder 2ch stereo capture feasibility experiment) +- **Dev & research pages:** https://idsinge.github.io/latency-test/dev/ (test pages and experiments, including the MediaRecorder 2ch and 1ch experiments) ## Local development @@ -105,14 +106,14 @@ npm run docs:dev Other commands: ```bash -npm test # run unit tests (Node 18, no install needed) +npm test # run unit tests (Node 18+, no install needed) npm run build:component # build the component bundle (dist/) npm run build:component:legacy # legacy build: lowers private fields + optional chaining for Safari 14 / Chrome 78 npm run docs:build # build VitePress docs npm run docs:preview # preview built docs locally ``` -**Requirement:** Node.js v18 or above (project pins v18.12.1 via `.nvmrc`). +**Requirement:** Node.js v18 or above (minimum tested). Development is pinned to Node 22 via `.nvmrc`; the docs CI deploy uses Node 24. ## Repository scope @@ -149,7 +150,7 @@ This project originates from research on browser round-trip audio latency presen This work is developed as part of the project *Hybrid and Interpretable Deep Neural Audio Machines*, funded by the **European Research Council (ERC)** under the European Union's Horizon Europe research and innovation programme (grant agreement No. 101052978). -European Research Council logo +European Research Council logo We also thank [Louis Bahrman](https://github.com/Louis-Bahrman) for his collaboration on this project, including his contributions to the [Python/Google Colab notebook for MLS-based latency estimation](https://gist.github.com/gilpanal/f6a64a8fe797190bba22123dfea29611). diff --git a/agents/CLAUDE_REVIEW.md b/agents/CLAUDE_REVIEW.md index 4a8d77b..2428dcc 100644 --- a/agents/CLAUDE_REVIEW.md +++ b/agents/CLAUDE_REVIEW.md @@ -569,7 +569,7 @@ git push --follow-tags # pushes tag → triggers the publish workflow - The 18 dB reliability threshold and 600 ms maxLag are research-derived constants — do not change them without asking. - The Safari-specific `getCorrectStreamForSafari()` method is **removed in Phase 1**. Gain compensation is now a general `input-gain` property set by the host — the component applies whatever value it receives and does no browser detection internally. - `helper.js` no longer exists — do not reference it or attempt to import from it. -- **Docs homepage expectation management:** `docs/index.md` must always carry a visible draft/work-in-progress signal near the top (currently at line 30). The homepage shows install and usage code snippets that read like a published package — without an explicit notice, readers will assume the package already exists. Do not remove or soften this notice until the package is actually published on npm. +- **Docs homepage expectation management (resolved):** while the package was unpublished, `docs/index.md` was required to carry a visible draft/work-in-progress notice so readers would not assume the package existed. `@adasp/latency-test` is now live on npm and the notice has been removed — this requirement no longer applies. - **Phase 3 dual-channel capture is not optional:** A WAC 2025 peer reviewer identified that the current single-channel MediaRecorder approach has an uncontrolled timing offset between `mediaRecorder.start()` and `noiseSource.start()`. The AudioWorklet processor must use `numberOfInputs: 2` — mic on input 0, reference signal loopback on input 1 — and cross-correlate the two captures. The `naomiaro/recording-calibration` reference implements this correctly. Do not implement Phase 3 as a direct MediaRecorder-to-AudioWorklet port without adopting this two-channel architecture. - **Three `recording-mode` values — each measures a different pipeline:** `"mediarecorder"` (default, dual-channel via `ChannelMergerNode` + `MediaStreamDestinationNode`, removes start-timing bias, channel-relative stable but not sample-accurate, measures a **different pipeline** due to extra Web Audio nodes — `createMediaStreamSource` is unavoidable; overhead direction is browser-dependent; emits `latency-error` if browser downmixes to mono); `"mediarecorder-1ch"` (single-channel, direct mic stream, has unknown start-timing *bias* — not just jitter — but is the closest to the production DAW recording path; use as fallback when `"mediarecorder"` fails); `"audioworklet"` (raw PCM, shared AudioContext clock, accuracy reference). Do not flatten these into a single implementation — the mode differences are research data. - **Timing bias vs. jitter distinction:** the single-channel `"mediarecorder"` path has a *systematic timing bias* (the unknown JS start offset between `noiseSource.start()` and `mediaRecorder.start()` shifts the measured lag on every run). `maxLag` makes the correlation peak searchable but does not cancel this offset. Always use the word "bias" not "jitter" when describing this effect in docs or code comments. diff --git a/agents/SESSION_HANDOFF.md b/agents/SESSION_HANDOFF.md index 53e0886..a92eec6 100644 --- a/agents/SESSION_HANDOFF.md +++ b/agents/SESSION_HANDOFF.md @@ -1,6 +1,6 @@ # SESSION_HANDOFF.md — Current State Handoff -Last updated: 2026-06-05 +Last updated: 2026-06-10 LLMs reading this file: read `CLAUDE.md`, `AGENTS.md`, and `agents/CLAUDE_REVIEW.md` first. Do not modify files without explicit user approval. @@ -11,14 +11,24 @@ See `agents/KNOWN_ISSUES.md` for open findings from code reviews (Codex, DeepSee ## Current Repo State - Working branch: `webcomponent`. All PRs merged — branch is ahead of `main` with active development. -- Phases 1–7 complete. **v1.0.2** is live on npm as `@adasp/latency-test`. +- Phases 1–7 complete. **v1.1.0** is live on npm as `@adasp/latency-test`. - `dist/` remains gitignored — generated with `npm run build:component`. -- `demo/` validates the built IIFE bundle via `../dist/latency-test.iife.js`. Run with `npm run build:component && npm run demo`. +- `demo/` validates the built IIFE bundle via `../dist/latency-test.legacy.iife.js`. Run with `npm run build:component:legacy && npm run demo`. - `src/dev-test/` contains development test pages served by `npm run dev` (no build needed), also published to GitHub Pages under `/dev/`. - `src/experiments/` contains research-only experiment pages (not part of the component test suite), also published to GitHub Pages under `/dev/`. - GitHub Pages deploys the VitePress docs site from `docs/.vitepress/dist/` via `.github/workflows/docs.yml`. `src/` is also copied to `dist/dev/` — dev and experiment pages accessible at `https://idsinge.github.io/latency-test/dev/`. - VitePress base is `/latency-test/` → site at `https://idsinge.github.io/latency-test/`. -- `.nvmrc` pins Node 22. `docs.yml` CI uses Node 24. CDN URLs in `docs/install.md` are pinned to `@1.0.2`. +- `.nvmrc` pins Node 22. `docs.yml` CI uses Node 24. CDN URLs in `docs/install.md` are pinned to `@1.1.0`. + +--- + +## Recently Completed (2026-06-10) + +### MediaRecorder 1ch experiment +- `src/experiments/mr1ch.html` + `mr1ch.js` added — standalone 1ch MediaRecorder experiment for comparison against `mr2ch`. Records mic directly from `inputStream` via `MediaRecorder`; plays MLS reference to `ac.destination`; cross-correlates decoded mic recording (`getChannelData(0)`) against pre-generated `noiseBuffer.getChannelData(0)`. Intentionally exposes the start-timing bias between `MediaRecorder.start()` and `noiseSource.start()` — the bias that the 2ch experiment eliminates. Includes the Codex-recommended safety improvement: `MediaRecorder.stop()` called in the `catch` path if `noiseSource.start()` throws after recording has started. +- `src/index.html` — link added under Experiments section. +- `CLAUDE.md` — file map updated; "Future goal" wording replaced with current status; phase references corrected to 1–3b throughout. +- `README.md` — dev & research pages description updated to mention both 2ch and 1ch experiments. --- @@ -99,7 +109,7 @@ See `agents/KNOWN_ISSUES.md` for open findings from code reviews (Codex, DeepSee - Do not commit `dist/` — keep it generated. - Do not move the interactive demo into `docs/public/` — `demo/` is an integration fixture, not VitePress source. -- Always call `getUserMedia` before `new AudioContext()` in demos and host examples — required for correct Firefox macOS sample rate selection. +- Create `new AudioContext()` before `getUserMedia()` in demos and host examples — ensures the AudioContext starts in running state in Firefox, making `outputLatency` available. The demos follow this order. See `agents/SESSION_MODEL_FIX.md` for historical context on why the order was previously reversed. - The component must not close a host-provided `AudioContext` or stop a host-provided `MediaStream`. - Three `recording-mode` values each measure a **different pipeline** — do not flatten them. The differences are research data. - `recording-mode` should match the host app's real capture pipeline. AudioWorklet mode measures a minimal direct graph and is a lower-bound estimate for hosts with more complex AudioWorklet graphs. See Decision #15 in `agents/CLAUDE_REVIEW.md`. @@ -118,6 +128,6 @@ git push --follow-tags npm publish ``` -`prepublishOnly` auto-runs `npm run build:component` before every publish. +`prepublishOnly` auto-runs `npm run build:component:all` before every publish. Version strategy: v1.x keeps `"mediarecorder"` as default; v2.0.0 switches default to `"audioworklet"` (breaking). diff --git a/agents/SESSION_MODEL_FIX.md b/agents/SESSION_MODEL_FIX.md index 30a9dd5..3a4cd2e 100644 --- a/agents/SESSION_MODEL_FIX.md +++ b/agents/SESSION_MODEL_FIX.md @@ -115,11 +115,13 @@ This indicates first-run cold-start on the newly created AudioContext — a sepa ## Firefox macOS — AudioContext Initialization Order -**Confirmed on Firefox macOS:** creating `AudioContext` *before* `getUserMedia` produces a different sample rate than creating it *after*. The old component demo called `#setupAudioContext()` first, then `#acquireMic()`, which caused the AudioContext to initialise at a different rate (likely 48000 Hz) than CoreAudio's mic path (typically 44100 Hz). The MLS buffer was then created at the wrong rate, giving audibly different spectral content. +> **Historical — superseded 2026-06-10.** Current guidance: create `new AudioContext()` before `getUserMedia()`. See the superseded note at the end of this section. -The fix in `index.js` follows the same order as the original `main` branch: `getUserMedia` first, `new AudioContext()` second. This incidentally resolved the sound difference on Firefox macOS as well as the stream lifetime instability. +**Confirmed on Firefox macOS (historical):** creating `AudioContext` *before* `getUserMedia` produced a different sample rate than creating it *after*. The old component demo called `#setupAudioContext()` first, then `#acquireMic()`, which caused the AudioContext to initialise at a different rate (likely 48000 Hz) than CoreAudio's mic path (typically 44100 Hz). The MLS buffer was then created at the wrong rate, giving audibly different spectral content. -**Do not swap this order.** Always call `getUserMedia` before creating the `AudioContext` in any demo or host app on this project. +The fix at the time in `index.js` followed the same order as the original `main` branch: `getUserMedia` first, `new AudioContext()` second. This incidentally resolved the sound difference on Firefox macOS as well as the stream lifetime instability. + +**Superseded (2026-06-10):** The demos and experiments now create `new AudioContext()` before `getUserMedia()`. This ensures the AudioContext starts in running state in Firefox, making `outputLatency` available — which outweighs the sample rate concern that motivated the original order. The sample rate mismatch described above is no longer treated as a blocking issue. --- diff --git a/demo/index.html b/demo/index.html index 5243e79..854722c 100644 --- a/demo/index.html +++ b/demo/index.html @@ -58,6 +58,10 @@

@adasp/latency-test · Demo

AudioWorklet Raw Float32 PCM capture — planned v2 default backend + + + + + + +
+ Configuration + 
<latency-test id="mr1ch-tester" recording-mode="mediarecorder-1ch"></latency-test>
+
+<script>
+  const tester = document.querySelector('#mr1ch-tester')
+  tester.inputStream = hostStream         // host-managed MediaStream
+  tester.audioContext = hostAudioContext  // host-managed AudioContext
+  tester.numberOfTests = 5
+
+  tester.addEventListener('latency-result', e => {
+    console.log(`${e.detail.latency.toFixed(2)} ms · ${e.detail.reliable ? 'reliable' : 'unreliable'}`)
+  })
+  tester.addEventListener('latency-error', e => {
+    console.error(e.detail.message)
+  })
+
+  tester.start()
+</script>
+
+ +