Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

**weblatencytest** is a proof-of-concept web application that measures browser round-trip audio latency using an MLS (Maximum Length Sequence) signal and cross-correlation. It is a research tool associated with a WAC 2025 paper (see README.md for citation).

**Future goal (in progress):** Convert this app into a reusable Web Component that can be embedded in other Web Audio projects. v1 ships with `MediaRecorder` as the default recording backend. v2 will switch the default to `AudioWorklet` for sample-accurate raw PCM capture.
**Status:** The `<latency-test>` Web Component is live and published as `@adasp/latency-test` (v1.x). v1 ships with `MediaRecorder` (2-channel) as the default recording backend. v2 will switch the default to `AudioWorklet` for sample-accurate raw PCM capture.

---

Expand Down Expand Up @@ -62,6 +62,8 @@ src/
experiments/ — Research-only experiment pages (served by npm run dev, also published to GitHub Pages under /dev/)
mr2ch.html — MediaRecorder 2ch stereo capture feasibility experiment
mr2ch.js — Experiment logic (standalone, no component dependency)
mr1ch.html — MediaRecorder 1ch experiment: direct mic capture, exposes start-timing bias
mr1ch.js — Experiment logic (standalone, no component dependency)
assets/
ERC_logo.png
docs/
Expand Down Expand Up @@ -93,7 +95,7 @@ tests/

```
demo/
index.html — Public integration showcase: loads dist/latency-test.legacy.iife.js; grid of 8 demo panels
index.html — Public integration showcase: loads dist/latency-test.legacy.iife.js; grid of 9 demo panels
style.css — Demo-only styles: card grid, panels, result boxes, event log, audio info table
js/
common.js — Shared setup: getUserMedia + AudioContext (created once), card-grid toggle, audio info
Expand All @@ -104,6 +106,7 @@ demo/
context-share.js — Panel: Context Share — demonstrates host-managed AudioContext & MediaStream pattern
mode-toggle.js — Panel: Mode Toggle — runs MediaRecorder then AudioWorklet sequentially for A/B comparison
audioworklet.js — Panel: AudioWorklet — recording-mode="audioworklet", multi-run, aggregate stats
mr1ch.js — Panel: MediaRecorder 1ch — recording-mode="mediarecorder-1ch" fallback, multi-run, aggregate stats
lifecycle.js — Panel: Lifecycle — logs all six latency-* events with timestamps
debug.js — Panel: Debug Mode — intercepts console.debug to surface [latency-test] lines on-page
host-gain.js — Panel: Host Gain — ChannelSplitter + GainNode chain for low-level mics (e.g. Safari)
Expand Down Expand Up @@ -208,7 +211,7 @@ Results are dispatched as `CustomEvent` from the element. The demo page renders

## Current Implementation Notes

The web component refactor (Phases 1–3a) is complete. Previous design issues are resolved. Remaining known limitations:
Phases 1–3b are complete. Previous design issues are resolved. Remaining known limitations:

1. **`input-gain` not yet wired** — The attribute is observed and the property is settable, but no `GainNode` is created. Setting `input-gain` has no effect in the current code. Use the host-gain pattern instead (see `docs/examples/host-gain.md`). Deferred to v2.

Expand All @@ -224,7 +227,7 @@ The web component refactor (Phases 1–3a) is complete. Previous design issues a

## Web Component Status

Phases 1–3a are complete. The `<latency-test>` Custom Element is implemented with:
Phases 1–3b are complete. The `<latency-test>` Custom Element is implemented with:

- Shadow DOM (open mode, empty — headless-first)
- `start()` / `stop()` public methods
Expand All @@ -233,7 +236,6 @@ Phases 1–3a are complete. The `<latency-test>` Custom Element is implemented w
- `worker.js` cross-correlates two buffers: in the audioworklet path these are `{ mic, ref }` Float32 PCM; in the mediarecorder (2ch) path these are `ch0` (mic) and `ch1` (reference) from the decoded stereo recording; in the mediarecorder-1ch path these are the decoded mono recording vs the pre-generated MLS AudioBuffer

**Still in progress:**
- Phase 3b: complete ✅
- Phase 4: histogram, browser verification matrix across all three modes

**Planned configurable attributes (beyond `number-of-tests`, `mls-bits`, `max-lag-ms`):**
Expand Down
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ A Web Component for measuring browser round-trip audio latency in Web Audio appl

## What it does

<img src="./assets/MLS_Test.png" alt="MLS round-trip latency measurement diagram" width="700"/>
<img src="https://raw.githubusercontent.com/idsinge/latency-test/main/assets/MLS_Test.png" alt="MLS round-trip latency measurement diagram" width="700"/>

- Measures round-trip browser audio latency using an [MLS (Maximum Length Sequence)](https://en.wikipedia.org/wiki/Maximum_length_sequence) signal and cross-correlation
- Designed for integration into Web Audio and DAW-like web applications
Expand Down Expand Up @@ -71,11 +71,12 @@ Multiple consecutive tests with aggregate statistics:

Full integration docs are published via VitePress (see `docs/`):

- **Docs site:** https://idsinge.github.io/latency-test/
- **API reference** — attributes, methods, events, algorithm constants: `docs/api.md`
- **Framework examples** — Vanilla JS, React, Vue, Svelte, Angular, Next.js: `docs/examples/`
- **Installation** — npm, CDN, AudioContext sharing: `docs/install.md`
- **Live demo:** https://idsinge.github.io/latency-test/demo/
- **Dev & research pages:** https://idsinge.github.io/latency-test/dev/ (test pages and experiments, including the MediaRecorder 2ch stereo capture feasibility experiment)
- **Dev & research pages:** https://idsinge.github.io/latency-test/dev/ (test pages and experiments, including the MediaRecorder 2ch and 1ch experiments)

## Local development

Expand Down Expand Up @@ -105,14 +106,14 @@ npm run docs:dev
Other commands:

```bash
npm test # run unit tests (Node 18, no install needed)
npm test # run unit tests (Node 18+, no install needed)
npm run build:component # build the component bundle (dist/)
npm run build:component:legacy # legacy build: lowers private fields + optional chaining for Safari 14 / Chrome 78
npm run docs:build # build VitePress docs
npm run docs:preview # preview built docs locally
```

**Requirement:** Node.js v18 or above (project pins v18.12.1 via `.nvmrc`).
**Requirement:** Node.js v18 or above (minimum tested). Development is pinned to Node 22 via `.nvmrc`; the docs CI deploy uses Node 24.

## Repository scope

Expand Down Expand Up @@ -149,7 +150,7 @@ This project originates from research on browser round-trip audio latency presen

This work is developed as part of the project *Hybrid and Interpretable Deep Neural Audio Machines*, funded by the **European Research Council (ERC)** under the European Union's Horizon Europe research and innovation programme (grant agreement No. 101052978).

<img src="./assets/ERC_logo.png" alt="European Research Council logo" width="250"/>
<img src="https://raw.githubusercontent.com/idsinge/latency-test/main/assets/ERC_logo.png" alt="European Research Council logo" width="250"/>

We also thank [Louis Bahrman](https://github.com/Louis-Bahrman) for his collaboration on this project, including his contributions to the [Python/Google Colab notebook for MLS-based latency estimation](https://gist.github.com/gilpanal/f6a64a8fe797190bba22123dfea29611).

Expand Down
2 changes: 1 addition & 1 deletion agents/CLAUDE_REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -569,7 +569,7 @@ git push --follow-tags # pushes tag → triggers the publish workflow
- The 18 dB reliability threshold and 600 ms maxLag are research-derived constants — do not change them without asking.
- The Safari-specific `getCorrectStreamForSafari()` method is **removed in Phase 1**. Gain compensation is now a general `input-gain` property set by the host — the component applies whatever value it receives and does no browser detection internally.
- `helper.js` no longer exists — do not reference it or attempt to import from it.
- **Docs homepage expectation management:** `docs/index.md` must always carry a visible draft/work-in-progress signal near the top (currently at line 30). The homepage shows install and usage code snippets that read like a published package — without an explicit notice, readers will assume the package already exists. Do not remove or soften this notice until the package is actually published on npm.
- **Docs homepage expectation management (resolved):** while the package was unpublished, `docs/index.md` was required to carry a visible draft/work-in-progress notice so readers would not assume the package existed. `@adasp/latency-test` is now live on npm and the notice has been removed — this requirement no longer applies.
- **Phase 3 dual-channel capture is not optional:** A WAC 2025 peer reviewer identified that the current single-channel MediaRecorder approach has an uncontrolled timing offset between `mediaRecorder.start()` and `noiseSource.start()`. The AudioWorklet processor must use `numberOfInputs: 2` — mic on input 0, reference signal loopback on input 1 — and cross-correlate the two captures. The `naomiaro/recording-calibration` reference implements this correctly. Do not implement Phase 3 as a direct MediaRecorder-to-AudioWorklet port without adopting this two-channel architecture.
- **Three `recording-mode` values — each measures a different pipeline:** `"mediarecorder"` (default, dual-channel via `ChannelMergerNode` + `MediaStreamDestinationNode`, removes start-timing bias, channel-relative stable but not sample-accurate, measures a **different pipeline** due to extra Web Audio nodes — `createMediaStreamSource` is unavoidable; overhead direction is browser-dependent; emits `latency-error` if browser downmixes to mono); `"mediarecorder-1ch"` (single-channel, direct mic stream, has unknown start-timing *bias* — not just jitter — but is the closest to the production DAW recording path; use as fallback when `"mediarecorder"` fails); `"audioworklet"` (raw PCM, shared AudioContext clock, accuracy reference). Do not flatten these into a single implementation — the mode differences are research data.
- **Timing bias vs. jitter distinction:** the single-channel `"mediarecorder"` path has a *systematic timing bias* (the unknown JS start offset between `noiseSource.start()` and `mediaRecorder.start()` shifts the measured lag on every run). `maxLag` makes the correlation peak searchable but does not cancel this offset. Always use the word "bias" not "jitter" when describing this effect in docs or code comments.
Expand Down
22 changes: 16 additions & 6 deletions agents/SESSION_HANDOFF.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# SESSION_HANDOFF.md — Current State Handoff

Last updated: 2026-06-05
Last updated: 2026-06-10

LLMs reading this file: read `CLAUDE.md`, `AGENTS.md`, and `agents/CLAUDE_REVIEW.md` first. Do not modify files without explicit user approval.

Expand All @@ -11,14 +11,24 @@ See `agents/KNOWN_ISSUES.md` for open findings from code reviews (Codex, DeepSee
## Current Repo State

- Working branch: `webcomponent`. All PRs merged — branch is ahead of `main` with active development.
- Phases 1–7 complete. **v1.0.2** is live on npm as `@adasp/latency-test`.
- Phases 1–7 complete. **v1.1.0** is live on npm as `@adasp/latency-test`.
- `dist/` remains gitignored — generated with `npm run build:component`.
- `demo/` validates the built IIFE bundle via `../dist/latency-test.iife.js`. Run with `npm run build:component && npm run demo`.
- `demo/` validates the built IIFE bundle via `../dist/latency-test.legacy.iife.js`. Run with `npm run build:component:legacy && npm run demo`.
- `src/dev-test/` contains development test pages served by `npm run dev` (no build needed), also published to GitHub Pages under `/dev/`.
- `src/experiments/` contains research-only experiment pages (not part of the component test suite), also published to GitHub Pages under `/dev/`.
- GitHub Pages deploys the VitePress docs site from `docs/.vitepress/dist/` via `.github/workflows/docs.yml`. `src/` is also copied to `dist/dev/` — dev and experiment pages accessible at `https://idsinge.github.io/latency-test/dev/`.
- VitePress base is `/latency-test/` → site at `https://idsinge.github.io/latency-test/`.
- `.nvmrc` pins Node 22. `docs.yml` CI uses Node 24. CDN URLs in `docs/install.md` are pinned to `@1.0.2`.
- `.nvmrc` pins Node 22. `docs.yml` CI uses Node 24. CDN URLs in `docs/install.md` are pinned to `@1.1.0`.

---

## Recently Completed (2026-06-10)

### MediaRecorder 1ch experiment
- `src/experiments/mr1ch.html` + `mr1ch.js` added — standalone 1ch MediaRecorder experiment for comparison against `mr2ch`. Records mic directly from `inputStream` via `MediaRecorder`; plays MLS reference to `ac.destination`; cross-correlates decoded mic recording (`getChannelData(0)`) against pre-generated `noiseBuffer.getChannelData(0)`. Intentionally exposes the start-timing bias between `MediaRecorder.start()` and `noiseSource.start()` — the bias that the 2ch experiment eliminates. Includes the Codex-recommended safety improvement: `MediaRecorder.stop()` called in the `catch` path if `noiseSource.start()` throws after recording has started.
- `src/index.html` — link added under Experiments section.
- `CLAUDE.md` — file map updated; "Future goal" wording replaced with current status; phase references corrected to 1–3b throughout.
- `README.md` — dev & research pages description updated to mention both 2ch and 1ch experiments.

---

Expand Down Expand Up @@ -99,7 +109,7 @@ See `agents/KNOWN_ISSUES.md` for open findings from code reviews (Codex, DeepSee

- Do not commit `dist/` — keep it generated.
- Do not move the interactive demo into `docs/public/` — `demo/` is an integration fixture, not VitePress source.
- Always call `getUserMedia` before `new AudioContext()` in demos and host examples — required for correct Firefox macOS sample rate selection.
- Create `new AudioContext()` before `getUserMedia()` in demos and host examples — ensures the AudioContext starts in running state in Firefox, making `outputLatency` available. The demos follow this order. See `agents/SESSION_MODEL_FIX.md` for historical context on why the order was previously reversed.
- The component must not close a host-provided `AudioContext` or stop a host-provided `MediaStream`.
- Three `recording-mode` values each measure a **different pipeline** — do not flatten them. The differences are research data.
- `recording-mode` should match the host app's real capture pipeline. AudioWorklet mode measures a minimal direct graph and is a lower-bound estimate for hosts with more complex AudioWorklet graphs. See Decision #15 in `agents/CLAUDE_REVIEW.md`.
Expand All @@ -118,6 +128,6 @@ git push --follow-tags
npm publish
```

`prepublishOnly` auto-runs `npm run build:component` before every publish.
`prepublishOnly` auto-runs `npm run build:component:all` before every publish.

Version strategy: v1.x keeps `"mediarecorder"` as default; v2.0.0 switches default to `"audioworklet"` (breaking).
8 changes: 5 additions & 3 deletions agents/SESSION_MODEL_FIX.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,11 +115,13 @@ This indicates first-run cold-start on the newly created AudioContext — a sepa

## Firefox macOS — AudioContext Initialization Order

**Confirmed on Firefox macOS:** creating `AudioContext` *before* `getUserMedia` produces a different sample rate than creating it *after*. The old component demo called `#setupAudioContext()` first, then `#acquireMic()`, which caused the AudioContext to initialise at a different rate (likely 48000 Hz) than CoreAudio's mic path (typically 44100 Hz). The MLS buffer was then created at the wrong rate, giving audibly different spectral content.
> **Historical — superseded 2026-06-10.** Current guidance: create `new AudioContext()` before `getUserMedia()`. See the superseded note at the end of this section.

The fix in `index.js` follows the same order as the original `main` branch: `getUserMedia` first, `new AudioContext()` second. This incidentally resolved the sound difference on Firefox macOS as well as the stream lifetime instability.
**Confirmed on Firefox macOS (historical):** creating `AudioContext` *before* `getUserMedia` produced a different sample rate than creating it *after*. The old component demo called `#setupAudioContext()` first, then `#acquireMic()`, which caused the AudioContext to initialise at a different rate (likely 48000 Hz) than CoreAudio's mic path (typically 44100 Hz). The MLS buffer was then created at the wrong rate, giving audibly different spectral content.

**Do not swap this order.** Always call `getUserMedia` before creating the `AudioContext` in any demo or host app on this project.
The fix at the time in `index.js` followed the same order as the original `main` branch: `getUserMedia` first, `new AudioContext()` second. This incidentally resolved the sound difference on Firefox macOS as well as the stream lifetime instability.

**Superseded (2026-06-10):** The demos and experiments now create `new AudioContext()` before `getUserMedia()`. This ensures the AudioContext starts in running state in Firefox, making `outputLatency` available — which outweighs the sample rate concern that motivated the original order. The sample rate mismatch described above is no longer treated as a blocking issue.

---

Expand Down
Loading