Skip to content

feat(control): OSC output, so the visualizer can drive other software - #2

Open
Barbaror4 wants to merge 9 commits into
mainfrom
feat/osc-output
Open

Barbaror4 wants to merge 9 commits into
mainfrom
feat/osc-output

Conversation

@Barbaror4

Copy link
Copy Markdown
Owner

Closes CaYatur#437

Note on the diff: this fork's main is one merge behind CaYatur/SoundVisualizer main, so the already-merged Stereo Trails commits (CaYatur#732: f9d0a32, 7f95e03, ecdea02, 619580c, 6a1900d) also appear here. The OSC output change itself is the single commit 1afca3a.

What

The app already listens to OSC. This adds the other direction: audio levels, drum hits, the tempo with a beat clock, and scene, mode and MilkDrop preset changes go out as OSC messages to Resolume, TouchDesigner, Max or a lighting desk.

Address (prefix /cayadev) Type When
/level /bass /mid /treble f, 0..1 at the chosen rate, up to 30 Hz
/hit/kick /hit/snare /hit/hat f, strength when a hit happens
/tempo/bpm /tempo/confidence f on change, else once a second
/tempo/beat i i each tempo beat: beat in the bar (1..N) and N
/scene /visualizer /background /milkdrop/preset s on change, and again every 5 s

The repeat lets a receiver that starts late still learn the current state.

How

  • src/main/osc-out.js (new) has three parts, in the style of the existing OSC input:
    • a hand-written OSC 1.0 encoder;
    • a pure planner that decides what to send each tick, testable without a network;
    • a UDP sender with Art-Net's lifecycle. The host name is resolved once at start, not on every packet.
  • Levels and hits come from the visualizer's ~30 Hz audio-meter message, the same hook Art-Net and the lights use. A hit lasts one frame, so the visualizer keeps the strongest kick, snare and hat seen since the last message.
  • The beat clock comes from the panel's tempo engine (autovj.js), which reports each beat, but only while OSC output is on. main accepts tempo:beat only from the admin window.
  • Settings: a new top-level oscOut section next to artnet, with the same MCP permission (output in PATH_GROUPS). control stays midi/osc only, so MCP's control validator is unchanged.
  • Panel: an OSC Çıkışı / OSC Output card in Control, with target, port, rate, prefix and one switch per group. Its live packet counter polls only while the card is on screen.

Safety

  • Off by default. The default target is 127.0.0.1:9001; the OSC input listens on 9000.
  • Feedback-loop guard. It refuses to send when the target is this computer's open OSC input on the same port. That covers loopback, the machine's own addresses and its subnet broadcast, since the packets would feed back into mapped settings.
  • Re-validation in main. Every field (host, port, rate, prefix) is checked again in the main process, so an MCP write or a hand-edited settings file falls back to safe defaults.
  • Silent in automation. Like Art-Net, nothing is sent in smoke or screenshot runs (HW_OFF).

Found by running the real app

Levels arrived at 17 Hz with the rate set to 30. An interval threshold dropped every meter message that arrived a millisecond early. The rate is now kept by a schedule. Re-measured in the app: 29.6 Hz, median gap 33 ms. A test reproduces the old behaviour: 205 sends in 10 s where ~300 are expected.

Checks

  • npm test: 2959 pass, 17 new in tests/osc-out.test.js:
    • the two example packets from the OSC 1.0 specification, byte for byte;
    • every planned message decoded by the app's own parser (parsePacket is now exported from osc-server.js for this);
    • the rate under meter jitter, and hits, tempo, state, beat, prefix and settings validation;
    • the loop guard;
    • two real UDP round trips over loopback.
  • Mutations: 18 of 18 caught (encoding, rate, clamping, tempo/state timing, beat bounds, loop guard, host and prefix checks, the panel-only beat sender, HW_OFF).
  • Live, in the real app: isolated --user-data-dir, with the floating window opened through MCP. 1383 packets arrived in 12 s, with levels at 29.6/s, kick/snare/hat hits, beats cycling 1/4..4/4, and the mode switch reported 0.01 s after it happened.
  • npm run smoke (isolated --user-data-dir): PASS. Its crawl reported no untranslated text in the English UI.

For the reviewer

  • Levels need an open visualizer. Like Art-Net and the lights, levels and hits are sent only while a visualizer window (or the floating window) is open. Tempo, the beat clock and state come from the panel and main.
  • Messages, not bundles. Some simple receivers (Max/Pd patches, some lighting desks) don't open #bundle, and ~120 small packets a second are trivial on loopback or a LAN.
  • UTF-8 strings. OSC 1.0 says ASCII, but scene names can be Turkish, so strings are UTF-8 (as Resolume, TouchDesigner and Max read them), truncated at a character boundary at 200 bytes.

🤖 Generated with Claude Code

Barbaror4 and others added 9 commits October 11, 2026 11:58
…reo image

Refs CaYatur#638

A new visualizer in Particles & Events. It is a fountain: every frequency
band launches its own particles. The launch angle comes from where the band
sits between the left and right channel, the scatter from how diffuse it
is, and the count and height from the band's level. A centred kick draws an
upright column, a guitar panned left bends its arcs to the left, a wide
reverb opens like a fan. Mono input stands straight up, which is correct.

The stereo measurement is new: src/shared/stereo-bands.js runs a band-pass
biquad per band over timeL/timeR and keeps sum(L^2), sum(R^2), sum(LR):
- pan = (R^2 - L^2) / (R^2 + L^2), exact at hard left, centre, hard right
  (sin(p*pi/2) under constant-power panning);
- spread = 1 - correlation: a single panned source is 0 wherever it sits,
  decorrelated or anti-phase sound is 1. analysis.js's width gives a
  hard-panned mono guitar and a wide reverb the same value (1), so it is
  deliberately not reused here.
Silence and 8-bit quantisation noise read as centred. 0.07 ms per frame
for 8 bands in Node; no allocation per frame.

Levels come from audio.getBars, so every control the catalog flags reveal
works: bar count (band count), frequency range, the whole Spectrum
Measurement group, line width, amplitude and the secondary colour. The
stereo bands use the spectrum's own edges (SVSpectrum.bandEdges), so with a
mel or bark scale the level and the pan come from the same band.

Drawn with source-over and butt caps: additive blending washed out on the
default light background, and round caps cost frames on integrated
graphics (Iris Xe, 1080p, uncapped: 118/92 fps with round, 124/118 butt).

Checks:
- npm test: 2941 pass (16 new); doc counts updated.
- tests/stereo-bands.test.js runs the real mode file in a vm with a
  recording canvas: left/right/mono lean, diffuse sound fans out, each
  band in its own place, levels from getBars, deterministic output, and
  every page loads the helper. 10 of 10 mutations caught.
- npm run smoke (isolated --user-data-dir): PASS, 60 visualizer modes drawn.
- npm run bench, vsync on, 1080p on Iris Xe: 58.2 / 60.0 / 58.4 fps,
  p95 16.9 ms in three runs (Goniometer 59.4 / 60.0, Bouncing Balls 60.0).
- The mode sheet in docs/screenshots is not regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fountain sat at 94% of the canvas, so a full screen left a dead band under the launch line. The floor is now the bottom edge.

trailHeight scales those same arcs. 1 keeps today's peak. Up to 3 lets them leave the top of the frame and fall back. The slider sits on the layer and on the classic visualizer card, and the MIDI/OSC target uses the same range.
The sheet is drawn from the catalogue and now holds 60 modes, including Stereo Trails. That mode waits with the other trail builders so its tile is not an empty first frame.
Uncapped, the full-screen trail and glow were submitted hundreds of times a second, faster than the GPU finished them. That backlog was charged to the following scene. Painting is now capped at 120 Hz. Physics still steps every frame, and 60 Hz and 75 Hz displays still paint every frame.

On this machine, uncapped 1080p Bars read 663 fps before Stereo Trails and 380 fps after. After the cap it reads 663 fps before and 665 fps after. With vsync, Stereo Trails and the next scene both hold 75 fps.
feat(modes): Stereo Trails, each band rises from its place in the stereo image
Closes CaYatur#437

The app already listens to OSC; this adds the other direction. Audio
levels, drum hits, the tempo with a beat clock, and scene, mode and
MilkDrop preset changes go out as OSC messages to Resolume,
TouchDesigner, Max or a lighting desk.

Addresses (prefix /cayadev by default, the input's namespace):
- /level /bass /mid /treble (f, 0..1) at the chosen rate, up to 30 Hz
- /hit/kick /hit/snare /hit/hat (f, strength) when a hit happens
- /tempo/bpm /tempo/confidence (f) on change, else once a second
- /tempo/beat (i i) beat in the bar 1..N and N, from the tempo engine
- /scene /visualizer /background /milkdrop/preset (s) on change and
  again every 5 s, so a receiver that starts late still learns them

How it is wired:
- src/main/osc-out.js: a hand-written OSC 1.0 encoder (like the input
  parser), a pure planner that decides what to send each tick, and a
  UDP sender with Art-Net's lifecycle. The host name is resolved once.
- The visualizer's ~30 Hz audio-meter message carries the strongest
  kick/snare/hat hit seen since the last message (a hit lasts one frame
  and would fall between messages). The panel's tempo engine reports
  each beat; main accepts that only from the admin window.
- New top-level `oscOut` settings next to `artnet`, with the same MCP
  permission ('output'). `control` stays midi/osc only, so MCP's
  control validator is unchanged.
- Panel: an "OSC Output" card in Control, with target, port, rate,
  prefix, one switch per group, and a live packet counter that only
  polls while the card is on screen.

Safety: off by default, target 127.0.0.1, port 9001 (the input listens
on 9000). It refuses to send when the target is this computer's open
OSC input on the same port (loopback, own addresses, subnet broadcast),
which would feed back into mapped settings. Every field is re-validated
in main, and like Art-Net nothing is sent in automation (HW_OFF).

Found by running the real app: levels arrived at 17 Hz with the rate
set to 30, because an interval threshold dropped every meter message
that arrived a millisecond early. The rate is now kept by a schedule;
measured again in the app: 29.6 Hz, median gap 33 ms.

Checks:
- npm test: 2959 pass (17 new in tests/osc-out.test.js): the two OSC
  1.0 spec example packets byte for byte, every message decoded by the
  app's own parser (parsePacket is now exported for this), rate under
  meter jitter, hits, tempo, state, beat, prefix, settings, the loop
  guard, and two real UDP round trips. 18 of 18 mutations caught.
- Live, real app, isolated --user-data-dir, opened through MCP:
  1383 packets in 12 s, levels 29.6/s, hits, beats 1/4..4/4, the mode
  switch reported 0.01 s after it happened. The real profile was not
  touched.
- npm run smoke (isolated --user-data-dir): PASS; no untranslated text
  in the English UI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Send beat and bar phase, a downbeat edge, transport, and an NTP timetag.
Levels, analysis and spectrum leave in one bundle. Hits close to 0 on the
draw frame. Show state covers blackout, palette, brightness, layers and the
deck. IPv6 and extra targets are accepted. Both READMEs list the address tree.

npm test: 2965 pass. npm run smoke: PASS.
Emit /crossfade and the panel sliders when they change, and again
every 5 seconds. The planner also emits /show/opacity when a master
value is supplied. This branch has no master fader, so the live
context does not send one.
Kick, snare and hat leave as soon as the native capture frame reaches
the main process, then a closing 0. A real UDP packet carried
/cayadev/hit/kick. File playback still sends from the window. The
control-page screenshot shows the OSC output card.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OSC output

2 participants