Merge branch 'dev' into codex/issue-1623-workspace-camera-animation

Signed-off-by: Hendrik D. <144935017+HendrikD2005@users.noreply.github.com>

Author: HendrikD2005

docs/keiko-editor/vscode-parity-roadmap.md

@@ -145,6 +145,14 @@ Server-side, `selectVoicePersonaVoice(config, persona)` resolves the cheapest co
 provider that maps the requested persona to its `voiceId`; the resolver result carries the `voiceId` and
 therefore stays server-side. It is the seam the assistant speech-output feature (Issue #1558) consumes.
 
+> **Realtime voice ids are a narrower set than text-to-speech voice ids.** The realtime models accept
+> `alloy`, `ash`, `ballad`, `coral`, `echo`, `sage`, `shimmer`, and `verse`; some text-to-speech-only voices
+> (for example `nova` and `onyx`) are **rejected** by the realtime model and would break realtime session
+> creation. For this reason a `keiko-realtime` provider's `voiceProfiles` must map each persona to a
+> realtime-valid voice id. The realtime negotiation applies `resolveRealtimeVoice` as a defense-in-depth
+> guard: a configured voice id outside the realtime set falls back to `alloy` rather than failing the
+> session. A `keiko-tts` provider keeps the broader text-to-speech voice set.
+
 ## 3. Reading the capability: the BFF endpoint
 
 The UI reads the resolved voice capability before rendering any voice affordance:

docs/voice/capability-configuration.md

@@ -0,0 +1,89 @@
+# Voice pipeline fluidity audit and optimization
+
+This note records a focused audit of the shipped voice dialogue pipeline (Epics #491 and #1556)
+prompted by user reports that the human↔assistant dialogue did not feel fluid (latency, choppiness,
+unconvincing quality), the fixes applied, and the prioritized follow-ups.
+
+## Method
+
+The pipeline hot path was audited statically across the Model Gateway voice adapters, the BFF voice
+handlers, and the browser dialogue/transport/playback hooks. In addition, the **live** Azure Foundry
+voice deployments were exercised directly to obtain real latency numbers and to verify provider
+request/response schemas (no mocks). Audio quality (timbre/naturalness) is a property of the
+configured provider model and is out of scope for pipeline changes; perceived fluidity (start
+latency, choppiness, turn-taking, robustness) is what this work targets.
+
+## Live baseline (real Azure Foundry calls)
+
+| Path                                     | Observation                                                                                         |
+| ---------------------------------------- | --------------------------------------------------------------------------------------------------- |
+| TTS `keiko-tts`, short reply, mp3        | ttfb ~0.78s, total ~1.4s, ~72KB                                                                     |
+| TTS `keiko-tts`, short reply, opus       | ttfb ~0.90s, total ~1.1s, ~18KB                                                                     |
+| TTS `keiko-tts`, long reply, mp3         | total ~3.0s, ~303KB                                                                                 |
+| TTS `keiko-tts`, long reply, opus        | total ~1.95s, ~75KB                                                                                 |
+| STT `keiko-stt`                          | ~1.2s round-trip; rejects audio without a recognized file extension                                 |
+| Realtime `keiko-realtime` session create | HTTP 200 ~0.3s; **default session ran the provider demo persona**; voice `nova` rejected (HTTP 400) |
+
+## Fixed in this change
+
+### Realtime dialogue (Wave A) — `fix(voice): make realtime dialogue audible and grounded`
+
+- **Silent assistant (critical):** the negotiated remote audio track was never attached to an output
+  sink (`useRealtimeVoice` wired `onConnectionStateChange` but never `onRemoteTrack`). The remote
+  stream is now attached to an autoplaying audio sink.
+- **Ungrounded persona (critical):** the realtime session was created with no `instructions`, so it
+  ran the provider default demo persona ("helpful, witty, friendly AI … talk quickly … knowledge
+  cutoff 2023-10"). It now uses the same system persona as the text chat, set via the GA nested
+  `audio.{input,output}` session schema (verified against the live endpoint; the older top-level
+  shape returns HTTP 500).
+- **Voice + transcription:** a realtime-valid output voice and `input_audio_transcription` +
+  `server_vad` turn detection are now configured.
+- **Invalid-voice guard:** `resolveRealtimeVoice` maps a TTS-only voice id (e.g. `nova`, rejected by
+  the realtime model with HTTP 400) to a safe default so a misconfigured persona mapping cannot break
+  session creation.
+- **Transient ICE `disconnected`** no longer tears the session down immediately; a bounded grace
+  window allows recovery.
+- **Full-duplex capture:** `getUserMedia` now requests echo cancellation / noise suppression / AGC /
+  mono, preventing the assistant echoing into the microphone.
+- **Negotiation timeout** clamped to 8s (was the generic 30s provider timeout).
+
+### Turn-based speech latency (Wave B) — `perf(voice): request opus for interactive assistant speech`
+
+- The assistant speak path now requests **opus (audio/ogg)** instead of mp3: measured ~25–35% faster
+  end-to-end and ~4x smaller for the same utterance, lowering both the synth-to-first-audio wait and
+  the base64 inflation of the JSON envelope. opus output is a valid, browser-playable Ogg/Opus
+  container, and the MIME stays inside the existing server allowlist.
+
+## Prioritized follow-ups (identified, deliberately deferred)
+
+These were confirmed real but are deferred to keep this change contained, well-tested, and within the
+shipped subsystem's invariants. They are ordered by perceived-fluidity value.
+
+1. **Turn-based VAD end-of-turn + voice-activated barge-in.** The shipped dialogue mode (STT+TTS)
+   ends a user turn only when the user manually taps the mic again, and barge-in is button-driven.
+   A WebAudio `AnalyserNode` on the dictation `MediaStream` (trailing-silence end-of-turn + energy-
+   onset barge-in) would make turns feel natural. Deferred because it requires exposing the shared
+   dictation recorder's stream and WebAudio test infrastructure (regression surface on composer
+   dictation, #495). _Files: `useVoiceDialogueSession.ts`, `voice-dialogue-session.ts`,
+   `dictation-recorder.ts`._ Note: realtime mode already gets natural turn-taking via `server_vad`.
+2. **Streaming assistant-speech playout (MSE/Web Audio).** Start playback on the first audio chunk
+   instead of buffering the whole clip. With opus the whole-clip wait is already ~1.1s versus a
+   ~0.9s time-to-first-audio floor, so the remaining benefit is ~0.2s; it is a larger architectural
+   change to the audible path and warrants its own change with perceptual sign-off. _Files:
+   `useAssistantSpeech.ts`, `voice-handlers.ts` (reuse the existing SSE seam), `lib/api.ts`._
+3. **Control-plane WS heartbeat + client negotiate() timeout.** Ping/pong liveness and a browser-side
+   handshake timeout so a half-open connection cannot stall on "negotiating". _Files:
+   `voice-realtime.ts`, `voice-realtime-client.ts`._
+4. **Per-user persona selection for realtime.** Plumb the selected `male`/`female`/`neutral` persona
+   through the control protocol so realtime honors it (today it uses the configured neutral default).
+5. **Live interrupt offset.** Source the barge-in offset from the live media position rather than the
+   previous interrupt's stored offset. _File: `useVoiceDialogueSession.ts`._
+6. **STT interim/verbose_json.** Request `verbose_json` for real duration (the current `json` path
+   parses `confidence`/`duration` that are never returned) and surface interim transcripts. _File:
+   `speech-to-text-adapter.ts`._
+
+## Invariants preserved
+
+Model Gateway boundary intact; no new runtime media dependencies; no raw audio persisted; the browser
+never receives the provider key; spoken Keiko derives from the same persona as written Keiko; voice
+remains capability-gated and fully optional.

docs/voice/pipeline-fluidity-audit.md

@@ -1140,12 +1140,26 @@ export interface FilesRenameRequest {
   readonly path: string;
   // Root-relative POSIX destination path; its parent must exist and it must not already exist.
   readonly newPath: string;
+  // Issue 2.6: optional version-aware precondition for a FILE move — the BFF rejects with
+  // STALE_SESSION (409) if the on-disk file changed since this revision. Only a caller holding an open
+  // buffer (editor/agent) has it; the metadata-only tree omits it.
+  readonly baseVersion?: EditorDocumentVersion;
 }
 
 export interface FilesDeleteRequest {
   readonly root: string;
   // Root-relative POSIX path of the entry to delete. A directory is removed recursively.
   readonly path: string;
+  // Issue 2.6: optional version-aware precondition for a FILE delete (see FilesRenameRequest).
+  readonly baseVersion?: EditorDocumentVersion;
+}
+
+export interface FilesCopyRequest {
+  readonly root: string;
+  // Root-relative POSIX path of the existing entry to copy (a directory is copied recursively).
+  readonly sourcePath: string;
+  // Root-relative POSIX destination path; its parent must exist and it must not already exist.
+  readonly destPath: string;
 }
 
 // Shared response for create / rename / delete: the affected entry's canonical root-relative path

packages/keiko-contracts/src/bff-wire.ts

@@ -15,7 +15,12 @@
 // realized on the existing loopback HTTP + Server-Sent Events seam; a bidirectional WebSocket upgrade
 // is an explicit, ADR-gated transport decision owned by Issue #497 (ADR-0058 D3 / ADR-0059).
 
-import type { VoiceProfile, VoiceProviderLocality, VoiceUnavailableReason } from "./gateway.js";
+import type {
+  VoicePersona,
+  VoiceProfile,
+  VoiceProviderLocality,
+  VoiceUnavailableReason,
+} from "./gateway.js";
 
 // ─── Protocol version ─────────────────────────────────────────────────────────
 export const VOICE_PROTOCOL_VERSION = "1" as const;
@@ -285,6 +290,10 @@ export interface VoiceSessionCreateMessage extends VoiceControlEnvelope<"session
   readonly idempotencyKey: string;
   readonly requestedProfile: VoiceProfile;
   readonly negotiationMode: VoiceNegotiationMode;
+  // Optional product voice persona ("male" | "female" | "neutral") the client selected. Content-free
+  // (an enum, never a provider voice id); the host resolves it server-side to a realtime-valid voice so
+  // the spoken voice matches the user's choice. Absent ⇒ the host uses its configured default voice.
+  readonly persona?: VoicePersona | undefined;
 }
 
 export interface VoiceSessionCreatedMessage extends VoiceControlEnvelope<"session.created"> {

packages/keiko-contracts/src/voice-protocol.ts

@@ -157,8 +157,9 @@ vi.mock("@monaco-editor/react", () => {
       focus: vi.fn(),
       setSelection: vi.fn(),
       revealRangeInCenterIfOutsideViewport: vi.fn(),
-      deltaDecorations: vi.fn((_oldDecorations: readonly string[], newDecorations: readonly unknown[]) =>
-        newDecorations.length > 0 ? ["reference-decoration"] : [],
+      deltaDecorations: vi.fn(
+        (_oldDecorations: readonly string[], newDecorations: readonly unknown[]) =>
+          newDecorations.length > 0 ? ["reference-decoration"] : [],
       ),
       mounted: false,
       fakeEditor: null as unknown as FakeEditorShape,
@@ -456,6 +457,25 @@ describe("KeikoCodeEditor — runtime errors", () => {
     expect(onRuntimeError).toHaveBeenCalled();
     expect(screen.getByLabelText("Editor: src/a.ts")).toBeInTheDocument();
   });
+
+  it("re-applies the theme on a theme-variant change without remounting (Issue 2.2)", async () => {
+    // A light/dark toggle must re-theme the SAME live editor (preserving the undo stack + view state),
+    // not remount it. The re-apply re-attempts theme registration; in jsdom that surfaces again
+    // through onRuntimeError — the observable that the re-theme path ran on the live instance.
+    const onRuntimeError = vi.fn();
+    const { rerender } = render(
+      <KeikoCodeEditor {...baseProps({ onRuntimeError, themeVariant: "dark" })} />,
+    );
+    await flushMount();
+    expect(onRuntimeError).toHaveBeenCalled();
+    onRuntimeError.mockClear();
+
+    rerender(<KeikoCodeEditor {...baseProps({ onRuntimeError, themeVariant: "light" })} />);
+    await flushMount();
+    expect(onRuntimeError).toHaveBeenCalled();
+    // Still the same mounted editor — no remount.
+    expect(screen.getByLabelText("Editor: src/a.ts")).toBeInTheDocument();
+  });
 });
 
 describe("KeikoCodeEditor — Monaco language", () => {
@@ -529,12 +549,15 @@ describe("KeikoCodeEditor — reference reveal", () => {
     expect(captured.editor?.revealRangeInCenterIfOutsideViewport).toHaveBeenCalledWith(
       expectedRange,
     );
-    expect(captured.editor?.deltaDecorations).toHaveBeenCalledWith([], [
-      {
-        range: expectedRange,
-        options: { isWholeLine: true, className: "keiko-editor-reference-target" },
-      },
-    ]);
+    expect(captured.editor?.deltaDecorations).toHaveBeenCalledWith(
+      [],
+      [
+        {
+          range: expectedRange,
+          options: { isWholeLine: true, className: "keiko-editor-reference-target" },
+        },
+      ],
+    );
   });
 
   it("replays the same range when the host sends a new reveal request id", async () => {
@@ -547,9 +570,7 @@ describe("KeikoCodeEditor — reference reveal", () => {
     const firstRevealCount = captured.editor?.setSelection.mock.calls.length ?? 0;
 
     rerender(
-      <KeikoCodeEditor
-        {...baseProps({ revealRequest: { ...revealRequest, id: "ref-2" } })}
-      />,
+      <KeikoCodeEditor {...baseProps({ revealRequest: { ...revealRequest, id: "ref-2" } })} />,
     );
 
     expect(captured.editor?.setSelection).toHaveBeenCalledTimes(firstRevealCount + 1);

packages/keiko-editor/src/components/KeikoCodeEditor.test.tsx

@@ -220,7 +220,18 @@ export function isSaveChord(event: Pick<KeyboardEvent, "key" | "metaKey" | "ctrl
   return (event.metaKey || event.ctrlKey) && event.key.toLowerCase() === "s";
 }
 
-function registerTheme(args: WireEditorOnMountArgs): void {
+/**
+ * (Re)define the Keiko Monaco theme for `themeVariant` from the live DOM tokens and apply it to the
+ * running editor — `registerKeikoEditorTheme` calls both `defineTheme` and `setTheme`. Used at mount
+ * and again when the app theme switches (Issue 2.2), so a light/dark toggle re-themes the SAME editor
+ * instance instead of forcing a remount that would discard the undo stack and view state.
+ */
+export function reapplyEditorTheme(args: {
+  readonly monaco: MountMonaco;
+  readonly container: HTMLElement;
+  readonly themeVariant: EditorThemeVariant;
+  readonly onThemeError?: ((message: string) => void) | undefined;
+}): void {
   try {
     const tokens = resolveEditorThemeTokensFromDom(args.container);
     registerKeikoEditorTheme(args.monaco.editor, args.themeVariant, tokens);
@@ -230,6 +241,15 @@ function registerTheme(args: WireEditorOnMountArgs): void {
   }
 }
 
+function registerTheme(args: WireEditorOnMountArgs): void {
+  reapplyEditorTheme({
+    monaco: args.monaco,
+    container: args.container,
+    themeVariant: args.themeVariant,
+    onThemeError: args.onThemeError,
+  });
+}
+
 function installSaveAction(args: WireEditorOnMountArgs): monaco.IDisposable {
   return args.editor.addAction(
     buildSaveActionDescriptor({