Gate hardening for #5198: the new stream-owner map leaked entries on three
paths that bypass the teardown finally — worker early-return when the stream
was cancelled before startup (streaming.py, gateway_chat.py) and the direct
ACTIVE_RUNS stale-zombie prune (routes.py). Each now calls unregister_stream_owner.
Adds a regression test proving the early-return path no longer leaks.
Co-authored-by: starship-s <starship-s@users.noreply.github.com>
Mirror the #5128 fast-path suffix repair when profile context is present
but requested_provider is omitted, which async delegation completion can
trigger via process_wakeup.
Co-authored-by: b3nw <b3nw@users.noreply.github.com>
The combined credential redactor (~15 regex passes per string) runs on the
full message history of every /api/session response. Under concurrent
dashboard polling of unchanged sessions this became the dominant CPU cost
and, behind the GIL in the single-process server, serialized requests to
10-44s each — surfacing as repeated 'connection lost' in the browser as
the proxy/browser timed out and the frontend retried, piling on threads.
The redactor is pure and deterministic (force=True, fixed masking), so
identical strings always map to identical output and are safe to memoize
without invalidation. Wrap it in an lru_cache(maxsize=4096), gated to
strings <=16KB so a few giant tool-output dumps can't evict the many small
recurring strings or balloon RSS. Repeated polls become O(1) cache hits
(measured 3.7ms -> 0.002ms per 3KB marker string).
_redact_fn_cached is preserved as the call-site name so the existing
monkeypatch-based redaction tests bypass the cache unchanged (69/69 pass).
Windows locks session JSON files briefly during antivirus scans or
browser polling, causing os.replace() to raise WinError 5
(ERROR_ACCESS_DENIED). Add _safe_replace() helper that retries with
exponential backoff (50ms to 800ms, max 5 attempts) on PermissionError.
No-op on non-Windows platforms.
Replaces all 5 os.replace() call sites in models.py:
- Session.save() (main file + backup)
- _write_session_index() (full rebuild + fast path)
- prune_session_from_index()
A turn started SERVER-SIDE (self-wake, cron, restart hook) fans a
`server_turn_started` frame onto the per-session live-view SSE channel so an
open tab renders it live. But while a tab is HIDDEN the WebUI deliberately does
NOT hold that persistent SSE open (connection-pool budget — #3992 / #4151), so
`get_session_channel()` returns None at fan-out time and the turn is dropped for
that tab. The user only saw the turn after a manual interaction (refresh / send)
re-subscribed the channel and replayed it.
Bridge the gap with a lightweight poll of `/api/session/status` (one short GET
per ~6s tick — NOT a held connection, so the connection-pool budget the hidden
guard protects is preserved) that attaches the existing live renderer when it
sees a *live* active stream.
Backend (`api/session_ops.py`):
- `session_status()` now exposes `active_stream_id`, derived through a new
`_live_active_stream_id()` helper that only returns the id when the stream is
genuinely live (present in STREAMS / ACTIVE_RUNS). A stale id left by a
crashed/restarted run surfaces as None, so the poller never attaches a
renderer to a dead stream. Additive field — existing consumers ignore it.
Frontend (`static/messages.js`):
- `_startHiddenActiveStreamPoll` / `_stopHiddenActiveStreamPoll` /
`_attachServerInitiatedStream` implement the poll lifecycle. The poll fires
one immediate tick on hide (so an in-flight turn is caught without waiting a
full interval), attaches mid-flight turns via the reconnecting/replay path,
and stops on re-show (the real SSE takes over), session switch, or once a
stream is rendering.
- Started on BOTH hidden paths: the `visibilitychange` hook's hidden branch (a
visible tab going to background) and `startSessionStream`'s hidden
early-return skip (a session opened while already hidden). The visible path
and `stopSessionStream` clear it.
Tests:
- `tests/test_hidden_tab_server_initiated_turn.py` — backend live-validation
(stale id → None; STREAMS/ACTIVE_RUNS id returned) and frontend lifecycle /
both-hidden-paths / replay-attach source locks.
- Widened the brittle fixed-width source-window slices in
`tests/test_issue3996_sse_visibility.py` (1600/1700 → 2400) so the existing
`visibilitychange` / hidden-skip assertions still find their markers after the
poll-start lines were inserted into `startSessionStream`.
Codex flagged that get_last_workspace() falls back to the GLOBAL last_workspace.txt
before the profile config.yaml — so a named profile without its own
last_workspace.txt would leak the global path, re-introducing the #5169 bug.
Add get_profile_default_workspace() (profile last_workspace.txt -> config.yaml ->
terminal.cwd -> default, NO global fallback) and call it from /api/profile/active.
Adds a regression test: global last_workspace.txt present + named profile with
only config.yaml workspace must return the profile path, never the global.
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Expose a profile-scoped default_workspace on GET /api/profile/active (resolved
by the profile-aware get_last_workspace(), fail-open) and hydrate boot.js from
it, overriding the global GET /api/settings value. GET /api/settings stays
global-only so the settings UI can't clobber the shared default. Combines
claw-io's #5173 (primary) with the fail-open backend hardening + regression test
from #5171.
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
Codex deep-gate caught a CORE regression in the original top-N cap: capping
_apply_sidebar_state_db_overrides also capped the SOURCE classification that
/api/sessions filters on BEFORE the lazy lineage correction, so a stale-CLI JSON
row whose state.db source is webui could be silently filtered out of a WebUI-only
sidebar once it fell beyond the cap. Split the read: the indexed sessions-table
source/title lookup runs for ALL rows; only the expensive messages COUNT/MAX
aggregation (the actual 5-18s bottleneck) is capped to top-N. Added a 500-row
end-to-end regression with a real state.db proving the beyond-cap stale-CLI row
stays webui.
On power-user instances with thousands of sessions, GET /api/sessions
blocked 5-18s in the all_sessions.state_db_overrides stage:
_apply_sidebar_state_db_overrides read state.db for every row (2400+ ids)
on every concurrent poll. With the gateway-watcher reading and the agent
writing the same state.db, this became lock/IO contention that piled up
concurrent polls and flapped the UI to 'Connection lost'.
Cap the override lookup to the top-N most-recent (paint-priority) rows
that are actually visible, mirroring the existing lineage-enrichment cap
(#4638). The caller passes an already pinned-first/newest-first sorted
list, so the top-N covers the visible window; rows beyond the cap keep
their JSON source/title and are corrected lazily on history-panel open.
Cap defaults to 300, env-configurable via
HERMES_WEBUI_STATE_DB_OVERRIDE_TOP_N (0 disables). Fails open.
Reported by @latipun, reproduced by @b3nw.
Co-authored-by: latipun <latipun@users.noreply.github.com>
When session model_provider differs from profile model.provider, do not
expose profile_default_model for suffix repair. Fast path also requires
profile_provider to match requested_provider when profile_provider is set.
Addresses cross-provider mis-resolution (custom:other-proxy + bare suffix).
Co-authored-by: b3nw <b3nw@users.noreply.github.com>
Rod Boev