Commit Graph

2413 Commits

Author SHA1 Message Date
nesquena-hermes 8c9d3635e2 lint: fix B904 raise-from in share session-resolve + drop unused importlib
CI ruff gate flagged 3 new violations: two api/routes.py KeyError re-raises in an
except-KeyError block (add 'from None') + an unused importlib import in the share
security test. All trivial.
2026-07-11 02:42:59 +00:00
nesquena-hermes a1312acc5d fix(share): reject dict-valued text-blocks and dict titles (re-gate edge)
The re-gate found the always-on sanitizer still had two str()-on-dict paths:
a list text-block whose 'text' is itself a dict (via /api/session/import), and
a dict-valued session title. Both are now type-checked to str; a dict title
falls back to 'Untitled'. Regression cases added. Co-authored-by: MicoRobot.
2026-07-11 02:14:07 +00:00
nesquena-hermes 75153f5b79 fix(share): always-on redaction boundary + rebase/UX fixes (gate findings)
Adversarial security re-gate on the rebased head found 4 issues; fixed:

1. [CORE] api/shares.py public snapshot could leak credentials/paths/tool
   payloads. The sanitizer relied on redact_session_data (gated on the
   user-toggleable api_redact_enabled) — disabling it exposed API keys, a dict
   assistant content was stringified as raw tool data, and session.workspace
   survived in transcript text. Now the per-message sanitizer applies ALWAYS-ON
   credential redaction (_redact_fn_cached, force=True, ignores the setting) +
   scrubs known session/workspace/home paths, and refuses to stringify non-text
   structured content. Title is force-redacted too. Regression test proves the
   boundary holds with api_redact_enabled=False.
2. [SILENT] api/models.py: my rebase wrongly kept anchor_activity_scenes in
   METADATA_FIELDS, moving 250-480KB scene bodies before messages and undoing
   master's metadata fast-path (#5854). Removed; 17 #5854 tests pass again.
3. [SILENT] static/index.html: the Share/StopSharing/ExportHTML button triplet
   was double-applied by the rebase (duplicate IDs → only the first wired).
   Deduped to one each + a count assertion test.
4. tests: fixed the stale Session.load-after-/api/session/new setup (master
   keeps new sessions memory-only until first message).

Also (UX, found in the visual flow sweep): a clipboard-copy failure after a
SUCCESSFUL share showed a red 'Share failed' toast and skipped opening the page.
The copy is now isolated — the share succeeds, the page opens, and the toast
shows the link if the clipboard write fails, never a false failure.

Co-authored-by: MicoRobot.
2026-07-11 02:06:34 +00:00
nesquena-hermes e061e8534c Rebase #5337 (public read-only conversation sharing) onto master
Rebased 363 commits forward (branch-point 2026-07-05). Applied the PR's code
diff cleanly across 16 files; resolved api/models.py additively (both master's
process_wakeup_pause/anchor_scene_index fields AND the PR's share_token/
share_created_at fields kept in the Session constructor, metadata list, and
to_dict). CI workflow (.github/workflows/tests.yml) intentionally NOT taken from
the stale branch — master owns it (the branch's apparent removal of Playwright/
Office steps was a PAT-workflow-scope revert; rebase restores master's CI).

Co-authored-by: MicoRobot. Concept approved by maintainer (opt-in public share,
2026-07-11). Note: 4 dynamic share tests fail identically on the original
unrebased head (pre-existing test-env Session.load fragility, not rebase-caused);
to be resolved during the security re-gate in a proper sandbox.
2026-07-11 01:39:42 +00:00
nesquena-hermes 0af0a5fcf0 Merge #5746: final answer only activity display mode 2026-07-10 23:50:28 +00:00
t 3188f88599 fix(#4812): break live SSE loop on a terminal event even when deduped at cutoff
Codex round-3 finding #2 [CORE]: when reconciliation replayed the active run's
terminal event at the subscribe cutoff, the live queue's copy hit the cutoff/dedup
'continue' BEFORE the terminal 'break' check — so the handler never broke, stayed
blocked on the dead run's queue, and missed subsequent session runs. Check the
terminal event type before the already-delivered continue: a terminal always ends
the loop (after emitting if not already sent). Regression test asserts the handler
unsubscribes (doesn't hang) when the terminal is deduped. 24/24 SSE tests green.

Co-authored-by: rodboev <rodboev@users.noreply.github.com>
2026-07-10 20:47:06 +00:00
t 2b22690fa4 fix(#4812): capture resume cursor + journal baseline before SSE headers commit
Codex round-3 finding #1 [SILENT]: baselining after end_sse_headers() left a window
where a run completing between header commit and baseline was absorbed into the
baseline + silently lost. Move resume-id parse + fingerprint baseline before
send_response/headers (both side-effect-free: header read + stat-only fingerprint).

Codex round-3 also flagged finding #2 [CORE] (reconciliation terminal-event-at-cutoff
dedup keeps the handler attached to a dead run queue) — NOT addressed here; escalating
to Nathan per Codex-depth diminishing-returns (3rd round into the same SSE seam class).

Co-authored-by: rodboev <rodboev@users.noreply.github.com>
2026-07-10 20:44:10 +00:00
t 3c642e08f1 fix(#4812): hoist idle SSE fingerprint baseline to earliest point (before preliminary lookup + initial replay)
Codex re-gate found the baseline was still after the preliminary
_active_run_stream_for_session() lookup + the initial cursor read_session_run_events()
replay — a run completing during either window folded into the baseline and was
silently missed. Move the fingerprint baseline to immediately after resume_event_id
parsing, before both operations, so no pre-baseline completion window remains. Remove
the now-redundant later baseline. 23/23 SSE tests green.

Co-authored-by: rodboev <rodboev@users.noreply.github.com>
2026-07-10 20:36:22 +00:00
t 3ac19a5518 fix(#4812): capture idle SSE fingerprint baseline BEFORE first attach (TOCTOU)
Final-gate Codex caught a TOCTOU race in the prior idle-seam fix: the fingerprint
baseline was captured AFTER the first attach_active_stream() lookup, so a run
completing DURING that first attach folded into the baseline and was silently
missed (only : keepalive emitted, no snapshot). Move the baseline capture before
the first attach so any completion from that point on is detected. Add an ordering
regression test (baseline fp precedes the idle-loop attach + advance still emits
one snapshot). 23/23 SSE tests green.

Co-authored-by: rodboev <rodboev@users.noreply.github.com>
2026-07-10 20:24:47 +00:00
t 2829843a2a fix(#4812): re-sync idle no-cursor SSE subscriber on run completing during keepalive wait
Fable gate found a seam Codex's 3 boundary findings didn't cover: an idle no-cursor
subscriber to /api/sessions/{id}/events misses a run that starts AND finishes inside a
single keepalive tick — the wait loop only looked for a live in-memory STREAMS entry to
attach, never noticing the journal advanced, so the run's transcript was silently missing
until manual refresh.

Add a cheap bounded session_journal_fingerprint() (file-count/max-mtime/total-size from
stat only, never parses bodies) and, inside the idle attach-wait loop, emit a session_snapshot
re-sync whenever the fingerprint advances with no live stream to attach — the same honest
recovery contract used for a failed reconciliation. Re-baselines after each re-sync so a quiet
idle connection never spams snapshots. 2 regression tests; 22/22 SSE tests green.

Co-authored-by: rodboev <rodboev@users.noreply.github.com>
2026-07-10 20:15:49 +00:00
t 9600e1546a Merge #5677: resumable session SSE stream (#4812)
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
2026-07-10 20:12:27 +00:00
t d967131056 fix(#3929 ux): gate wakeup-pause disclosure on actual pause + cover both error paths
Codex re-gate caught 2 defects in the prior UX commit:
1. streaming.py:8921 — the disclosure fired for ANY process_wakeup failure (e.g.
   rate-limit) even when no pause was recorded -> falsely claimed 'retries paused'
   while process_wakeup_pause stayed empty. Now gated on the record_* return value.
2. streaming.py:10130 — the raised-exception credential_pool_empty path (exercised
   by the existing agent test) recorded the pause but omitted the disclosure. Added
   the same conditional disclosure there.
Both paths now keep _error_payload['hint'] in sync with the persisted bubble, and
the disclosure appears only when a pause was actually recorded. 69 tests green.
2026-07-10 19:28:06 +00:00
t c26bc21194 ux(#3929): disclose wakeup pause in error card + changelog
Fable UX-gate follow-up on #5732: after the first credential-exhausted error,
automatic wakeups are silently suppressed. Disclose it in the wakeup-source
error card ('Automatic retries for this conversation are paused until you send
a message, switch the model/provider, or fix the credentials.') so the silence
reads as intentional, not a stuck agent — Gmail/Dropbox account-needs-attention
convention. Scoped to the process_wakeup + credential_pool_empty seam only.

Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
2026-07-10 19:16:48 +00:00
Frank Song 3c95b3a9d5 Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-11 02:07:37 +08:00
Rod Boev 4cbc70c778 fix(#4812): satisfy replay boundary lint gate 2026-07-10 10:51:34 -04:00
Rod Boev 956c276362 fix(#4812): close session SSE replay recovery gaps 2026-07-10 10:46:47 -04:00
Frank Song 4c2400261a Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 21:21:47 +08:00
igorroncevic dea5e8a96d fix: fail closed on incomplete OpenRouter pricing 2026-07-10 15:18:39 +02:00
nesquena-hermes d7c3c2b734 Release: repair stale session provider binding at chat start (#5873, #5731) (#5881)
* fix(#5731): repair poisoned session provider at chat start

* fix(#5731): fail safe on incomplete catalog ownership

* changelog: session provider-repair on stale binding (#5873, #5731)

---------

Co-authored-by: Rod Boev <rod.boev@gmail.com>
Co-authored-by: t <a@b>
2026-07-10 06:11:34 -07:00
Rod Boev da885659bd fix(streaming): satisfy session SSE lint gate 2026-07-10 07:37:16 -04:00
Rod Boev b2a9ca28ad fix(streaming): harden session SSE replay (#4812) 2026-07-10 07:31:00 -04:00
Frank Song c641a9d37c Keep streaming terminal events after no-pause success 2026-07-10 14:43:41 +08:00
Frank Song a61586bd7f Preserve latest session on wakeup pause clear 2026-07-10 13:20:40 +08:00
Frank Song 92ea69c7c3 Require fingerprint evidence for wakeup credential recovery 2026-07-10 12:23:02 +08:00
Frank Song 8eabf63049 Lock streaming wakeup pause success clear 2026-07-10 12:01:24 +08:00
Frank Song e604b76720 Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 11:39:50 +08:00
Frank Song 920b5e8864 Preserve wakeup pause on cancelled stale credential failure 2026-07-10 11:39:31 +08:00
nesquena-hermes de0e6eb452 Release: centralize auxiliary-model task catalog (#5865) (#5870)
Auxiliary-model task definitions were maintained in BOTH Python and JS and could
drift (the retired session_search task lingered in one). Centralize to a single
backend catalog that panels.js renders+saves from exclusively; purge the retired
slot on reset while preserving other custom bindings; old configs still load.

Gate-pass: Codex SAFE (only session_search removed, 11 other tasks + defaults
unchanged, 22 keys match across 15 locales, legacy migrate preserves custom
bindings, executable DOM harness rendered all rows). 53 own + 887 regression green.

Co-authored-by: t <a@b>
Co-authored-by: starship-s <starship-s@users.noreply.github.com>
2026-07-09 20:31:54 -07:00
Frank Song bd083c8b54 Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 11:19:07 +08:00
nesquena-hermes 9c1e77de48 Release: fix Experimental channel version display on stable-tagged installs (#5864, #5862) (#5868)
* fix(updates): scope installed-version fallback to webui + verified current_sha (#5864 gate fixes)

Gate (Codex) found 2 CORE in the #5862/#5864 experimental-version-display fix:
1. api/updates.py:947 — the installed-version fallback ran in shared _check_repo_release
   for BOTH repos, injecting WEBUI_VERSION (v0.52.0, not an Agent ref) into Agent payloads.
   Now scoped to name=='webui'.
2. api/updates.py:996 — current_sha reused current_version_display, but WEBUI_VERSION can be
   vX.Y.Z-dirty-<hash>/-N-g<sha>/bare-SHA/unknown (not git refs) -> broken /compare link +
   lost update-summary subjects. current_sha now derives from a git-VERIFIED exact-match tag
   (channel-neutral) or None; current_version keeps the real display string.

+2 regression tests (agent-repo-no-inject, dirty-HEAD-verified-sha). 21/21 update-channel green.

* changelog: #5864 experimental-channel version display fix (#5862)

---------

Co-authored-by: t <a@b>
2026-07-09 20:18:03 -07:00
Frank Song fc262a4aec Fix named-profile wakeup credential recovery 2026-07-10 11:14:08 +08:00
Frank Song 63dda7fe2d Merge remote-tracking branch 'origin/master' into paperclip/pr-5732
# Conflicts:
#	api/models.py
2026-07-10 10:15:21 +08:00
nesquena-hermes e4342a65cf Release: split anchor_activity_scenes after messages to fix #4633 server memory growth (#5858, #5854) (#5863)
Reasoning-heavy sessions accumulate 250-480KB of anchor_activity_scenes serialized
BEFORE the messages array, so the 64KB metadata-prefix read always overflowed ->
full-parse fallback on every read -> memory growth. Move scene data AFTER messages;
metadata prefix now ~2KB, modern reads skip the full parse. Old-layout files still
load (one authoritative parse then cached); round-trip preserves scenes exactly.

Independent review: nesquena APPROVED (end-to-end, on head). Gate-pass: Codex SAFE
(933KB legacy sidecar loads clean + round-trip verified), 17 own + 2945 regression green.

Co-authored-by: t <a@b>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
2026-07-09 19:05:54 -07:00
Frank Song d7e7fb3013 Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 07:08:53 +08:00
Frank Song 67b001e57c Fix wakeup pause recovery races 2026-07-10 07:08:36 +08:00
nesquena-hermes e0afdcb546 Release: write Office upload sidecars (#5734, #540) (#5859)
On .docx/.xlsx/.pptx upload, write a best-effort <name>.md sidecar with extracted
text (reuses api/office_documents.py — no new dep). Path-contained (O_NOFOLLOW
openat-walk within workspace), collision-safe (O_CREAT|O_EXCL never overwrites an
existing .md), best-effort (sidecar failure leaves the Office file intact, partial
cleanup on failure). Nathan concept-approved.

Gate-pass: Codex SAFE (containment + exclusivity + best-effort all verified), 27 own
+ 1443 upload/workspace regression tests green.

Co-authored-by: t <a@b>
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
2026-07-09 15:57:21 -07:00
Frank Song 2ce9801088 Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 06:36:26 +08:00
nesquena-hermes 34342b9f4d fix(security): trusted-proxy-verified local-origin gate (fixes #5764 terminal-gate bypass) (#5857)
* fix(security): trusted-proxy-verified local-origin gate (maintainer fix of #5764)

Rebuild _onboarding_request_is_local with a single, symmetric trust model that
closes the embedded-terminal local-gate bypass AND fixes the direct-LAN lockout
regression from the contributor's #5764 attempt (bounced with these findings):

1. SPOOF CLOSED: a forwarded client IP is honored ONLY when the un-spoofable raw
   socket peer is a trusted proxy (loopback, or HERMES_WEBUI_TRUSTED_PROXY_CIDRS).
   A public direct client sending X-Forwarded-For: 127.0.0.1 is judged on its raw
   public peer -> DENY. New _raw_peer_is_trusted_proxy + _forwarded_client_ip_from
   _trusted_proxy consume the FULL XFF chain (get_all, repeated headers), walk it
   right-to-left skipping trusted hops, and fail closed on empty/blank/garbage.
2. NO LAN LOCKOUT: a direct loopback/LAN client with no proxy header stays local
   (onboarding, first-password/passkey, passwordless terminal keep working).
3. COMPAT: TRUST_FORWARDED_FOR=1 stays the opt-in that consults the chain; the new
   TRUSTED_PROXY_CIDRS only WIDENS trust to a non-loopback proxy (loopback always
   trusted). Shipped private-peer-with-forwarded-header deny nuance preserved.

Updated the one existing test that encoded the vulnerable behavior; added a
21-case adversarial truth-table regression (tests/test_cvd3_terminal_local_origin_gate.py).
66 gate tests + 720 auth/terminal regression pass.

Co-authored-by: Hinotoi-agent <Hinotoi-agent@users.noreply.github.com>

* fix(security): family-aware trusted-proxy membership (IPv4-mapped-IPv6)

Codex re-gate CORE: a mapped-IPv6 address (::ffff:10.9.9.9) never matched an
IPv4 CIDR allowlist via 'addr in net' -> (1) legit proxy treated as untrusted =
lockout; (2) mapped trusted HOP in the XFF chain mis-returned as client -> a
preceding public client admitted to the terminal gate. New _ip_in_networks
helper checks addr AND addr.ipv4_mapped; both the peer check and the chain-walk
_is_trusted_hop route through it. +3 mapped-IPv6 regression cases (incl the
public-client-hidden-behind-mapped-trusted-hop security case).

---------

Co-authored-by: t <a@b>
Co-authored-by: Hinotoi-agent <Hinotoi-agent@users.noreply.github.com>
2026-07-09 15:33:16 -07:00
Frank Song 4bbce7bc45 Persist restored wakeup pause after streaming cancel 2026-07-10 05:43:44 +08:00
Frank Song d9e45f1d07 Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 05:27:46 +08:00
nesquena-hermes a0067b0b24 Release: classify external_agent imports as CLI to match client render (#5846, #5831) (#5855)
Fixes the #5831 brick: "WebUI sessions (N)" chip showed a count but an empty list.
Server is_cli_session_row classified read-only external_agent imports (Claude Code,
Codex) with a real title as non-CLI (counted under webui_session_count), while the
client _isCliSession filed them in the CLI bucket -> mismatch. Server now classifies
external_agent/external-agent as CLI, matching the client. WebUI/messaging/subagent
unaffected (their exclusions precede the new branch); external_agent imports stay read-only.

Self-built (nesquena-hermes); independent nesquena review APPROVED on this head (17:28);
gate-pass (Codex SAFE, no over-capture, caller audit clean), 14 own + 180 regression green.

Co-authored-by: t <a@b>
Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com>
2026-07-09 14:20:20 -07:00
nesquena-hermes f449ff21c6 Release: hard-prune post-compression tool context (#5667) (#5852)
WebUI hard token-cap on protected-tail tool-result payloads in the MODEL CONTEXT
only, applied once after a confirmed compression event, to prevent post-compression
context blowup. Deep-copy content-replace only (no message removal/reorder → no
orphaned tool_result); visible transcript (s.messages) untouched.

Deep-review re-gate: Codex SAFE (pairing-safety + context-only + idempotent verified
by reproduction), 65 own + 992 regression tests green. Crown-jewel streaming path.

Co-authored-by: t <a@b>
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
2026-07-09 13:41:31 -07:00
Frank Song 4efab593fe Preserve wakeup pause on cancelled gateway recovery 2026-07-10 04:03:40 +08:00
Frank Song 4af5334ebb Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 03:33:07 +08:00
nesquena-hermes 2b457f1646 Release: cap StreamChannel offline replay buffer (#5788) (#5849)
Bounded deque(maxlen) drop-oldest so a disconnect-without-cancel can't grow the
per-turn offline replay buffer unbounded (per-abandoned-turn OOM risk, #4633 family).
Dropped frames recovered via run-journal last_event_id replay path.

Deep-review gate: Codex SAFE (journal-recovery verified end-to-end), 20 own + 1106
regression tests green. Crown-jewel streaming path.

Co-authored-by: t <a@b>
Co-authored-by: ai-ag2026 <ai-ag2026@users.noreply.github.com>
2026-07-09 12:26:05 -07:00
Frank Song 2547ec98d1 Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 02:02:04 +08:00
nesquena-hermes d6e5dc302d Release Batch D: terminal resource-leak fix + fsync atomic writers (#5835, #5843) (#5848)
Phase 2 backend reliability/data-integrity batch. Each byte-verified vs PR head,
Codex SAFE, 20 own + 355 regression tests pass, disjoint files.

- #5835 (ai-ag2026): stop leaking fd + reader thread + shell on client/shell exit (#4633 family)
- #5843 (ai-ag2026): fsync atomic JSON writers (oauth/passkeys/discoverability) + thread-safe temp

Co-authored-by: t <a@b>
Co-authored-by: ai-ag2026 <ai-ag2026@users.noreply.github.com>
2026-07-09 10:54:06 -07:00
Frank Song fe656a48ce Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 01:42:11 +08:00
nesquena-hermes 7a1a122e6e Release Batch C: hidden-tab dashboard poll + SSE write-deadline + rate-limit map bound (#5840, #5841, #5842) (#5847)
Phase 2 low-risk backend/perf batch (ai-ag2026 reliability cluster). Each byte-verified
vs PR head, Codex SAFE, 33 own + 720 regression tests pass, disjoint routes.py regions.

- #5840: skip dashboard status poll while tab hidden (perf; forced polls unaffected)
- #5841: arm write deadline on session-events invalidation SSE (stuck-client teardown)
- #5842: bound client rate-limit maps by sweeping stale keys (long-uptime memory)

Co-authored-by: t <a@b>
Co-authored-by: ai-ag2026 <ai-ag2026@users.noreply.github.com>
2026-07-09 10:38:47 -07:00
Frank Song a9ba8d8368 Merge remote-tracking branch 'origin/master' into paperclip/pr-5732 2026-07-10 01:07:49 +08:00