get_state_db_session_messages() returned archived (active=0) rows that in-place
compaction had marked inactive. WebUI reconciliation feeds this reader straight
into the next model-facing context (reconciled_state_db_messages_for_session,
prefer_context=True), so those archived rows got pulled back in — resurrecting
pre-compaction history and making every later turn re-trigger compression. Now,
WHEN the messages table exposes an `active` column, inactive rows are excluded by
default (`AND (active IS NULL OR active != 0)` — NULL/legacy rows preserved), with
an `include_inactive=True` escape hatch for explicit recovery/audit callers.
Schemas without the column are unaffected (guarded by `'active' in available`).
Contributor stage; ungated on arrival. Co-authored-by: ai-ag2026 <ai-ag2026@users.noreply.github.com>
_webui_sidecar_lineage_messages_for_display stitches a session's pre-compression
snapshot parents for full-transcript display. A cleared fork child could resurrect
its ORIGINAL (non-fork) parent's transcript when that parent was later compressed.
Now, when the root session is itself a fork (session_source=='fork'), the stitch
walk allows fork-sourced snapshot parents but stops at the first non-fork
ancestor — so a compressed fork continuation still shows its own pre-compression
turns, while an ordinary fork child stays isolated from the original parent.
Additive guard (one extra break condition); non-fork lineage display unchanged.
Contributor stage; gate-pass. Co-authored-by: rodboev <rodboev@users.noreply.github.com>
OpenCode Go's /v1/models endpoint returns models from the full public catalog that
are NOT enabled on the Go tier; selecting any such probe-only model 404s
('model not found') when the chat request is sent. get_available_models now skips
the live probe for opencode-go (bare branch) so the next `if not raw_models` falls
through to the curated static _PROVIDER_MODELS['opencode-go'] list — the actual
Go-tier models. Narrow, provider-scoped; no other provider affected.
Contributor stage; gate-pass. Co-authored-by: webtecnica <webtecnica@users.noreply.github.com>
The initial fix added ~16 lines to server.py, tripping test_server_py_under_750_lines
(server.py must stay a thin dispatcher; logic belongs in api/). Moved the whole
preflight-response into apply_cors_preflight_headers(handler) in api/routes.py;
server.py's do_OPTIONS is now a 4-line dispatcher. server.py 764 -> 746 lines.
Same security behavior (same-origin/allowlist echo, Vary: Origin, no wildcard,
header-less denial); test rewritten to the header-emitting API + added Vary and
no-header-on-denial assertions and a routes.py wildcard regression guard.
do_OPTIONS answered every CORS preflight with Access-Control-Allow-Origin: *,
advertising broader cross-origin access than a real request is granted — on a
password-less deployment that lets any site read authenticated responses. It now
echoes the request Origin only when _check_same_origin_browser_request approves
it (the exact same-origin / HERMES_WEBUI_ALLOWED_ORIGINS policy the CSRF gate
enforces for real requests), adds Vary: Origin, and never emits *. A disallowed
origin gets a 200 with no CORS headers (browser treats as preflight denial).
Default deployments (no allowlist) are unaffected.
Contributor stage; gate-pass. Co-authored-by: mo7al876any <mo7al876any@users.noreply.github.com>
Codex gate CORE: the exact-empty sentinel only suppressed restore for a bit-empty
cleared sidecar. If the user cleared, then sent ONE post-clear message, and the
stale pre-clear .json.bak still existed (unlink failed), recovery saw
bak_count>live_count and RESTORED the pre-clear transcript on top of the new
message. Added _live_supersedes_backup_by_clear_generation: suppress restore when
the live sidecar carries a clear_generation the backup lacks AND has a non-empty
transcript AND still shows the clear boundary reset (watermark==boundary==0.0).
Scoped to non-empty live so empty clear-shaped sidecars keep their existing
recovery semantics (active/pending/partial variants still restore). Fails open on
unreadable/partial reads. +2 regression tests (post-clear-message not restored;
post-clear-after-real-compaction still restores).
Closes the crash-window resurrection corner in the #5532 clear-conversation
data-loss fix. /api/session/clear writes a .json.bak before truncating, and
because the cleared sidecar's truncation_watermark is 0.0, startup recovery saw
bak_count > live_count and could restore the backup — resurrecting the cleared
transcript after a crash/restart in the clear window.
Fix: stamp a unique per-clear `clear_generation` UUID on the sidecar when
clearing a session that had messages (persisted via Session model). Recovery's
_session_records_clear_sentinel() returns no_action when the live sidecar carries
a clear_generation the backup doesn't share AND the live sidecar matches the
exact cleared-empty shape (messages/context empty, watermark/boundary 0.0,
pending fields cleared). Same-generation backups stay recoverable; unreadable or
partial matches FAIL OPEN (normal recovery), so a real crash-loss is never
suppressed.
Contributor stage; gate-pass. Co-authored-by: rodboev <rodboev@users.noreply.github.com>
The streaming settlement path could show a misleading "No response from provider"
error card AFTER a completed answer, because a stale terminal/partial result flag
still forced the generic no_response error path even when the merged current turn
already had a final assistant answer. Now the generic no_response terminal error
is suppressed ONLY when the already-merged current turn has a completed assistant
reply (scoped to soft `partial` results per the issue-thread recommendation),
letting normal `done` settlement continue. Real silent failures, partial
failures, auth/quota errors, cancellation, replayed rows, and terminal
tool/compression cases all keep their existing error behavior.
Self-built stage of contributor PR; gate-pass. Co-authored-by: rodboev <rodboev@users.noreply.github.com>
'All 0 credential(s) exhausted for <provider>' did not match _is_quota_error_text
so it fell through to a generic 'Error' with no hint (the exception path's else
branch discarded the hint entirely). Adds a distinct credential_pool_empty
classification (ordered before the quota check) + an explicit exception-path
branch + an actionable pool-specific hint, so the user sees why the turn failed
and what to do. Does not over-match (doesn't swallow real quota errors) and
doesn't block partial-harvest; downstream tolerates the new type.
NOTE: this is the CLASSIFICATION half of #3929 only — the data-loss half
(reasoning/partial-work restore on reload-reconcile) is a scoped follow-up that
needs the reporter's server logs.
Self-built (nesquena-hermes); nesquena independent review APPROVED.
An expired-auth 401 on the login page fed the login redirect its own address:
each of the three redirect guards (_safeNextPath in login.js,
_safe_login_redirect_path in routes.py, the api() 401 redirect in workspace.js,
_safe_login_inner_next in auth.py) validated against open-redirect but none
rejected a `next` pointing back at the login page, so every auth bounce wrapped +
re-encoded the previous login URL one level deeper — growing the URL exponentially
until the browser's length limit broke the tab. All guards now collapse/reject a
`next` whose decoded leading PATH resolves to the login route (bounded
percent-decoding up to 8 levels, fail-closed at the cap), while preserving the
existing open-redirect protections AND legitimate non-login paths that merely
carry their own `next=` query key. Length cap as belt-and-suspenders.
Self-built (nesquena-hermes); nesquena independent review APPROVED. Gate (Codex)
found + fixed an over-broad nested-next reject (regressed next-carrying paths) and
a deep-encode decode off-by-one; both root-fixed, login guards verified clean.
profile_env_for_background_worker mirrored the profile home into process-global
os.environ["HERMES_HOME"] and yielded the worker body OUTSIDE the setup lock, so a
concurrent cross-profile worker (e.g. background title generation on another
profile) could clobber os.environ mid-body — the agent config reader then resolved
the WRONG profile's config (intermittent turn-init failures citing an unused
provider; b3nw's v0.51.849 repro). Now installs hermes-agent v0.18.0's
context-local home override (set_hermes_home_override, a ContextVar that
get_hermes_home() consults before os.environ) for the resolved non-default profile
home — set before yield, reset in finally — so a config read in the worker body
resolves THIS profile's home from task-local state, immune to a concurrent
os.environ clobber. No worker serialization, no os.environ mutation, no detection
heuristic (both earlier WebUI-only mitigations were rejected as regression-unsafe).
Graceful degradation: the override module is resolved lazily+optionally
(_resolve_hermes_home_override, mirroring _resolve_secret_scope_module). On agents
< v0.18.0 the symbol is absent -> returns None -> pre-existing os.environ-mirror
behavior unchanged (no regression, no new failure mode).
Self-built (nesquena-hermes); nesquena independent review APPROVED.
Codex round-3 gate finding: _dedupe_replayed_context_messages deep-copies the
stale-user repaired boundary row (streaming.py:4501) into the context array. When
the mint ran AFTER dedupe, that deep-copied context row was no longer the shared
result dict, so it stayed id-less while the display copy got the minted id —
re-opening the cross-array id divergence for the stale-repair path. Fix: move
_assign_stable_message_ids() to run immediately after _restore_reasoning_metadata
and BEFORE _dedupe_replayed_context_messages at all four commit sites (streaming
main / in-band self-heal / except-path self-heal + routes runs/MoA), so the shared
result rows are stamped before any deep-copy — both arrays inherit the id.
Codex/Opus gate finding: in eager session-save mode the current user turn is
checkpointed into s.messages before the agent runs (no id yet); after
_assign_stable_message_ids stamps the returned result row, the display merge
keeps the durable eager checkpoint and skips the stamped result — leaving the
display row id-less while its context twin carries the minted id, silently
defeating id-based fork/truncate alignment for eager-mode users. The merge now
copies the minted id onto the kept checkpoint when it lacks one (guarded so it
can't overwrite or duplicate). Opus verified cross-array id pairing holds across
the real 3-turn transform pipeline.
WebUI keeps two parallel arrays per session: messages (display transcript) and
context_messages (what's sent to the model). In large/compacted sessions they
diverge. Forking / in-place truncation copies a prefix of both and relies on
truncate_context_for_display_keep() to translate a display keep-index into the
matching context index — an aligner that prefers a stable per-row `id`. But the
model-context rows carried neither id nor timestamp (a 989-session live scan
found 0 with any id), so alignment fell back to fragile content-signature
matching that goes ambiguous on repeated tool calls / empty assistant turns.
This closes the DATA gap (the alignment logic itself already prefers `id` and
handles the compacted case via the matcher, shipped in #5563): mint a monotonic,
session-unique integer `id` on the per-turn result rows AFTER the context restore
and BEFORE both arrays are built, so the display and model-context copies of a
logical row share the same id.
- api/streaming.py: new _assign_stable_message_ids(); _restore_reasoning_metadata
carries `id` forward across turns (as it already does timestamp); wired into all
three streaming commit sites (main turn, retry, self-heal).
- api/routes.py: same mint on the runs/MoA _handle_chat_sync commit path.
- api/gateway_chat.py: mints ids on the two new rows of a gateway turn.
- The `id` is stripped before the provider API call (not in _API_SAFE_MSG_KEYS),
same treatment as timestamp — nothing new reaches the provider.
session_ops.py alignment was already delivered by #5563 (id-preferring matcher +
compacted-case fall-through), so no change is needed there now; tests updated to
assert the post-#5563 behavior (id-bearing = exact cut; id-less = errs toward
under-keeping, never the old raw-index mis-cut).
Co-authored-by: b3nw <b3nw@users.noreply.github.com>
Move the 1-5s commit_session_memory() extraction off the POST /api/session/new
request thread (was blocking + New Chat) into a fire-and-forget daemon thread,
guarded by the existing per-session in-flight serialization.
Data-loss prevention on managed stop:
- server.py installs a SIGTERM handler that requests an orderly stop so
serve_forever()'s finally block runs drain_all_on_shutdown() (the default
SIGTERM disposition terminated without unwinding, losing in-flight commits).
- session_lifecycle: bounded self-unregistering background-commit registry;
a _draining flag so no NEW worker starts once the drain begins (the inline
generation-drain commits any late arrival instead); a 30s overall drain
deadline so a stuck worker cannot hang the stop path forever.
Co-authored-by: luperrypf <luperrypf@users.noreply.github.com>
Fork ("Fork from here") on a large/compacted session could cut context off a
real turn boundary — slicing mid-turn or leaving a dangling tool call, feeding
a malformed context to the model. Align the fork cut to a resolved turn boundary
via a signature matcher handling both compacted and non-compacted display
coordinate spaces; errs toward under-keeping with send-time sanitization backstop.
Co-authored-by: b3nw <b3nw@users.noreply.github.com>
Rebuilt the combined #5556+#5553 fix on v0.51.859 (now has the flake fix + the
#5542 RFC de-brittle, so no anchor-test collateral). Nathan's call: rodboev's
#5556 primary + fold self-built #5553 improvements, credit both.
Clear handler now:
- routes through shared truncate_session_at_keep(s,0) (single source of truth,
sets watermark=_truncation_watermark_for([])==0.0, the #2914 sentinel that
blocks state.db append-merge replay)
- detaches compression lineage ONLY when the parent is a pre_compression_snapshot
(Codex-caught: preserve genuine fork parent links for nesting + "Forked from")
- rodboev's persisted-clear read-back verification + stale .bak removal
RFC conflict resolved in favor of the de-brittled (symbol-anchor) master version.
Ships both PRs' test files (state_db_replay + clear_truncation_watermark incl the
fork-preservation regression). Opus (on the amended tree): both resurrection paths
closed, detach lineage-safe, verification sound — SHIP. 3 non-blocking follow-ups
filed (#5570 .bak crash-window, #5571 fork-stitch corner, #5572 messaging clear).
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
When the WebUI server is launched from a venv python whose PATH does not
include git (e.g. hermes-agent venv on Windows), shutil.which('git')
returns None and WEBUI_VERSION falls back to 'unknown'. That freezes the
?v=<stamp> static-asset cache key, so browsers keep serving stale cached
ui.js/messages.js even after the server is restarted with fixed code —
frontend fixes silently never reach clients (recurring scroll jump-back).
Add a Windows fallback to _resolve_git_executable(): read the
Git-for-Windows registry InstallPath (HKLM + WOW6432Node + HKCU), then
probe common install dirs. Mirrors the existing darwin /usr/bin/git
fallback. No behavior change on posix or when git is already on PATH.
Codex 5th CORE: the ownership set only covered static _PROVIDER_MODELS, not the
active provider's own config.yaml providers.<active>.models allowlist. So an
active provider defined purely via providers: (no static catalog) could be
hijacked by another slug's overlapping entry earlier in config order. Fix folds
the active provider's own providers: allowlist (list + dict shapes, copilot
excluded) into _provider_models_set — completing the ownership model so the
active provider owns its declared models from EITHER the static catalog OR its
own providers: entry. Regression test (fail-without-fix verified). 40 model_resolver
tests green.
Co-authored-by: akay64 <akay64@users.noreply.github.com>
Codex 4th CORE (root latent bug in the shared canonicalizer): _canonicalise_provider_id
only accepted an alias resolution if the target was in _PROVIDER_DISPLAY. But some
canonical targets (e.g. 'gemini') are indexed in _PROVIDER_MODELS and NOT
_PROVIDER_DISPLAY, so 'google-gemini'->'gemini' was rejected and returned raw
'google-gemini' — leaving the ownership guard empty and letting providers.openai.models
hijack an active Gemini-owned model. Fix mirrors the direct-hit check (line 1445):
accept alias targets in EITHER _PROVIDER_DISPLAY OR _PROVIDER_MODELS. This is the
root fix for the whole alias-canonicalization class (gemini/google-gemini/
google-ai-studio etc.), not just the reported site. Blast-radius verified: 81
canonicalise/provider-alias tests across 5 files still green; x-ai still stays x-ai,
opencode_go still folds. Gemini regression test added (fail-without-fix verified).
Co-authored-by: akay64 <akay64@users.noreply.github.com>
Codex 3rd CORE (pre-existing latent bug the providers: scan exposes): the
_provider_models_set ownership lookup used the RAW config_provider (e.g. 'z-ai')
but _PROVIDER_MODELS is keyed by canonical slug ('zai'), so an aliased active
provider got an empty ownership set -> _skip_custom_providers guard-2 silently
failed -> another providers.<slug>.models entry could hijack an active-owned
model (verified: active z-ai + glm-5 + providers.openai.models.glm-5 -> openai).
Fix: canonicalise config_provider before the _PROVIDER_MODELS lookup (and reuse
the canonical slug for the scan's ownership guard). Regression test with an
aliased active provider (verified fail-without-fix).
This is the root-cause fix for the alias class — the ownership guard now holds
for every aliased provider (zai/kilo/etc.), not just the reported site.
Co-authored-by: akay64 <akay64@users.noreply.github.com>
Codex 2nd CORE: the new providers: scan didn't honor the _skip_custom_providers
active/default ownership guard (config.py:2535) that the custom_providers scan
uses. So with an active provider (e.g. ai-gateway) + model.default=gpt-5 AND a
providers.openai.models.gpt-5 entry, the scan hijacked gpt-5 to openai instead of
the active ai-gateway path. Fix: when _skip_custom_providers is true, the
providers: scan only considers the active provider's own canonical slug (skip all
other slugs). Regression tests: active-provider-owns-default not hijacked (fails
without fix); active provider's OWN providers: entry still matches.
Co-authored-by: akay64 <akay64@users.noreply.github.com>
Gate-cert (nesquena-hermes) + Codex found 1 CORE: akay64's new providers: scan in
resolve_model_provider() iterated ALL providers.<slug>.models INCLUDING
providers.copilot.models — but that key is a documented per-model SETTINGS map
(reasoning_effort/limits), NOT a routing allowlist (config.py exception). So a
Copilot per-model settings entry hijacked that model's routing to Copilot
(Codex reproduced gpt-5 -> copilot). Fix (Codex-exact): skip copilot in the scan.
Added the regression tests the PR shipped 0 of: user-defined allowlist routes;
active-openai + providers.copilot.models (dict + list shapes) stays openai
(both verified to fail without the exclusion).
Co-authored-by: akay64 <akay64@users.noreply.github.com>
Codex v4 SILENT finding: my audit Finding-2 fix made the orphan-.bak branch emit
a state_db_deleted_webui_tombstone item, but the state_db_missing_rows loop still
emitted the same sid/kind again when both a surviving .json.bak AND a state.db row
exist for one deleted session — double-counting unsafe_to_repair, violating the
no-double-classification invariant. Fix: track _bak_tombstoned_ids from the .bak
branch and skip them in the state.db loop. Regression test
test_audit_no_double_count_when_bak_and_state_db_row_both_survive (verified fails
without the dedup).
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
@rodboev independently pushed fixes for messaging-tombstone (matches ours) and an
audit orphan-.bak skip — but the audit skip used _marks_deleted_webui_session
(index heuristic), which re-introduces the crash-suppression bug, and the other 4
resurrection/recovery paths were still open. Rebuilt the stage on rodboev's new
head (fc21c3f3, attribution preserved) and applied the complete superset:
- routes.py claim path: tombstone 404 gated on _state_db_session_source in
('','webui','fork') so foreign-source rows always materialize (finding B)
- models.py sidebar projection: skip tombstoned source='webui' rows w/ no live
sidecar so a deleted session can't resurface in /api/sessions (finding C)
- session_recovery.py orphan-.bak startup: skip restore when durable tombstone
present, durable-only (finding D)
- session_recovery.py audit orphan-.bak: durable-only classify as
deleted_session_skipped (corrects rodboev's index-heuristic version)
- session_recovery.py _read_state_db_missing_sidecar_rows: durable-tombstone-only
so a genuine crash (index intact, no durable tombstone) still repairs (finding F)
Regression tests (all verified fail-without-fix): sidebar-projection, orphan-.bak
startup skip, messaging-not-tombstoned, index-only-crash-repairable. Both advisors
(Codex SAFE TO SHIP + Opus ship-with-fixes) cleared the prior stage; this rebuild
carries the same fixes onto the contributor's latest work.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Mirrors the existing custom_providers: scan immediately above. User-defined
providers declared under config.yaml's providers: section with explicit
models: lists were invisible to model-to-provider resolution — bare model
IDs fell through to the stale config_provider, routing through the wrong
endpoint.
The insertion site sits after custom_providers: (unchanged) and before
all heuristics (@provider:model, slash, fallback — all unchanged).
- Add test_profile_model_selection_accepts_model_in_overflow_extra_models
(the extra_models overflow-bucket regression test the warm dossier flagged
as the #5453 ship-pass note).
- Drop the stray dead pseudo-code comment left in handle_post branch route
(#5449 cosmetic nit).
read_importable_agent_session_rows() is a pure read, but it opened a
read-write sqlite connection on the live multi-GB WAL state.db and re-ran a
defensive CREATE INDEX self-heal on every sidebar build. Holding a
write-capable handle while the agent streams into the same DB adds needless
checkpoint/lock surface.
Open the DB read-only (file:...?mode=ro) for the projection, and self-heal a
missing idx_messages_session through a separate short-lived writable
connection only. With the index present (the normal case) the read path
performs zero writes; when it is missing the self-heal still runs and rows
still come back.
Behavior-neutral: identical rows when the index exists; self-heal preserved
when missing; read-only-fs degradation to the pre-aggregated path unchanged.
Tests: tests/test_issue5455_listing_readonly_connection.py (read-only open,
no writable connection when index present, self-heal when missing).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rod Boev