Codex gate caught that the new load_pool() path runs under the process-default
profile (sibling endpoints /api/providers + /api/models/live already wrap).
Without it a multi-profile client sees+seeds the default profile's pool
(#4247/#4067 profile-isolation class).
Codex+Opus gate caught that the forward-scan-only keep produced [user, _recovered
user, assistant] -> adjacent users -> strict-provider 400 once the anchoring
assistant's prev kept neighbour was a user. Keep a recovered user ONLY when it
separates two assistants (prev_kept==assistant AND next==assistant); mirror in
_api_safe_message_positions. + 3 regression tests.
6-round RCE-class security hardening, converged. Codex SAFE (live exploit probes) + Opus SAFE (flag-off-reachability enumeration) + full suite 9508 on the byte-identical gated head. Nathan signed off.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Closes#3879Closes#3777
Codex CORE findings on the fingerprint: (1) default busy timeout could stall
/api/sessions ~10s on a locked state.db; (2) COUNT(*) forced a full messages
table SCAN (~16-22ms/refresh on 200k rows). Fix: open file:...?mode=ro with
timeout=0.05 + PRAGMA busy_timeout=50 (returns fast, falls back to stat stamp on
lock), and use index-only MAX(rowid) per table (no COUNT, no scan). Verified:
EXPLAIN shows SEARCH not SCAN, 0.36ms on 200k msgs, 0.16ms while EXCLUSIVE-locked.
Fingerprint shape is now (sessions_max_rowid, messages_max_rowid); regression
tests updated.
Codex root-cause consult confirmed: the CLI-session cache (_CLI_SESSIONS_CACHE,
5s TTL) and session-list cache were keyed on (st_mtime_ns, st_size) of state.db
+ WAL sidecars. Under WAL-mode writes those stamps collide (same mtime-ns bucket
+ WAL frame size after checkpoint), serving a stale cache -> freshly-committed
gateway/CLI session intermittently absent from /api/sessions (the recurring
test_gateway_sync flake AND a real up-to-5s sidebar delay for gateway writes).
PRAGMA data_version is NOT usable here (fresh per-request connection always
reports its own initial value, never advances — verified). Fix = a cheap
content fingerprint (COUNT/MAX(rowid) over sessions+messages) read on a fresh
connection, which DOES advance on every commit and is immune to mtime
granularity. Wired into _sqlite_file_stat_cache_key (models) + the routes-layer
source stamp. + 3 regression tests + corrected the earlier false root-cause
comments.
Codex completeness finding: show_previous_messaging_sessions is a third
visibility setting in the /api/sessions cache key (bypasses messaging dedupe)
with the same stale-cache bug. Added it to the invalidation tuple.
ROOT FIX (product bug): POST /api/settings never invalidated the session-list
cache, so toggling show_cli_sessions/show_cron_sessions could serve a stale
sidebar for up to the cache TTL when the settings-file mtime stamp collided with
a cached entry. The handler now clears the cache directly on those toggles. This
is also the root cause of the recurring test_gateway_sync CI flake (session
absent from /api/sessions right after the visibility toggle).
DEFENSE-IN-DEPTH: cleanup_test_sessions fixture now resets show_cli_sessions to
default BEFORE each test too, not only in teardown.
FLAKE FIX (test infra): bump 31 node-subprocess correctness-harness timeouts from
10s -> 30s across 23 test files. Node spawn + V8 warmup under parallel CI shard
load can exceed 10s, causing TimeoutExpired flakes (e.g. test_1325 on #4445).
Only real node subprocess.run() sites changed — mock signatures, HTTP urlopen,
and deliberate timeout-behavior tests (test_api_timeout, timeout=0/1/2) untouched.
Codex re-gate found two issues in the uncapped-read fix: (1) open_anchored_fd
opens blocking O_RDONLY, which HANGS on a workspace FIFO/special file swapped in
at a regular-file path; (2) the non-regular fstat branch returned without
closing the fd (leak). Fix: lstat the symlink-resolved target BEFORE any open to
skip non-regular leaves (no blocking open on a FIFO), and close the fd on the
non-regular fstat branch. Adds a threaded FIFO-no-hang regression test.
Codex regression gate found that routing workspace reads through
read_file_content() introduced a 400KB MAX_FILE_BYTES cap not present on master
(Path.read_text), so a large but legitimate workspace file raised ValueError ->
caught -> None -> the rollback diff falsely reported it as DELETED. Replace with
an uncapped anchored O_NOFOLLOW read (safe_resolve_ws + open_anchored_fd +
S_ISREG fstat check) that keeps the symlink-escape protection but has no size
cap. Adds a >400KB regression test (RED on the capped path, GREEN with the fix).
Co-authored-by: Hinotoi-agent <Hinotoi-agent@users.noreply.github.com>
Codex regression gate found that _session_model_identity_matches normalized
only the @provider:model form, not provider/model (slash). A session stored as
'deepseek/deepseek-v4-1m' resolving to bare 'deepseek-v4-1m' was treated as a
model change -> the 256k fallback bypassed the accept-guard and clobbered the
persisted 1M window (the same #4248 bug on slash-qualified IDs). Mirror the
slash-aware split already used by _model_matches_configured_default. Adds a
route-level regression + unit test for the slash shape.
Co-authored-by: franksong2702 <franksong2702@users.noreply.github.com>
Restore previously used pathname-based shutil.copy2() which follows an in-tree
symlink whose target escapes the workspace. Route restore writes through the
workspace-anchored file helpers so a restored file always lands inside the
workspace tree. +regression test (escaping symlink refused, normal restore ok).
Co-authored-by: hinotoi-agent <paperlantern.agent@gmail.com>
A user prompt that exists only in Hermes state.db (no sidecar copy) and falls
chronologically BEFORE a newer sidecar message tail was being appended after the
tail (or skipped) by merge_session_messages_append_only — losing or mis-ordering
it on load. The merge now inserts such a state.db user message at its correct
chronological position via _insert_state_message_chronologically, with a
role-aware tie-break for equal timestamps. Append-only invariant + dedup
(seen_content_keys) preserved.
Deterministic-ordering guard (maintainer add, addresses Codex review): on an
equal-timestamp collision the insert no longer lands at a slot whose left
neighbour shares the message's role — it advances past the same-role
equal-timestamp run, so an already-answered earlier turn can't be re-ordered
into user,user,assistant. (The agent core merges adjacent users before send, so
this was benign in practice, but the guard keeps the merged transcript correctly
ordered regardless.) +1 same-second multi-turn regression test, toothless-checked.
Co-authored-by: Dennis Soong <dso2ng@gmail.com>
Follow-up to #4385/#4388: that fix stopped stale cron sidecars from being
treated as CLI rows, but get_cli_sessions() still re-projected the same cron
run from Hermes state.db with a hardcoded archived: false after all_sessions()
omitted the hidden sidecar — so an archived run reappeared on refresh.
- New _state_projection_sidecar_metadata(sid) helper returns UI-owned title +
archived flag from the WebUI sidecar (load_metadata_only), degrading
gracefully to {title:None, archived:False} on any miss.
- Both state.db-projection paths (standard CLI + cron fallback) now use it and
carry the archived flag onto the projected row (was hardcoded False), so an
archived cron/tool/API run stays hidden regardless of which path surfaces it.
Title-preference behavior (#1486 rename persistence) is preserved.
Co-authored-by: TomBanksAU <246584738+TomBanksAU@users.noreply.github.com>
A stale sidecar with is_cli_session=true could make an archived
cron/tool/API session resurface in the Sessions tab on refresh. The
session loader now treats explicit cron/tool/api source metadata as
non-CLI even when the sidecar claims is_cli_session, and preserves cron
identity when the archive/materialization fallback creates a session.
Co-authored-by: TomBanksAU <246584738+TomBanksAU@users.noreply.github.com>
Adds WebUI controls to manage the messaging gateway lifecycle without a
terminal. _handle_gateway_lifecycle routes start/stop/restart to the
existing platform-aware `hermes gateway {action}` CLI via a list-argv
subprocess (no shell), scoped to the active profile, with bounded
timeout/error handling and a status-card refresh. Auth + CSRF gated;
profile name charset-validated; stdout/stderr never leaked to the client.
Maintainer hardening (warm-up SHOULD-FIX): added _GATEWAY_ACTION_LOCK, a
server-side single-flight guard. The client disables its button in-flight,
but a scripted authed client could otherwise fire overlapping
start/stop/restart calls and spawn concurrent gateway subprocesses. A
non-blocking acquire now returns 409 on contention (mirrors the self-update
_apply_lock pattern), released in finally. Added a regression test
(toothless-checked: without the lock the 2nd action spawns a subprocess +
returns 200).
Cross-platform: delegates to the mature hermes_cli gateway subcommand
(systemctl/launchctl), pathlib paths, PYTHONUTF8=1 + BROWSER=echo in env.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Surfaces the Hermes agent's LLM Wiki as a read-only knowledge-base
observability section in Insights (status counts + browsable read-only
pages). Unconfigured shows a clear "Unavailable" empty state.
Strictly read-only, auth-gated. The wiki file-serving path is hardened
against the full traversal/symlink/TOCTOU class (Codex + Opus review, each
vector regression-tested + toothless-checked):
- _llm_wiki_page_files resolves the wiki root once as the trust base;
skips a section whose resolved path escapes the root; requires each *.md
target's RESOLVED path under both real root and real section; rejects
dot-prefixed segments on lexical AND resolved-relative paths (closes
symlink-to-hidden/out-of-tree + symlinked-section escapes). Browse + read
share this allowlist.
- /api/wiki/page rejects abs paths + a real '..' SEGMENT (legit v1..v2.md
still opens); requires the resolved target in the allowlist; reads the
resolved path with O_NOFOLLOW; and fstat-verifies the open fd's
(st_dev, st_ino) against the identity captured at allowlist time — closing
final-component AND parent-directory symlink-swap TOCTOU races. Any
mismatch/vanish -> clean 404, never a 500.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
The WebUI gateway chat backend routed every browser chat turn through the
gateway's /v1/runs API whenever the gateway advertised approval/runs
support (gateway_chat.py capability gate). The runs API creates an isolated
Api_Server session per turn, so mid-conversation context was lost between
messages. Routing through /v1/runs is now gated behind an explicit opt-in
(_gateway_use_runs_api_enabled: HERMES_WEBUI_GATEWAY_USE_RUNS_API env var or
webui_gateway_use_runs_api config key, env-over-config precedence, default
off). The capability gate now requires BOTH the opt-in flag AND gateway
approval support before using /v1/runs; otherwise normal chat stays on
/v1/chat/completions, preserving the session via X-Hermes-Session-Id.
Operators who want the in-gateway-chat approval-events flow re-enable it
with the flag.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Docker images and pip installs have no .git directory, so _check_repo
returned None and the frontend interpreted both update-target parts being
falsy as "Up to date" — a stale Docker image would misleadingly look
current. _check_repo now returns a minimal sentinel dict ({name, behind:
None, no_git: True}) instead of None so the Settings panel can distinguish
"can't check" from "up to date", and _formatUpdateTargetStatus /
checkUpdatesNow surface a localized "Can't check for updates" state
(settings_update_no_git, added across all 13 locale blocks). In mixed
deployments where one component has a git checkout and the other does not,
the valid git-side update info is still shown. The sentinel's behind: None
flows safely through the existing notification/summary builders, which all
gate on int(info.get('behind') or 0) > 0.
Co-authored-by: Rod Boev <rod.boev@gmail.com>
Ships @dso2ng's fix: /api/session now reconciles stale CLI source flags
against the state.db WebUI-origin row, and _isExternalSession() refuses to
classify an explicit WebUI-source session as external — so a WebUI-native
transcript with a stale sidecar isn't force-reloaded like an imported one.
Rebased onto master (contributor's net code-diff applied; byte-identical to
PR head). CHANGELOG stamped v0.51.474.
Co-authored-by: dso2ng <dso2ng@users.noreply.github.com>
Opus review note: the substring match in _stale_prefix_matches_prior_user_context
is intentionally loose, but correctness depends on the caller's
suffix-equals-entire-submitted-turn invariant (cleaning can only ever rewrite a
flagged row to the user's actual current turn — never drop legitimate content).
Comment-only; no behavior change.
Virtualization (#500/#4214) caused a scroll-up flicker on long sessions with
tall tool-call rows (variable-height anchor oscillation), distinct from the
stream-completion jump fixed in #4325. Per maintainer decision, flip the
'Virtualize long transcripts' preference to experimental/opt-IN (default OFF)
for everyone until the scroll behavior is reliable.
- api/config.py: virtualize_transcript default True->False; add
virtualize_transcript_optin marker (both bool-allowlisted); load_settings
force-off migration — a stored true WITHOUT the post-flip opt-in marker is
reset to off, so 100% of existing users land on off; an explicit opt-in
(marker present) is honored.
- boot.js / panels.js: default-off semantics (===true), persist the opt-in
marker on enable, checkbox not pre-checked.
- index.html / i18n.js (en): relabel experimental + flicker warning.
- Tests: rewrite the #4325 toggle assertions for the opt-in contract + 4 new
migration tests (unset/stale-true/explicit-optin/marker-without-true).
Phase B (root-cause scroll-stability) tracked separately. Thanks @b3nw.
Adds a Preferences setting 'Virtualize long transcripts' (default ON, opt-out).
When disabled, _currentMessageVirtualWindow returns a full non-virtualized window
so the entire transcript renders at once (also short-circuits the virtual
re-measure loop). Server-persisted (virtualize_transcript bool, default True),
hot-applies via renderMessages on toggle. i18n in all 13 locales.
Pairs with @rodboev's #4328 stream-end anchor scroll fix (cherry-picked into
this branch) to fully resolve#4325.
Independent review (APPROVED clean) + greptile both flagged one non-blocking nit:
_is_known_model_provider swallowed a transient _is_plugin_model_provider failure
at debug level, so a real plugin-backed provider could briefly vanish from the
picker with no visible log. Promoted to logger.warning(exc_info=True).
Also corrected the endpoint-error-hint comments in ui.js: they cited #4253, which
is actually a cron release (mislabel carried from #4279's body). Comment-only —
no logic change to the browser-verified collapsible-groups code.
Fixes the Photon phantom-providers regression from #4247 (v0.51.461): the
credential-pool detection loop added every auth.json credential_pool key to
detected_providers without checking it names a model provider, so the Photon
iMessage plugin's photon/photon_project/photon_user messaging credentials each
rendered as a phantom provider carrying the full auto-detected model catalog
(251 models on the reporter's machine).
Gated on a new _is_known_model_provider() check (custom:* slug, static
_PROVIDER_DISPLAY/_PROVIDER_MODELS entry, or registered model-provider plugin)
on both the load_pool and raw-dict fallback paths; the group-builder's generic
fallback no longer paints an unrecognized id with the global catalog. Real
pool-keyed providers (copilot, aliased google->gemini) still surface.
Integrates the stronger of two independent same-fix PRs — #4330's implementation
(thorough docstring, try/except-hardened plugin check, preserved #1881 comment,
defensive else branch) + its 8-case test that mocks at the WebUI boundary
(stubs hermes_cli, never imports the bundled agent package that's absent in CI,
exercises both the load_pool and ImportError raw-dict paths) — over #4329's
2-case test. Both rodboev (#4329) and the maintainer agent (#4330) independently
found and fixed the same regression.
Reported by Nic.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
Gate follow-ups on the converged head, all fail-closed hardening with no
contradiction to the existing security model:
- _as_posix_path rejects embedded null bytes (returns None).
- _safe_resolve catches ValueError (null-byte .resolve()) -> raw path so the
block gate rejects cleanly.
- _workspace_access_error returns a clean 'Invalid path' validation error on
stat() ValueError instead of an uncaught 500.
- /api/workspaces/add route preflight resolve() now catches
(ValueError, OSError, RuntimeError) -> clean 400.
Reconciliation note: an interim gate suggestion to restore ~user/~root
expansion and block /root was REVERTED — the full suite surfaced
tests/test_batch_fixes.py::TestRootWorkspaceUnblocked, which pins the
deliberate #510/#521 decision that /root is NOT blocked (Hermes runs as root
in many deployments, where /root is the legitimate home, allowed via the home
carve-out). Expanding ~root for a non-root deployment would register root's
home, so the PR's original 'leave ~user literal' behavior is the correct,
safer choice. Kept only the null-byte fixes; added a guard test pinning that
/root stays unblocked.
Co-authored-by: rodboev <rodboev@users.noreply.github.com>
nesquena-hermes