diff --git a/CHANGELOG.md b/CHANGELOG.md index 53dbd713..6498698a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,20 @@ ## [Unreleased] +## [v0.51.121] — 2026-05-24 — Release CS (stage-batch3 — 4-PR low-risk batch — state.db merge / display counts / compression marker / Windows launcher) + +### Fixed + +- **PR #2788** by @Carry00 — Prevent `state.db` messages being silently dropped during sidecar merge. Two related bugs were combining to discard historical messages: (1) `get_state_db_session_messages()` was selecting `role, content, timestamp` but NOT `id`, so every row was assigned a `("legacy", ...)` merge key instead of `("message_id", ...)`; (2) when a WebUI-origin session was continued via another Hermes surface (Gateway, CLI), the reader was always hitting the *active* profile's `state.db` rather than the session's own profile. Symptom: a 189-message session showed only 50 in the WebUI. Fix: include `id` in the SELECT when the column exists, and accept an optional `profile=` arg so cross-profile reads use the right database. Both callers in `api/routes.py handle_get` now thread `profile=getattr(s, 'profile', None)` through. + +- **PR #2797** by @ai-ag2026 — Align messaging session display counts with deduped display messages. The `message_count` returned by `/api/session` is the display coordinate space used for pagination and the header badge. Messaging-thread `state.db` metadata can carry raw duplicate transport rows (blank assistant separators between Discord/Slack thread turns) that `_merged_session_messages_for_display()` intentionally dedupes for rendering. The advertised count was the raw row count, so the frontend expected phantom messages after dedupe — `len(display_msgs) < message_count` triggered "load older" UI states that immediately returned nothing. Fix: `raw["message_count"] = _merged_message_count` for messaging sessions, computed from the same merge that produced the displayed messages. Adds `tests/test_gateway_sync.py::test_messaging_session_message_count_matches_deduped_display_messages` covering the regression. + +- **PR #2803** by @simjak — Compression-summary cards no longer use ordinary tool output that merely mentions context compression. The streaming auto-compression path was using a local broad substring matcher that fired on any message containing the strings "context compaction" / "context compression" / "context was auto-compressed" / "active task list was preserved across context compression", including skill/tool JSON output and ordinary user discussion about compaction. The strict predicate at `api/compression_anchor._is_context_compression_marker()` was already correctly scoped to synthetic marker prefixes on non-tool messages. Fix: expose the strict predicate as `is_context_compression_marker()` (public name) and route `api/streaming._is_context_compression_marker` through it as a backward-compatible alias. Tool/skill output that mentions compression no longer seeds `compression_anchor_summary` cards. + +### Added + +- **PR #2783** by @Koraji95-coder — Native Windows launcher and community-guide README link (squashed from 3 commits). `start.ps1` is a PowerShell equivalent of `start.sh` that bypasses `bootstrap.py`'s `ensure_supported_platform()` refusal and invokes `server.py` directly on native Windows. It mirrors `start.sh`'s discovery (load optional `.env` with the same readonly-var filter for `UID`/`GID`/`EUID`/`EGID`/`PPID`, find Python via `HERMES_WEBUI_PYTHON` env → `python3` → `python` → `py`, validate `HERMES_WEBUI_AGENT_DIR` on disk before use, prefer the agent's `venv\Scripts\python.exe`, set `HERMES_WEBUI_HOST` / `HERMES_WEBUI_PORT` / `HERMES_WEBUI_STATE_DIR` / `HERMES_HOME` defaults). The README adds a community-maintained native Windows setup section pointing to @markwang2658's `hermes-windows-native-guide` and `hermes-windows-native` repos with the documented memory delta (~330 MB native vs ~1080 MB WSL2+Docker). Closes both halves of #1952. Assumes Python + agent venv are already set up — first-time setup still needs WSL2 once to create the venv (`bootstrap.py` still refuses on native Windows). + ## [v0.51.120] — 2026-05-24 — Release CR (stage-batch2 — 3-PR low-risk batch — Bedrock provider / update check past-tag / CORS preflight) ### Added diff --git a/README.md b/README.md index f0074e2b..d82ecf3c 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,13 @@ The bootstrap will: > Native Windows is not supported for this bootstrap yet. Use Linux, macOS, or WSL2. > For Windows / WSL auto-start at login, see [`docs/wsl-autostart.md`](docs/wsl-autostart.md). -> A community-maintained native Windows guide is tracked in [#1952](https://github.com/nesquena/hermes-webui/issues/1952). + +A community-maintained native Windows setup is documented at [@markwang2658/hermes-windows-native-guide](https://github.com/markwang2658/hermes-windows-native-guide) (companion setup repo: [@markwang2658/hermes-windows-native](https://github.com/markwang2658/hermes-windows-native)). Notes from the community report in [#1952](https://github.com/nesquena/hermes-webui/issues/1952): + +- **Memory:** community-measured ~330 MB native vs ~1080 MB with WSL2+Docker (varies by configuration). +- **What works:** chat, workspace browser, session management, all themes. +- **Known limitations:** some POSIX-style file paths surface in the workspace browser; bash-assuming agent tools may not work natively. +- **WSL2 relationship:** WSL2 is recommended *once* for first-time venv creation (since `bootstrap.py` currently refuses on native Windows). After the venv exists, `start.ps1` at the repo root runs the WebUI natively by invoking `server.py` directly — no WSL2 needed for day-to-day use. If provider setup is still incomplete after install, the onboarding wizard will point you to finish it with `hermes model` instead of trying to replicate the full CLI setup in-browser. For a step-by-step walkthrough of the wizard, provider choices, local model server Base URLs, and safe re-runs, see [`docs/onboarding.md`](docs/onboarding.md). diff --git a/api/compression_anchor.py b/api/compression_anchor.py index f251851c..12d96415 100644 --- a/api/compression_anchor.py +++ b/api/compression_anchor.py @@ -53,7 +53,7 @@ def _content_has_part_type(content, part_types): ) -def _is_context_compression_marker(message): +def is_context_compression_marker(message): """Return true for synthetic compression/reference cards, not user turns.""" if not isinstance(message, dict): return False @@ -71,6 +71,11 @@ def _is_context_compression_marker(message): ) +def _is_context_compression_marker(message): + """Backward-compatible alias for callers that have not switched yet.""" + return is_context_compression_marker(message) + + def visible_messages_for_anchor(messages, *, auto_compression: bool = False): """Return transcript messages that can anchor compression UI metadata. diff --git a/api/models.py b/api/models.py index 652bde3f..1fde1b51 100644 --- a/api/models.py +++ b/api/models.py @@ -2815,21 +2815,28 @@ def _json_loads_if_string(value): return value -def get_state_db_session_messages(sid, *, stitch_continuations: bool = False) -> list: - """Read messages for a Hermes session from the active profile's state.db. +def get_state_db_session_messages(sid, *, stitch_continuations: bool = False, profile=None) -> list: + """Read messages for a Hermes session from state.db. - This generic reader intentionally works for any session source, including - WebUI-origin sessions that were later updated through another Hermes surface - such as the Gateway API Server. When ``stitch_continuations`` is true it - preserves the historical CLI/external-agent behavior of walking compatible - compression/close parent segments before reading messages. + When *profile* is supplied, reads from that profile's state.db; otherwise + falls back to the active profile's state.db. This generic reader works for + any session source, including WebUI-origin sessions that were later updated + through another Hermes surface such as the Gateway API Server. When + ``stitch_continuations`` is true it preserves the historical CLI/external-agent + behavior of walking compatible compression/close parent segments before reading + messages. """ try: import sqlite3 except ImportError: return [] - db_path = _active_state_db_path() + if isinstance(profile, str) and profile: + db_path = _get_profile_home(profile) / 'state.db' + if not db_path.exists(): + db_path = _active_state_db_path() + else: + db_path = _active_state_db_path() if not db_path.exists(): return [] @@ -2852,7 +2859,8 @@ def get_state_db_session_messages(sid, *, stitch_continuations: bool = False) -> 'reasoning_content', 'codex_message_items', ] - selected = ['role', 'content', 'timestamp'] + [c for c in optional if c in available] + id_col = ['id'] if 'id' in available else [] + selected = id_col + ['role', 'content', 'timestamp'] + [c for c in optional if c in available] session_chain = [str(sid)] if stitch_continuations: diff --git a/api/routes.py b/api/routes.py index efc76b06..6dc8dfb5 100644 --- a/api/routes.py +++ b/api/routes.py @@ -3752,17 +3752,18 @@ def handle_get(handler, parsed) -> bool: cli_messages = [] state_db_messages = [] sidecar_metadata_messages = None + _session_profile = getattr(s, 'profile', None) or None if is_messaging_session: cli_messages = get_cli_session_messages(sid) elif load_messages: - state_db_messages = get_state_db_session_messages(sid) + state_db_messages = get_state_db_session_messages(sid, profile=_session_profile) elif not is_messaging_session: # Metadata-only callers still need the same append-only # reconciliation contract as full loads. A raw state.db summary # can count stale rows that the merge intentionally filters out, # which makes sidebar polling think the transcript is always # newer than the loaded conversation. - state_db_messages = get_state_db_session_messages(sid) + state_db_messages = get_state_db_session_messages(sid, profile=_session_profile) sidecar_metadata_session = Session.load(sid) sidecar_metadata_messages = ( getattr(sidecar_metadata_session, "messages", []) or [] @@ -3926,6 +3927,13 @@ def handle_get(handler, parsed) -> bool: ) if cli_meta and _is_messaging_session_record(cli_meta): raw = _merge_cli_sidebar_metadata(raw, cli_meta) + # ``message_count`` in /api/session is the display coordinate + # space used for pagination and the header badge. Messaging + # state.db metadata can include raw duplicate transport rows that + # _merged_session_messages_for_display() intentionally dedupes; + # keep the raw count available as ``actual_message_count`` but + # do not let it make the frontend expect phantom messages. + raw["message_count"] = _merged_message_count # Signal to the frontend that older messages were omitted. # For msg_before paging, compare against the filtered set, # not the full list — otherwise we signal truncation even when diff --git a/api/streaming.py b/api/streaming.py index 663d84d1..5718ee1c 100644 --- a/api/streaming.py +++ b/api/streaming.py @@ -35,7 +35,7 @@ from api.config import ( load_settings, ) from api.helpers import redact_session_data, _redact_text -from api.compression_anchor import visible_messages_for_anchor +from api.compression_anchor import is_context_compression_marker, visible_messages_for_anchor from api.metering import meter from api.run_journal import RunJournalWriter from api.turn_journal import append_turn_journal_event_for_stream @@ -2299,15 +2299,7 @@ def _dedupe_replayed_active_context(previous_context, result_messages): def _is_context_compression_marker(msg): - if not isinstance(msg, dict): - return False - text = _message_text(msg.get('content', '')).lower() - return ( - 'context compaction' in text - or 'context compression' in text - or 'context was auto-compressed' in text - or 'active task list was preserved across context compression' in text - ) + return is_context_compression_marker(msg) def _compact_summary_text(raw_text: str | None, limit: int = 320) -> str | None: diff --git a/start.ps1 b/start.ps1 new file mode 100644 index 00000000..191db6c1 --- /dev/null +++ b/start.ps1 @@ -0,0 +1,156 @@ +<# +.SYNOPSIS + Native Windows launcher for Hermes WebUI - PowerShell equivalent + of start.sh, bypassing bootstrap.py's platform refusal. + +.DESCRIPTION + Mirrors start.sh's discovery: load optional .env, find Python, + locate the hermes-agent install, set sensible env defaults, then + invoke server.py directly. The bootstrap.py path is skipped + because it currently raises on platform.system() == 'Windows'; + server.py itself runs cleanly on native Windows. + + Assumes Python + hermes-agent + the WebUI Python deps are already + installed - same assumption start.sh makes when invoked outside + a fresh bootstrap. For first-time setup, run bootstrap.py inside + WSL2 once to create the venv, then this script can use that venv. + +.PARAMETER Port + TCP port the WebUI binds to. Overrides HERMES_WEBUI_PORT env. + Default: 8787. + +.PARAMETER BindHost + Bind address. Overrides HERMES_WEBUI_HOST env. + Default: 127.0.0.1. + +.EXAMPLE + .\start.ps1 + # Bind to 127.0.0.1:8787, foreground. + +.EXAMPLE + .\start.ps1 -Port 9000 + # Bind to 127.0.0.1:9000. + +.EXAMPLE + $env:HERMES_WEBUI_HOST = '0.0.0.0' + .\start.ps1 + # Bind to all interfaces (set a password first via env or Settings). + +.LINK + https://github.com/nesquena/hermes-webui/issues/1952 +#> + +[CmdletBinding()] +param( + [int]$Port = 0, + [string]$BindHost = '' +) + +$ErrorActionPreference = 'Stop' +$RepoRoot = Split-Path -Parent $PSCommandPath + +# === Load .env (mirroring start.sh's filtering) ======================== +$envFile = Join-Path $RepoRoot '.env' +if (Test-Path $envFile) { + foreach ($line in Get-Content $envFile -Encoding UTF8) { + $trimmed = $line.Trim() + if (-not $trimmed -or $trimmed.StartsWith('#') -or -not $trimmed.Contains('=')) { continue } + $kv = $trimmed -split '=', 2 + $key = ($kv[0].Trim() -replace '^export\s+', '') + # Filter out shell-readonly vars (UID, GID, EUID, EGID, PPID) per start.sh + if ($key -in @('UID', 'GID', 'EUID', 'EGID', 'PPID')) { continue } + if ($key -notmatch '^[A-Za-z_][A-Za-z0-9_]*$') { continue } + # Explicit $null check — an env var explicitly set to '' should still + # be considered "set" and NOT overridden by .env (empty string is + # falsey in PowerShell, so a plain truthy check would mis-skip). + if ($null -ne [Environment]::GetEnvironmentVariable($key)) { continue } + $val = $kv[1] + if ($val -match '^"(.*)"$') { $val = $Matches[1] } + elseif ($val -match "^'(.*)'$") { $val = $Matches[1] } + [Environment]::SetEnvironmentVariable($key, $val) + } +} + +# === Find Python (matches start.sh order) ============================== +$Python = $env:HERMES_WEBUI_PYTHON +if (-not $Python) { + foreach ($candidate in @('python3', 'python', 'py')) { + $cmd = Get-Command $candidate -ErrorAction SilentlyContinue + if ($cmd) { $Python = $cmd.Source; break } + } +} +if (-not $Python) { + Write-Error 'Python 3 is required to run server.py (set HERMES_WEBUI_PYTHON or add python to PATH).' + exit 1 +} + +# === Find Hermes Agent dir (server.py imports from it) ================= +# When HERMES_WEBUI_AGENT_DIR is set we still validate it on disk — +# an explicit override pointing at a missing dir should fail FAST +# with a clear message, not silently progress into a python3 launch +# that's about to crash on missing imports. Smoke-test feedback on +# PR #2783: nesquena/hermes-webui requested this guard. +$AgentDir = $env:HERMES_WEBUI_AGENT_DIR +if ($AgentDir -and -not (Test-Path (Join-Path $AgentDir 'hermes_cli'))) { + Write-Error "HERMES_WEBUI_AGENT_DIR is set to '$AgentDir' but no hermes_cli/ folder exists there. Unset the variable to fall back to auto-discovery, or fix the path." + exit 1 +} +if (-not $AgentDir) { + $candidates = @( + (Join-Path $env:USERPROFILE '.hermes\hermes-agent'), + (Join-Path (Split-Path -Parent $RepoRoot) 'hermes-agent') + ) + foreach ($c in $candidates) { + if (Test-Path (Join-Path $c 'hermes_cli')) { $AgentDir = $c; break } + } +} +if (-not $AgentDir) { + $expectedPrimary = Join-Path $env:USERPROFILE '.hermes\hermes-agent' + $expectedSibling = Join-Path (Split-Path -Parent $RepoRoot) 'hermes-agent' + Write-Error "hermes-agent not found at $expectedPrimary or $expectedSibling. Set HERMES_WEBUI_AGENT_DIR explicitly." + exit 1 +} + +# === Prefer the agent's venv Python if available ======================= +$agentVenvPython = Join-Path $AgentDir 'venv\Scripts\python.exe' +if (Test-Path $agentVenvPython) { + $Python = $agentVenvPython +} + +# === Resolve bind + state defaults ===================================== +$BindHostFinal = if ($BindHost) { $BindHost } elseif ($env:HERMES_WEBUI_HOST) { $env:HERMES_WEBUI_HOST } else { '127.0.0.1' } +$PortFinal = if ($Port) { $Port } elseif ($env:HERMES_WEBUI_PORT) { [int]$env:HERMES_WEBUI_PORT } else { 8787 } +$env:HERMES_WEBUI_HOST = $BindHostFinal +$env:HERMES_WEBUI_PORT = "$PortFinal" +if (-not $env:HERMES_WEBUI_STATE_DIR) { + $env:HERMES_WEBUI_STATE_DIR = Join-Path $env:USERPROFILE '.hermes\webui' +} +if (-not $env:HERMES_HOME) { + $env:HERMES_HOME = Join-Path $env:USERPROFILE '.hermes' +} + +# === Ensure dirs exist ================================================= +New-Item -ItemType Directory -Force -Path $env:HERMES_HOME | Out-Null +New-Item -ItemType Directory -Force -Path $env:HERMES_WEBUI_STATE_DIR | Out-Null + +# === Launch (foreground, matches start.sh) ============================= +Write-Host "[start.ps1] Hermes WebUI native Windows launcher" -ForegroundColor Cyan +Write-Host "[start.ps1] Python: $Python" +Write-Host "[start.ps1] Agent dir: $AgentDir" +Write-Host "[start.ps1] State dir: $env:HERMES_WEBUI_STATE_DIR" +Write-Host "[start.ps1] Binding: ${BindHostFinal}:${PortFinal}" +Write-Host "" + +$serverPath = Join-Path $RepoRoot 'server.py' +if (-not (Test-Path $serverPath)) { + Write-Error "server.py not found at $serverPath - is this the hermes-webui repo root?" + exit 1 +} + +Push-Location $RepoRoot +try { + & $Python $serverPath @args + exit $LASTEXITCODE +} finally { + Pop-Location +} diff --git a/tests/test_gateway_sync.py b/tests/test_gateway_sync.py index 1cca29de..7181972d 100644 --- a/tests/test_gateway_sync.py +++ b/tests/test_gateway_sync.py @@ -1708,6 +1708,77 @@ def test_session_prefers_state_db_messages_over_stale_local_snapshot(cleanup_tes pass +def test_messaging_session_message_count_matches_deduped_display_messages(cleanup_test_sessions): + """Thread sessions must not advertise raw DB rows that display merge dedupes away.""" + from api.models import Session + + conn = _ensure_state_db() + sid = 'gw_display_count_regression_001' + cleanup_test_sessions.append(sid) + base_ts = time.time() - 60 + rows = [ + ("user", "Thread question", base_ts + 1), + ("assistant", "", base_ts + 2), + ("assistant", "", base_ts + 2), + ("tool", '{"ok": true}', base_ts + 3), + ("assistant", "Thread answer", base_ts + 4), + ] + raw_db_count = len(rows) + try: + _insert_gateway_session( + conn, + session_id=sid, + source='discord', + title='Discord Thread Count Regression', + message_count=raw_db_count, + started_at=base_ts, + ) + conn.execute("DELETE FROM messages WHERE session_id = ?", (sid,)) + for role, content, ts in rows: + _insert_message(conn, sid, role, content, ts) + conn.execute( + "UPDATE sessions SET message_count = ? WHERE id = ?", + (raw_db_count, sid), + ) + conn.commit() + + # A stale WebUI sidecar can exist for the same messaging thread. The API + # display merge dedupes repeated blank assistant separators, so the + # advertised count must match the returned display coordinate space, not + # the raw state.db row count. + s = Session( + session_id=sid, + title='Legacy Discord Snapshot', + workspace='/tmp/hermes-webui-test', + model='openai/gpt-5', + messages=[{"role": "user", "content": "Thread question", "timestamp": base_ts + 1}], + session_source='messaging', + raw_source='discord', + source_tag='discord', + source_label='Discord', + ) + s.save(touch_updated_at=False) + + post('/api/settings', {'show_cli_sessions': True}) + data, status = get(f'/api/session?session_id={sid}&messages=1&resolve_model=0&msg_limit=100') + assert status == 200, data + session = data.get('session', {}) + msgs = session.get('messages', []) + assert msgs[-1].get('content') == 'Thread answer' + assert len(msgs) < raw_db_count, "fixture must exercise display dedupe" + assert session.get('message_count') == len(msgs) + finally: + try: + _remove_test_sessions(conn, sid) + conn.close() + except Exception: + pass + try: + post('/api/settings', {'show_cli_sessions': False}) + except Exception: + pass + + def test_sessions_prefers_state_db_metadata_for_messaging_overlap(cleanup_test_sessions): """Sidebar metadata for messaging sessions should come from state.db, not local JSON snapshots.""" conn = _ensure_state_db() diff --git a/tests/test_issue2028_compression_anchor_helpers.py b/tests/test_issue2028_compression_anchor_helpers.py index 1fcb4f6a..d675d2a0 100644 --- a/tests/test_issue2028_compression_anchor_helpers.py +++ b/tests/test_issue2028_compression_anchor_helpers.py @@ -4,7 +4,8 @@ Regression coverage for shared compression-anchor visibility helpers (#2028). from pathlib import Path -from api.compression_anchor import visible_messages_for_anchor +from api.compression_anchor import is_context_compression_marker, visible_messages_for_anchor +from api.streaming import _compression_summary_from_messages, _is_context_compression_marker def test_legacy_duplicate_anchor_helpers_are_removed(): @@ -57,3 +58,42 @@ def test_visible_messages_for_anchor_keeps_manual_user_messages_simple(): [user_tool_metadata, user_attachment, assistant_tool_metadata], auto_compression=True, ) == [user_tool_metadata, user_attachment, assistant_tool_metadata] + + +def test_context_compression_marker_detection_is_prefix_and_role_scoped(): + real_marker = { + "role": "assistant", + "content": "[CONTEXT COMPACTION — REFERENCE ONLY] Earlier turns were compacted.", + } + preserved_tasks_marker = { + "role": "user", + "content": "[Your active task list was preserved across context compression] - [ ] follow up", + } + tool_noise = { + "role": "tool", + "content": "{\"description\": \"Troubleshoot frequent context compression indicators\"}", + } + user_discussion = { + "role": "user", + "content": "Why do I see context compression after every message?", + } + + assert is_context_compression_marker(real_marker) + assert is_context_compression_marker(preserved_tasks_marker) + assert _is_context_compression_marker(real_marker) + assert not is_context_compression_marker(tool_noise) + assert not is_context_compression_marker(user_discussion) + + +def test_compression_summary_ignores_tool_output_that_mentions_compression(): + marker = { + "role": "assistant", + "content": "[CONTEXT COMPACTION — REFERENCE ONLY] Keep this handoff as reference.", + } + skill_tool_output = { + "role": "tool", + "content": "{\"name\": \"hermes-webui-operations\", \"content\": \"Troubleshooting frequent context compression indicators...\"}", + } + + assert _compression_summary_from_messages([marker, skill_tool_output]) == marker["content"] + assert _compression_summary_from_messages([skill_tool_output]) is None