Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -505,7 +505,7 @@ When `session.enabled` is true (default) and `listener.session_api_addr` is non-
| Method & Path | Format | Purpose |
|---|---|---|
| `GET /` | text | One-line-per-endpoint index. Answers "is this the session API, and on the right port?" — the reason a 404 here was worth replacing. |
| `GET /v1/sessions` | `application/json` | List active sessions: `{sessions: [{id, createdAt, updatedAt, eventCount, title, agent, adopted, totalTokens, costMicros, avoidedMicros, saturated, active, promptContext}]}`. `id`, `createdAt`, `updatedAt`, `eventCount` and `active` are always present; every other field is `omitempty` — absent rather than zero, on the standing rule that an unknown value must not render as a real one. (Do not read that off the position of `active`: it sits second-to-last, between two `omitempty` fields.) **`title` is a suggestion, not an identifier:** the proxy derives it from the session's own events (a `/rename`, else a `<user_query>`, else ordinary user prose, with `<system-reminder>` blocks excised), so it is a display convenience and nothing addresses a session by it. Absent when nothing in the events named it. **Folded at append time and FIRST-WINS, except that a `/rename` always overrides** — so ordinary conversation does not re-title a session on every turn, and a `/rename` survives eviction of the event that carried it. agentop reads this field as a FALLBACK: its TITLE column prefers a harvested Claude Code transcript title and uses the served title only for a session the harvest cannot name. That precedence is fixed rather than a judgement about which string is better — both sides rank candidates their own way and do not agree on every session. That is the case worth having: an agent with no transcript tree on the operator's disk still routes through the proxy, so a row that used to render blank now has a name. agentop deliberately still treats such a row as unnamed for its own re-harvest backoff, so a served title does not stop it looking for a harvested one. **`?archived=true` adds history** where the proxy runs a session archive (`session.archive`, laptop only — see the chatty-traffic gotcha): every session the archive holds and memory does not, marked `resident: false`, plus a top-level `archive` object — `{bytes, maxBytes, retentionDays}` and, only when nonzero, `droppedEvents`, `writeErrors`, `droppedRenames`, `paused`, any of which means the history has gaps. A session in both is listed once, as its resident row. Without an archive the parameter is ignored and no `archive` object is sent, which is how a client tells "no history here" from "no history yet". |
| `GET /v1/sessions` | `application/json` | List active sessions: `{sessions: [{id, createdAt, updatedAt, eventCount, title, agent, adopted, totalTokens, costMicros, avoidedMicros, saturated, active, promptContext}]}`. `id`, `createdAt`, `updatedAt`, `eventCount` and `active` are always present; every other field is `omitempty` — absent rather than zero, on the standing rule that an unknown value must not render as a real one. (Do not read that off the position of `active`: it sits second-to-last, between two `omitempty` fields.) **`title` is a suggestion, not an identifier:** the proxy derives it from the session's own events (a `/rename`, else a `<user_query>`, else ordinary user prose, with `<system-reminder>` blocks excised), so it is a display convenience and nothing addresses a session by it. Absent when nothing in the events named it. **Folded at append time and FIRST-WINS, except that a `/rename` always overrides** — so ordinary conversation does not re-title a session on every turn, and a `/rename` survives eviction of the event that carried it. agentop reads this field as a FALLBACK: its TITLE column prefers a harvested Claude Code transcript title and uses the served title only for a session the harvest cannot name. That precedence is fixed rather than a judgement about which string is better — both sides rank candidates their own way and do not agree on every session. That is the case worth having: an agent with no transcript tree on the operator's disk still routes through the proxy, so a row that used to render blank now has a name. agentop deliberately still treats such a row as unnamed for its own re-harvest backoff, so a served title does not stop it looking for a harvested one. **`?archived=true` adds history** where the proxy runs a session archive (`session.archive`, laptop only — see the chatty-traffic gotcha): every session the archive holds and memory does not, marked `resident: false`, plus a top-level `archive` object — `{bytes, maxBytes, retentionDays}` and, only when nonzero, `droppedEvents`, `writeErrors`, `droppedRenames`, `paused`, any of which means the history has gaps. A session in both is listed once, as its resident row. **That row covers the session's whole history**: `eventCount`, `totalTokens`, `costMicros`, `avoidedMicros`, `currencies`, `saturated`, `title`, `agent`, `createdAt` and `promptContext` continue from the archive's fold of what came before this process's entry, each by its own rule, in the default list too — so with an archive and `session.max_events` unset (the default) a restart changes no figure, and without an archive they start over. Without an archive the parameter is ignored and no `archive` object is sent, which is how a client tells "no history here" from "no history yet". |
| `GET /v1/sessions/{id}` | `application/json` | The session's most recent events. `?limit=N` (default 500, max 2000) sets the window; `?before=<seq>` returns the page ending just before that event, so the whole session is reachable by paging backward from the tail. `totalEvents` is the session's true length and `oldestSeq` the oldest event the store still holds — both present only when this response is not the whole session, so a client can tell "this is the beginning" from "there is more behind me" without a second request. 404 if unknown/expired. **One response is still not a full snapshot:** with `session.max_events` unset a session can hold thousands of events, and one real session's whole history encoded to 1.1GB — 17s to write, against clients that time out in 10. That cap is why `before` exists — until it did, a session past 2000 events had a beginning no request could reach at any limit, while still costing memory. The response is written one event at a time rather than encoded whole, so serving it costs the proxy heap proportional to one event; see the chatty-traffic gotcha below. **With a session archive, a page continues from disk** below the oldest event memory holds: a session resumed after a restart pages back through its whole history, one evicted from memory is served instead of 404'd, and `totalEvents`/`oldestSeq` describe the merged history. `GET /v1/sessions/{id}/events/{seq}` falls back to the archive the same way, and answers 404 rather than a neighbour when neither holds that seq. |
| `DELETE /v1/sessions` | `application/json` | Clear every session: the store, and the session archive where one runs. **The cost ledger is kept** — spend totals, no content. Answers `{sessions, archivedSessions, bytes}`; a failed disk half is a 500 adding `archiveError`, with memory cleared regardless (the store clears under its lock before the archive is waited on, up to 10s). **The one state-changing route on this unauthenticated API, so it is guarded against a browser**, each guard pinned by its own test in `core/sessionapi/clear_test.go`: DELETE only (a cross-site DELETE needs a preflight this server never approves; a simple cross-site POST needs none); a loopback `Host` — exactly `localhost`, `127.0.0.1` or `::1`, which defeats DNS rebinding; no `Origin` header; and `WithClearAllowed`, which `cmd/cortex` sets from `listener.bind_loopback_only`. A refusal is a 403 `{"error": ...}`. agentop's `X` drives it. |
| `GET /v1/events` | `text/event-stream` | SSE stream of new events. Optional `?session=<id>` filters to one session. Heartbeat every 30s. |
Expand Down Expand Up @@ -1095,7 +1095,7 @@ resulting `/shared/client-id.txt` and `/shared/client-secret.txt`.

**And the client is the other end of the same problem.** Measured on a laptop, `agentop` sat at 1.64GB against the proxy's 1.03GB — larger than the process it was watching, and the only one of the two still climbing. It made both mistakes the server had just stopped making: it decoded each snapshot as one document (`json.Decoder` only bounds its buffer between *values*, and a whole snapshot is one value), and it kept every string the server's encoder had expanded back out of the store's sharing. Both are fixed in `cmd/agentop/apiclient/snapshot.go`: the response is decoded one event at a time and fed through the same `session.Interner` the store uses, exported for the purpose rather than reimplemented. On a 23.5MB document, `+32.0MB HeapSys` and 26.49MB retained became +0.0MB and 2.31MB (`BenchmarkDecodeSessionViewHeap`). When a memory figure here looks wrong, measure **both** processes — the proxy has not been the larger one for a while.

**The session archive is disk, not memory** (`core/session/archive`, `session.archive`, on by default for a local install bound to loopback only and off everywhere else — `config.ArchiveRunsOnLocalInstall` says why; `DELETE /v1/sessions` erases it with the store). It relieves the store of nothing: every bound above applies unchanged with it on. What it changes is what eviction *costs* — an evicted session is still listed under `?archived=true` and paged from disk, and a `max_events` trim drops nothing a reader can no longer reach — so with it on, a cap or a `ttl` stops being a trade against history. It records through the same `Recorder` hook as the cost pipeline, off the request path: a burst the writer cannot keep up with is dropped from the archive and counted, never from memory. On disk it interns the same five fields the store does, one zstd segment at a time, and it is bounded by `retention_days` (30) and `max_bytes` (2 GiB), not by anything here. `docs/laptop-service.md` has the user-facing version, including what the files contain.
**The session archive is disk, not memory** (`core/session/archive`, `session.archive`, on by default for a local install bound to loopback only and off everywhere else — `config.ArchiveRunsOnLocalInstall` says why; `DELETE /v1/sessions` erases it with the store). It relieves the store of nothing: every bound above applies unchanged with it on. What it changes is what eviction *costs* — an evicted session is still listed under `?archived=true` and paged from disk, a resumed one lists its whole history, and a `max_events` trim drops nothing a reader can no longer reach — so with it on, a cap or a `ttl` stops being a trade against history. It records through the same `Recorder` hook as the cost pipeline, off the request path: a burst the writer cannot keep up with is dropped from the archive and counted, never from memory. On disk it interns the same five fields the store does, one zstd segment at a time, and it is bounded by `retention_days` (30) and `max_bytes` (2 GiB), not by anything here. `docs/laptop-service.md` has the user-facing version, including what the files contain. At startup, before any listener, `cmd/cortex` replays the archive's last six hours into the usage ring — `usage.Aggregator.Replay`, never through the store, so the ledger is not written twice — within `usageReplayBudget` (5s), or not at all.

Two layered defenses keep the inbound A2A user intent visible to IBAC even when an agent generates dozens of outbound events per turn:

Expand Down
37 changes: 15 additions & 22 deletions cmd/agentop/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -757,17 +757,13 @@ agentop is for, and the other three are surfaces you visit and leave.
that column also carried, for sessions the server has forgotten but agentop
still holds events for, moved into `UPDATED`.

`UPDATED` also carries `archived`, but only with history on (`H`): a session
the proxy's memory no longer holds and its session archive still does — after
a restart, say, or once `session.max_sessions` evicted it. The two markers
answer the same question from opposite ends: `cached` is agentop remembering
what the proxy forgot, `archived` is the proxy remembering it on disk. Enter
opens an archived row like any other; its events are read back from disk a page
at a time — the latest on Enter, older ones with `o`, as for a resident
session — so a long one costs no more to open than a short one.
A proxy without an archive — any cluster sidecar, and a laptop that has turned
it off — answers `H` with the live list and a one-line note saying so. `X`
erases all of it, after asking; see the keybindings below.
With a session archive — a local install — the table lists every session the
archive holds beside the ones in memory, so neither a restart nor
`session.max_sessions` eviction takes a row off it. Enter opens an old session
like any other; its events are read back from disk a page at a time — the
latest on Enter, older ones with `o`, as for a resident session — so a long one
costs no more to open than a short one. `X` erases all of it, after asking; see
the keybindings below.

**Every figure in this table is a per-session total**, summed over that
session's whole history rather than over a clock window — which is why its
Expand All @@ -776,11 +772,13 @@ agentop is for, and the other three are surfaces you visit and leave.
is a check on the other.

The table is also **not** a longer span than the band, which is the reading
worth heading off. The session store is in memory, so a session's figures only
reach back as far as the current proxy process — on a freshly restarted proxy
the whole COST column can sum to less than `TODAY`, because `TODAY` comes from
the durable cost ledger and survives restarts. `[?]` states both facts; the
title deliberately does not, since no single span is true of every row.
worth heading off. With a session archive (a local install) a session's figures
reach back through restarts to its first archived event. Without one — any
cluster sidecar — the store is in memory, so they reach back only as far as the
current proxy process, and on a freshly restarted proxy the whole COST column
can sum to less than `TODAY`, because `TODAY` comes from the durable cost ledger
and survives restarts. `[?]` states both facts; the title deliberately does
not, since no single span is true of every row.

Rendered at 100 columns, where every column has its declared width; 93 is the
narrowest terminal that carries them all at once:
Expand All @@ -797,13 +795,9 @@ agentop is for, and the other three are surfaces you visit and leave.
default 1h ago 8 — — — —

● connected 2.1 events/sec feedback: https://github.com/rossoctl/cortex/issues/new/choose
… [H] history [u] usage [$] spend [/] filter [p] pause [P] pipeline [?] keys [q] quit
[↑↓] nav [↵] drill [u] usage [$] spend [/] filter [p] pause [P] pipeline [?] keys [q] quit
```

The footer is 111 columns whole, so at 100 it has already given up `[↑↓] nav`
and `[↵] drill` — the two most guessable keys on the line, and the ones placed
first so they are the ones to go. `[u]` and `[$]` hold to 80.

The two money columns are dropped entirely on a terminal too narrow to show a
sub-cent charge honestly — below 93 columns — rather than rounded to `$0.00`
or blanked. A charge under a cent reads `<$0.01`. `TITLE` is fitted first and
Expand Down Expand Up @@ -1258,7 +1252,6 @@ Layered on top of all of them:
| `Esc` / `←` / `h` | detail, events | back out |
| `Esc` | sessions | back to the agents picker when the list was reached by picking an agent there; otherwise (picker mode) tear down port-forward and back to pods. Sessions and the agents picker above it are the only panes that tear down — every key-opened surface returns to its caller instead |
| `/` | sessions, events | filter (substring match; Enter commits and saves, Esc cancels the edit and saves nothing; clear the box and press Enter to remove a saved filter) |
| `H` | sessions | toggle history: also list the sessions the proxy's session archive holds and memory no longer does, marked `archived` in `UPDATED`. Capital because `h` backs out. Ahead of `[u]` in the footer so an 80-column cut drops it before the cost keys |
| `X` | sessions | clear all history: every session this Cortex holds, in memory and on disk. Asks first, with the count and the size on disk; `y` erases, `n`/`esc` keeps. The cost ledger is kept. Refused — with the proxy's reason — anywhere but a loopback-only laptop install. Not in the footer, like `A`: a destructive key should not be advertised on the always-visible line |
| `s` | events | toggle skip-row visibility (default: hidden; the events footer shows the hidden count) |
| `c` | events | open the column picker (`↑↓`/`jk` move, `space`/`x` toggle, `s` sort, `r` reset, `Esc`/`Enter`/`c` close); the selection and sort are saved on close |
Expand Down
4 changes: 2 additions & 2 deletions cmd/agentop/cmd_service.go
Original file line number Diff line number Diff line change
Expand Up @@ -1006,8 +1006,8 @@ func reportHistoryCleared(wasServing, archived bool, stdout io.Writer) {
return
}
if archived {
fmt.Fprintln(stdout, " The proxy's memory is cleared, but its session archive is not: H in")
fmt.Fprintln(stdout, " agentop lists the sessions it holds.")
fmt.Fprintln(stdout, " The proxy's memory is cleared, but its session archive is not: agentop")
fmt.Fprintln(stdout, " still lists every session it holds.")
return
}
fmt.Fprintln(stdout, " Captured session history is cleared: the store is in memory, so any")
Expand Down
4 changes: 2 additions & 2 deletions cmd/agentop/cmd_service_characterize_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -528,8 +528,8 @@ func TestCharacterize_ServiceInstall_OverAnArchivingProxy(t *testing.T) {
f.Close()

run := sc.install(t, true, false)
const want = " The proxy's memory is cleared, but its session archive is not: H in\n" +
" agentop lists the sessions it holds.\n"
const want = " The proxy's memory is cleared, but its session archive is not: agentop\n" +
" still lists every session it holds.\n"
if run.code != 0 || !strings.HasSuffix(run.out, want) || strings.Contains(run.out, "history is cleared") {
t.Fatalf("exit %d, stdout:\n%s\nwant it to end with:\n%s", run.code, run.out, want)
}
Expand Down
Loading
Loading