Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,30 @@

## [Unreleased]

### A `recall_file` line range the file cannot satisfy no longer kills the server

`recall_file` with a `start_line` five or more lines past the end of the file, or one far enough
after its `end_line`, panicked on a slice out of range. The server exited (status 101) and answered
nothing after that, so one bad guess, such as a line number from before the file shrank, took every
Lumen tool away for the rest of the session.

- A range the file cannot satisfy is now an error reply (invalid params, `-32602`) that says how many
lines the file has and what was asked for. An `end_line` past the last line is clamped to it, and a
number too big for the platform's integer no longer overflows the arithmetic.
- A range the file can satisfy is answered exactly as before, byte for byte, and `start_line` 0 is
still read from the first line. Three requests that used to be answered now get the error instead,
because the answer was not what they asked for: a `start_line` one to four lines past the end (the
last lines of the file, or nothing, under the range asked for), a `start_line` slightly after the
`end_line` (a window around lines nobody named), and a range that starts at line 1 or later on an
empty file (a reply that cost tokens for a file that had none).
- A panic anywhere in a request is now caught per request: the call is answered with an error
(`-32603`), the server keeps serving, and a `request_panic` fault is recorded with the tool's name
and the file and line it panicked at. The panic message is in the error reply and on stderr but not
in the fault, since it can quote the text being read and a report holds no file contents. `lumen
report` ranks it just below a fired fail-open guard. What the call had remembered for repeat reads
is dropped, since the model never received it. `LUMEN_CAPTURE=0` keeps the fault out of the spool,
as it does for the hooks.

### A transcript deleted before the daemon read it is not a fault

The daemon opens a transcript after it has noticed it, from a watcher event or a directory
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -626,8 +626,9 @@ Claude with no way to read the file at all:
for it and the read is allowed through. Without this the hook can deadlock a session —
it blocks the built-in `Read` while the replacement is unreachable.
- **A fault record** — one JSON line appended to `faults.jsonl` beside the database when
a hook lets through a read it would have redirected, or loses an event it should have
metered. It contains the file path, its line count, what went wrong, and the session id
a hook lets through a read it would have redirected, loses an event it should have
metered, or a tool call crashes and is answered with an error. It contains the file path,
its line count, what went wrong, and the session id
— **never file contents**. `lumen report` reads these; nothing is sent anywhere unless
you explicitly file a report. Set `LUMEN_CAPTURE=0` to keep the hooks and record nothing.

Expand Down
94 changes: 85 additions & 9 deletions crates/lumen-core/src/report.rs
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,7 @@ struct Group {
/// `every_emitted_kind_is_classified` fails if that happens again.
pub const FAULT_KINDS: &[&str] = &[
"hook_fail_open",
"request_panic",
"schema_drift",
"meter_write_failed",
"ingest_failed",
Expand All @@ -302,31 +303,33 @@ pub const FAULT_KINDS: &[&str] = &[
const UNCLASSIFIED: &str = "Unclassified fault. See the table below.";

/// Where a kind this build does not know sorts: after every known one.
const CATCH_ALL_PRIORITY: u8 = 9;
const CATCH_ALL_PRIORITY: u8 = 10;

/// Sort order for the table and for picking the headline. A fired fail-open guard
/// outranks everything: it means the routing contract broke in the field.
fn kind_priority(kind: &str) -> u8 {
match kind {
"hook_fail_open" => 0,
"schema_drift" => 1,
// A crashed tool call is a bug in lumen, not a degraded saving; only a released Read outranks it.
"request_panic" => 1,
"schema_drift" => 2,
// A read that never reached the ledger. Below schema drift only because drift is
// the commonest cause of it, so the cause should head the report.
"meter_write_failed" => 2,
"meter_write_failed" => 3,
// Silent data loss: the daemon logged and continued, so the gauge is wrong and
// nothing said so. Ranks above the two that merely retry.
"ingest_failed" => 3,
"ingest_failed" => 4,
// The report itself is incomplete. Not the fault being reported, but it changes
// how much the rest of the report can be trusted.
"reporter_degraded" => 4,
"ws_restart" => 5,
"reporter_degraded" => 5,
"ws_restart" => 6,
// Costs the calls it touches their saving and their attribution, and nothing else:
// they are answered in full. Above a ranked decline, which is not a failure.
"identity_unrecorded" => 6,
"identity_unrecorded" => 7,
// The repeat-reads switch could not be read, so they are off. Costs only their saving
// and the user was not told by anything else; above a ranked decline for that reason.
"repeat_reads_config" => 7,
"ranked_decline" => 8,
"repeat_reads_config" => 8,
"ranked_decline" => 9,
_ => CATCH_ALL_PRIORITY,
}
}
Expand All @@ -338,6 +341,12 @@ fn impact(kind: &str) -> &'static str {
fail-open guard released the Read. Routing is degraded, not broken — context \
was spent that lumen was supposed to save."
}
"request_panic" => {
"A lumen tool call hit a bug and panicked. The server caught it, answered that \
call with an error and kept serving, so the model got no result for it; the \
same call is likely to fail the same way. The detail is where in the code it \
panicked."
}
"schema_drift" => {
"The database's `read_events` columns do not match the set this build expects. \
Metering rows may be dropped or written to the wrong column."
Expand Down Expand Up @@ -559,6 +568,10 @@ fn headline(top: &Group, version: &str) -> String {
files,
if files == 1 { "" } else { "s" }
),
"request_panic" => format!(
"{} panicked {}× and was answered with an error",
top.variant, top.count
),
"schema_drift" => "read_events schema drift".to_string(),
"meter_write_failed" => format!(
"metering write failed {}× ({}) — reads missing from the ledger",
Expand Down Expand Up @@ -2015,6 +2028,69 @@ mod tests {
);
}

fn a_request_that_panicked() -> Fault {
Fault {
kind: "request_panic".into(),
guard: Some("recall_file".into()),
detail: Some("panicked at crates/lumen-mcp/src/lib.rs:1125:36".into()),
..Default::default()
}
}

#[test]
fn a_panicked_request_outranks_every_kind_but_a_fired_fail_open_guard() {
let first_line = |faults: &[Fault]| {
render(faults, &test_env(), &RenderOpts::default())
.expect("non-empty")
.lines()
.next()
.unwrap_or_default()
.to_string()
};

for kind in FAULT_KINDS
.iter()
.filter(|k| !matches!(**k, "hook_fail_open" | "request_panic"))
{
let noisier = Fault {
kind: (*kind).into(),
count: 500,
..Default::default()
};
let first = first_line(&[noisier, a_request_that_panicked()]);
assert!(
first.contains("recall_file panicked"),
"a panic should head the report over {kind}, got: {first}"
);
}

let released = Fault {
kind: "hook_fail_open".into(),
guard: Some("lumen_mcp_missing".into()),
..Default::default()
};
let first = first_line(&[a_request_that_panicked(), released]);
assert!(first.contains("lumen mcp missing"), "got: {first}");
}

#[test]
fn the_headline_for_a_panicked_request_names_the_tool_and_how_often() {
let faults = vec![Fault {
count: 3,
..a_request_that_panicked()
}];
let body = render(&faults, &test_env(), &RenderOpts::default()).expect("non-empty");
let title = title_from(&body);
assert_eq!(
title,
"lumen 1.4.0 — recall_file panicked 3× and was answered with an error"
);
assert!(
body.contains("panicked at crates/lumen-mcp/src/lib.rs:1125:36"),
"where it panicked is the detail"
);
}

#[test]
fn single_occurrence_and_single_file_read_naturally() {
let faults = vec![Fault {
Expand Down
Loading