Skip to content

An end the timeline derived cites the fact that closed the row, and a corrected date says which end a person changed - #986

Merged
WaylandYang merged 5 commits into
deeplethe:devfrom
Maya-Kid:fix/derived-end-cites-what-closed-it
Sep 28, 2026
Merged

WaylandYang merged 5 commits into
deeplethe:devfrom
Maya-Kid:fix/derived-end-cites-what-closed-it

Conversation

@Maya-Kid

Copy link
Copy Markdown
Contributor

Closes #970: step 2 as decided there, on top of #976 (step 1), with the three follow-ups from #976's review.

Problem

Change

The fact that closed the row.

  • A derived end names the row that closed it and cites that row's passage through the same cite: (2024-07-05 → 2025-09-01, closed when Zhou Qi took over [3]). [3] opens the leadership update, which states 2025-09-01.
  • The row is found as the timeline found it (temporal::desired_ends):
    • the same predicate and timeline (by subject for a functional property, by object for an inverse functional one);
    • a start equal to this row's end, and a different value;
    • neither an unanchored start nor described evidence;
    • the lowest id when several fit, as the engine orders them.
  • It is read with the facts: a LEFT JOIN LATERAL in graph::entity_detail and paths::details, only for end_derived rows, on the record axis as the facts are read.
  • EntityFact and PathEdge gain closed_by_id, closed_by (the successor's name) and closed_by_value (a new literal value). The line renders them with literal_text.
  • Evidence is still one first_live_evidence query: Evidence::load takes the shown facts and their closers together. The closer's chunk gets its number where the line prints it, so one chunk keeps one number: Zhou Qi's own line and Li Si's closer share it.
  • With A graph citation's source entry carries the sentence the fact was read from, and the preview and the document page mark it #977, the entry carries the closer's sentence as well, so [3] opens "Zhou Qi leads Project Aurora from 2025-09-01." with that sentence marked.
  • A derived end whose successor has no start (ended, date unknown) has no row to cite and keeps end derived.
  • MCP has no source list and reads closed when Zhou Qi took over without a number.

Which end a person changed, written with the row. Migration 0099 adds facts.corrected_ends: start, end or both, NULL when no person changed the interval.

  • temporal::correct_interval writes it in the correction's own transaction. It compares the old row with the new interval: an end whose value or precision changed is marked, joined with the old row's marks, so a start and then an end make both.
  • A person's interval is theirs. The correction clears end_derived, and the timeline no longer redraws that end: a_revert_keeps_the_interval_a_person_corrected pins this. So a derived end the person kept is marked as theirs too. An end the passage stated and the person left alone stays unmarked, and stays the passage's.
  • Every rewrite carries the mark as it carries end_derived: rewrite_end_tx (a close or a new end), rehome_tx and graph::adopt.
  • An end the timeline derives later, on a row a person had left open, is not reported as the person's: the read drops end when end_derived is set.
  • The reads take the column instead of walking the supersedes chain for an audit entry. The flag no longer depends on the audit.
  • Backfill. Rows corrected before the column get it from their audit entries.
    • Since A wrong date can be corrected without deleting the document #311, detail records the interval before (from) and after (to). The row the entry targets is still in the table, so its end_derived is known.
    • Each entry marks the corrected row and every rewrite below it, several entries joined.
    • An entry without the two intervals marks both ends.
    • Only empty rows are filled, and running it again changes nothing.
  • An errata revision writes a new row rather than a rewrite. As with Graph fact lines say when an end was derived by the timeline or an interval corrected by a person #976's audit read, it starts unmarked.

The line and rule 4.

  • The mark names the end: (2023-02-01 → 2024-07-05, start corrected: The charter date was the approval date); end corrected; corrected for both.
  • The note is still the latest fact.time_corrected entry's on the chain. It is read only for marked rows, through Graph fact lines say when an end was derived by the timeline or an interval corrected by a person #976's partial index. It cites nothing, since it is a person's statement and not a passage, and it is flattened to one line and cut at 120 characters.
  • Rule 4 now says:
    • an end written closed when X took over [m] was set by the timeline when that fact began, so cite [m] for it, never the line's own [n];
    • start corrected or end corrected means that date was changed by a person, and corrected means both;
    • a derived or changed date is never attributed to the line's passage;
    • a date the line does not mark still is the passage's.

Docs. docs/design/chat-and-mcp.md follows.

CLI. CURRENT_SCHEMA_VERSION is 89: the 88 files on dev plus 0099. #975 holds 0098, so the numbers do not collide; whichever of the two lands second raises the constant to 90.

Verification

Run in containers on a Linux host: Rust 1.98.1 and PostgreSQL 16 with pgvector.

New tests

  • time_text:
    • a closer with and without its number;
    • a note flattened, cut, or empty;
    • the three corrected marks;
    • a start mark beside closed when ….
  • Store, a_fact_says_where_its_dates_came_from:
    • Li Si's row is closed by Zhou Qi's, a functional employer by the next employer, and, on the record axis before the close, by nothing;
    • Zhang San's start correction marks start, still after a close by hand;
    • Li Si's start is corrected with no audit entry and is still marked; his derived end, kept by the person, makes it both;
    • Zhang San's end corrected as well makes both, with the latest note.
  • Store, a_corrected_row_says_which_end_changed, its own binary, since it runs 0099 again over the whole database. It checks what correct_interval writes, then clears the column. The backfill restores:
    • start, both and end, and nothing for a row closed by hand;
    • both for an entry without the two intervals, and for a derived end the person kept;
    • it leaves a mark already set alone, and a second run changes nothing.
  • Server, a_fact_line_says_when_a_date_is_not_the_passages:
    • Li Si's line reads closed when Zhou Qi took over [k]. [k] is the update chunk, the same number as Zhou Qi's own line, and its entry carries Zhou Qi's sentence.
    • Zhang San's line reads start corrected: …, and the note is not a source.
    • Both show on path hops, and MCP prints the text without numbers.
    • Rule 4 carries the new sentences.
    • After a second correction the line reads corrected: The handover was on 1 July.

The new tests on dev

The control branch is dev plus every new test, with the new fields stubbed empty and span() as on dev, so a correction prints corrected. The new tests fail:

  • Store: the facts test fails at its first closer check, and the backfill test finds nothing written.
  • Server: the citation test and two time_text tests fail.

This branch (f9869d0)

  • The migration chain runs on a fresh database (89 files).
  • cargo test --locked -p utopia-cli passes (13), the schema guard among them.
  • The store crate passes (379, the store suite 279), and so does the server with graph_citation, time_text, chat_sources and api:: (227).
  • cargo fmt --all --check and cargo clippy --locked --workspace --all-targets -- -D warnings are clean.
  • UTOPIA_TEST_REQUIRE_PDFTOTEXT=1 cargo test --locked --workspace passes: 1256 passed, 0 failed.

Measurement

The temporal bench with DeepSeek (deepseek-chat), measured as step 1 was.

dev 1 dev 2 branch 1 branch 2
Ledger 32/37 32/37 32/37 32/37
Chat (the bench's check) 23/37 24/37 19/37 18/37
Answers that open a source 36/37 35/37 35/37 36/37
Cited clauses fully found 28/50 33/52 34/53 35/58
Misses whose only missing items are a derived end or a corrected date 13 9 11 10
Li Si's derived end cited to a passage that states it 2/6 1/6 4/8 2/6
Answers that say a date was derived or corrected 11 10 16 14

The derived end. With a number of its own, DeepSeek cites it in some answers: "He held the role until 2025-09-01, when Zhou Qi took over [2]", where [2] opens the leadership update. Li Si's end is cited to a passage that states it in 6 of 14 clauses, against 3 of 12 on dev. The spread between runs is as wide as that difference.

The corrected dates.

  • Most remaining misses are Zhang San's corrected dates cited to the charter. A corrected date has no passage to cite, by design, and the model still puts the charter's number after it, as on dev.
  • In this base the correction also pinned the end the timeline had derived, so the line reads corrected for both ends. The one-end wording is covered by the tests but not exercised by this corpus.
  • The note reaches the answers: "the charter date was the approval date, and he actually took over on 1 February".

The chat count. The lower count on the branch comes from the bench's name check.

  • The four Aurora "who led it then" questions pass on dev and fail on the branch in both runs.
  • Every one of those answers names the right lead, and also the lead before or after, which the bench counts as a wrong name: "Li Si led Project Aurora in August 2024 — their tenure as lead ran from 2024-07-05 until 2025-09-01, when Zhou Qi took over [3][2]." The line now says who took over, and the model repeats it.
  • Three other questions fail the same way on dev in some runs (a later salary, "Li Si" beside "L. Si", the months before the project).

Maya-Kid and others added 3 commits September 27, 2026 23:46
… corrected interval carries the person's note

Signed-off-by: dada-yan <BinjunYann@gmail.com>
…row instead of read from the audit, and its fact line names that end

Signed-off-by: dada-yan <BinjunYann@gmail.com>
WaylandYang added a commit that referenced this pull request Sep 28, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>
WaylandYang and others added 2 commits September 28, 2026 12:42
…er is named without a verb that fits only people

Maintainer edits on deeplethe#986. When the end of a row is written again, by a document stating it or by
the timeline deriving one, the row no longer says a person corrected that end; the start's mark
stays. The derived end reads superseded by X: took over was right for an inverse-functional
property and wrong for a functional one or a value. Square brackets in a person's note are
replaced, since a [2] there would read as a citation. CURRENT_SCHEMA_VERSION is 90.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Writing which end a person changed on the row itself, in the same insert, is the sturdier form I hoped for after #976: a lost audit entry now loses the note and not the mark. The closer goes through the same cite, so one chunk keeps one number, the closing fact's number is registered before the line's own, and MCP carries the words without numbers.

I pushed one commit:

  • A stale mark. rewrite_end_tx copied corrected_ends as it was. After a person reopened an end and a document then stated it, the row still read end corrected, and rule 4 told the model not to attribute a date the passage does state. When the end is written again the end's mark goes and the start's stays.
  • The wording. took over is right for an inverse-functional property, where the successor is the subject. For a functional one it named the new object, so employer → Acme read as closed when Globex took over, and an attribute as closed when 35000 CNY took over. The line now reads superseded by X [m] in every case; rule 4, the design doc and the tests say the same.
  • Brackets in a note. A person's note keeps its words but [ and ] become parentheses, since see [2] on the line would read as a citation.
  • CURRENT_SCHEMA_VERSION is 90, as #975 landed with 0098.

With dev merged in: store 291, server 522, CLI 13, all passing.

On the bench: the drop in the chat check comes from answers now naming the successor, which the name check counts as a wrong name. That is the check's limit, not the answers'. If you want to tighten it, score the name the question asks for rather than any name present.

Worth a follow-up test: two successors starting on the closing date, a closer that was itself rewritten, and a closer with no live evidence. Landing it. Thanks again.

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer review: sound, landing.

@WaylandYang
WaylandYang merged commit d7756a4 into deeplethe:dev Sep 28, 2026
WaylandYang added a commit that referenced this pull request Sep 28, 2026
dev reached 90 migrations with #986; this PR adds one more and the CLI test compares the constant
with the number of files.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Wayland Yang <wayland0916@gmail.com>
@Maya-Kid
Maya-Kid deleted the fix/derived-end-cites-what-closed-it branch September 28, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A graph fact's [n] can open a passage that does not state the line's end or start

2 participants