Research: advisory freshness and review scoring for long-term governed knowledge

[ajdelaguila]

Source signal

The OpenSearch memory-retention RFC discussion sharpened a point that applies to Data Olympus too:

• working/session/history cleanup can use retention controls such as count and age;
• long-term governed knowledge needs freshness, revalidation, supersession, promotion, and scoring signals more than simple deletion;
• any score should remain advisory until a retrieval or promotion policy interprets it.

Related closed work:

• Research: add freshness and validity metadata for facts without named successors #107 added freshness and validity metadata for facts without named successors.
• Research: expose authority and provenance state for retrieved and proposed knowledge #109 exposed authority/provenance state for retrieved and proposed knowledge.
• Research: validate typed lifecycle relationships as executable retrieval policy #110 validated lifecycle relationships as executable retrieval policy.

This issue is a follow-up, not a replacement for those slices.

Problem

Data Olympus can decide whether a document is in force, but operators also need a way to identify long-term knowledge that should be refreshed, reverified, promoted, or reviewed before it becomes stale enough to hurt retrieval quality.

A binary in-force filter is necessary but not always enough for long-lived KBs.

Proposed research / design

Evaluate an advisory freshness/scoring layer for long-term governed knowledge. Candidate signals:

• days since last_verified_at;
• approaching or past valid_until;
• supersession depth or conflict links;
• repeated retrieval without recent verification;
• repeated agent proposals touching the same concept;
• source evidence age;
• operator-confirmed “still current” refresh events.

Possible surfaces:

• kb_curate recommendation: “review / refresh / verify this guidance”;
• compact search metadata: freshness_score or review_due as advisory only;
• maintenance ledger entries for soon-stale high-value docs;
• optional sorting boost/downrank for stale-but-still-in-force documents, behind a mode flag.

Acceptance criteria

• Scoring never promotes or demotes authority by itself.
• The score is clearly advisory and separate from status / in_force.
• Operators can see why a document is review-due.
• Retrieval can optionally expose the signal without changing default in-force semantics.
• Documentation explains the difference between retention, freshness, and authority.

Non-goals

• Do not add automatic deletion for long-term project knowledge.
• Do not make access counters write on every read.
• Do not introduce an authority_state enum parallel to status.

[ajdelaguila]

Cross-repo datapoint for the freshness and scoring design here, from the OKF maintenance-signals discussion in GoogleCloudPlatform/knowledge-catalog#158.

A maintainer on that thread (andrewcrenshaw, describing the remember framework) proposed a validity window with two bounds rather than one. Alongside valid_until he carries an opening bound (valid_from) for facts written before they take effect: a policy dated forward, a price filed in advance. He also drew a separation worth flagging here: the interval over which a fact is asserted to hold in the world is a different thing from when the record was written or last checked. In our current model valid_until is the closing bound of the first, and last_verified records the second as verification metadata, so those two are already distinct fields, but the opening bound is missing.

That gap has a concrete consequence for the advisory layer proposed here. The current candidate signals key off valid_until and last_verified, so a forward-dated fact that has not yet reached a valid_from would look indistinguishable from an in-force one, or worse could be scored review-due before it ever takes effect. "Not yet in force" would be a third advisory state next to "in force" and "past valid_until," and it should suppress review pressure rather than raise it.

Two research questions this suggests for the scope above:

• Should the scoring layer read an opening bound (valid_from) and treat pre-effective facts as their own advisory state, distinct from in-force and from past-valid_until? The current model (from Research: add freshness and validity metadata for facts without named successors #107) carries valid_until and last_verified but no opening bound.
• Should the validity window and the verification/write signals be scored on separate axes rather than folded into one freshness_score, given they answer different questions? Whether that separation earns the added surface is the open question, not a settled classification.

The same shape recurs in the capture-event provenance research (#141): whatever the freshness layer reads has to agree with when and how the envelope attaches. Recording the cross-repo datapoint here so the design can weigh the opening bound and the validity-versus-verification separation before the scoring surface lands.