Skip to content

fix: ignore ingest items without canonical impact#267

Merged
duncanleo merged 1 commit into
mainfrom
codex/harden-ingest-irrelevant-evidence
May 26, 2026
Merged

fix: ignore ingest items without canonical impact#267
duncanleo merged 1 commit into
mainfrom
codex/harden-ingest-irrelevant-evidence

Conversation

@duncanleo

@duncanleo duncanleo commented May 26, 2026

Copy link
Copy Markdown
Member

Summary

  • Add an MRT/LRT-only domain gate to the triage prompt so bus-only incidents are classified as irrelevant.
  • Skip creating new canonical issues when extraction returns no impact claims.
  • Add deterministic ingest coverage for the Bus Service 26 failure mode and a paid-eval fixture for bus-only diversion triage.

Root Cause

The latest automated ingest PR was created from a bus-only Mastodon alert. Triage marked it as a new disruption, extraction returned claims: [], and ingestion still persisted a hollow issue with empty impact data.

Validation

  • npm run test:triage
  • npm run build:triage
  • npm run check
  • npm run data:validate

Summary by CodeRabbit

Release Notes

  • New Features

    • Triage now focuses on Singapore MRT/LRT rail operations and station facilities only, filtering irrelevant bus and road incidents.
  • Bug Fixes

    • System no longer creates empty issues when no impact claims are extracted.
  • Tests

    • Added test coverage for filtering irrelevant bus service incidents.
    • Added test coverage for claim extraction with no impact claims.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 26, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f8638cd3-13b0-4bd4-bbe0-4540f16c4c70

📥 Commits

Reviewing files that changed from the base of the PR and between 8cd4772 and 06f8c11.

📒 Files selected for processing (4)
  • packages/triage/src/llm/functions/triageNewEvidence/eval.test.ts
  • packages/triage/src/llm/functions/triageNewEvidence/prompt.ts
  • packages/triage/src/util/ingestContent/index.test.ts
  • packages/triage/src/util/ingestContent/index.ts

📝 Walkthrough

Walkthrough

The PR adds domain-gate filtering to the triage system prompt to exclude bus-only and road-related incidents, adds an eval test case to verify this filtering, and updates ingestContent to gracefully skip persistence when claim extraction yields no results.

Changes

Irrelevant content filtering and handling

Layer / File(s) Summary
Triage domain filtering rules and evaluation
packages/triage/src/llm/functions/triageNewEvidence/prompt.ts, packages/triage/src/llm/functions/triageNewEvidence/eval.test.ts
System prompt adds a "Domain Gate" section that constrains scope to Singapore MRT/LRT operations and explicitly marks bus-only, bus-stop, and road/private-vehicle incidents as irrelevant unless they impact train or station facilities. Eval test case verifies that a bus-only service diversion correctly produces kind: 'irrelevant-content'.
IngestContent empty-claims handling
packages/triage/src/util/ingestContent/index.ts, packages/triage/src/util/ingestContent/index.test.ts
IngestContent adds an early-exit guard that returns null when triage indicates a new issue but claim extraction yields an empty list, avoiding unnecessary downstream processing. Test suite with fixture setup, mocks, and teardown verifies that ingestContent skips title/translation generation and produces no issue output when claims are empty.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • foldaway/mrtdown-data#245: Both PRs modify the triageNewEvidence/buildSystemPrompt() system-prompt guidance in packages/triage/src/llm/functions/triageNewEvidence/prompt.ts, affecting triage classification behavior.

Poem

🐰 A rabbit hops through triage gates,
Filtering what's in, what waits—
Bus-bound news now stays outside,
While MRT claims take the ride.
Empty claims? Skip right past,
Making triage lean and fast! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix: ignore ingest items without canonical impact' directly matches the main change: adding an early-exit condition to skip persistence when no impact claims are extracted, preventing empty issue creation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/harden-ingest-irrelevant-evidence

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@duncanleo duncanleo marked this pull request as ready for review May 26, 2026 15:08
@duncanleo duncanleo merged commit a426384 into main May 26, 2026
5 of 9 checks passed
@duncanleo duncanleo deleted the codex/harden-ingest-irrelevant-evidence branch May 26, 2026 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant