fix(citation): give fetch results citation labels - #1045
Conversation
Fetch results carried no label, so a turn that only fetched a URL had a mandatory citation and nothing to cite, and the model invented S<n> labels that rendered as dead anchors. Search and fetch now share one label allocator per turn, fetch labels are counted when seeding the next turn and resolved in the citation map, and the prompts say fetch results carry labels too. Fixes #1043
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 911f8947e0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - Prioritize recency when relevant and reference dates | ||
| - Your FIRST action for informational questions without URLs MUST be the \`search\` tool. Do not produce the final answer until at least one search has completed in this turn | ||
| - Citation integrity: Each search result carries a \`label\` field. Cite that label exactly as it appears on the result you used and never invent one | ||
| - Citation integrity: Each search or fetch result carries a \`label\` field. Cite that label exactly as it appears on the result you used and never invent one |
There was a problem hiding this comment.
When the adaptive answer evaluation follows its Search → Fetch flow, run-answer-task.ts uses this prompt with createFixtureFetchTool, but that fixture returns the original unlabelled SearchFixture objects from evals/tasks/fixture-tools.ts. The model is therefore told that every fetched result has a label and must cite it exactly, while the eval tool supplies none, encouraging invented or missing citations and making experiment behavior diverge from production; stamp fixture fetch results with the same allocator/labels as the production fetch tool.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in the latest push: the fixture fetch tool now returns results carrying the same label the fixture search gave that document, so a citation of the fetched page resolves in the eval as in production.
This branch was successfully deployed
1 participant
Problem
Answers built only from a
fetchtool call cite labels that do not exist (for example[1](#S0)). No result in the conversation carries such a label, so the citation renders as a dead in-page anchor instead of linking to the fetched page.Root cause
Citation labels were stamped only on search results.
lib/tools/fetch.tsreturned results withtitle,contentandurlbut nolabelon every path (regular, Jina Reader, Tavily extract), andnextCitationLabelNumber/extractCitationMapsonly walkedtool-searchparts. The prompts make citations mandatory and describe the target only as the search result'slabel, so a turn that fetched a user-provided URL had nothing to cite and the model guessed a label in theS<n>shape.Fix
lib/utils/citation.ts: addcreateCitationLabelAllocator, a synchronous block allocator shared by the tools of one turn.nextCitationLabelNumberandextractCitationMapsnow includetool-fetchoutputs, so fetch labels seed later turns and resolve to the fetched URL (label map only, no toolCallId map for fetch). The existing ambiguous-label handling applies to labels from either tool.lib/tools/fetch.ts:createFetchTool({ labelAllocator })stamps labels on fetched results after retrieval succeeds.fetchToolis still exported for the type definitions.lib/tools/search.ts: accepts the shared allocator, falling back to one built fromlabelSeed.lib/agents/researcher.ts: one allocator fromcitationLabelSeedis passed to both search and fetch, so labels stay unique when a turn mixes them.label" now reads "Each search or fetch result carries alabel".Fixes #1043
Verification
nextCitationLabelNumbercounts fetch labels; a fetch label resolves throughprocessCitationsto the fetched URL; the researcher passes one allocator to both tools.bun lint,bun typecheck,bun format:check,bun run build,bun run testall pass.