Skip to content

fix(framework): keep Codex LLM spans non-empty during backgrounded commands - #382

Merged
Rodriguespn merged 2 commits into
mainfrom
Rodriguespn/read-slack-thread-braintrust
Oct 9, 2026
Merged

Rodriguespn merged 2 commits into
mainfrom
Rodriguespn/read-slack-thread-braintrust

Conversation

@Rodriguespn

@Rodriguespn Rodriguespn commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Fixes the empty/0s LLM spans Matt reported in Braintrust traces (Slack thread). Since #356, Codex tool spans end at item_completed, so a backgrounded command (supabase stack start … &) can end after later model requests. logTranscript started each LLM span at min(latestTime, at), which collapsed to start == end whenever an earlier tool outlived the request.

An LLM span now starts at the latest span edge at or before its own timestamp. Start times are unchanged when nothing overruns.

Verification

  • New regression test in upload-braintrust.test.ts. It fails on main and passes with the fix.
  • Replayed all 15 transcripts from run 37786343131 through the old and new logTranscript: zero-length LLM spans went from 31 to 0, and no durations are negative.
  • Validation run → Braintrust experiment: 0 zero-length LLM spans. A request made during a backgrounded command now spans 133.48→137.04s. Before the fix it would have been empty.

@Rodriguespn
Rodriguespn requested a review from a team as a code owner October 8, 2026 14:31
@vercel

vercel Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
evals Ignored Ignored Preview Oct 8, 2026 2:31pm UTC

Request Review

@mattrossman mattrossman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

@Rodriguespn
Rodriguespn merged commit b608a14 into main Oct 9, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants