Repository navigation
feat(pipecat): trace GPT-Live calls and record session audio - #1136
Conversation
2d403a8 to
68d6e0a
Compare
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
> 🤖 **This text was AI-generated.** Pipecat connects directly to GPT-Live, so OpenAI SDK instrumentation cannot capture its model execution. Add transcripts, recorded audio, usage, backend responses, and tool execution to the Pipecat trace: ```text session ├─ pipecat.live llm ├─ user_turn / agent_turn │ └─ transcript_received ├─ user_speech / agent_speech audio selections └─ llm_response task ├─ pipecat.live.backend_response llm └─ tool execution ``` Enable tracing and recording with `setup_pipecat(capture_audio_attachments=True)`. Framework frames define transcript groups; observed IDs define tool relationships. Preserve partial output on cancellation and keep model usage on its owning LLM span. Supports one Live session per pipeline with hosted Responses delegation.
68d6e0a to
4c220d4
Compare
> 🤖 **This text was AI-generated.** Live used a separate OpenAI usage parser that dropped reported cache-write usage: ```text cache_write_tokens -> omitted ``` Reuse the existing OpenAI parser to preserve the standard metric: ```text cache_write_tokens -> prompt_cache_creation_tokens ``` Keep Live hook compatibility checks in the patcher applicability gate and let the shared composite handle patch state and idempotency. Cover either missing hook through the existing parametrized diagnostic test. Move the caller-context assertion outside pytest.raises so it runs after recognition errors and cancellation.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b9e391dd0e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| if s.live is not None: | ||
| s.live.tool_finished(frame.tool_call_id) |
There was a problem hiding this comment.
Clear pending submission state when a Live tool is cancelled
When a Live tool is cancelled before submitting its function_call_output, this callback only clears pending_tool_spans through tool_finished(). The call remains in pending_calls, which is cleared exclusively by tool_result_submitted(), so _finish_if_ready() cannot close the response even after backend inference and event processing finish. Consequently, the llm_response span stays open for the remainder of the conversation and is eventually marked incomplete at shutdown, inflating its duration. Handle cancellation separately so it also releases the submission requirement for the cancelled call.
Useful? React with 👍 / 👎.
> 🤖 **This text was AI-generated.** Several tests drove private trackers and hand-built frame timelines: ```text test -> tracker methods / observer callbacks ``` Remove those tests and their shared fixture. Keep the cassette-backed real pipeline coverage and extend the voice session assertion to check playback text against the synthesized text: ```text test -> real pipeline -> recorded provider -> exported spans ``` The exact synthetic timing cases are no longer checked independently. Keep the narrow span-export failure injection in its cassette-backed pipeline test because provider recordings cannot trigger an instrumentation failure. Remove the version-specific Live compatibility warning and its diagnostic cases. Incompatible or absent hooks now skip quietly, while the existing compatibility gate still requires both hooks.
> 🤖 **This text was AI-generated.** Live lifecycle rules were spread across independent flags and parallel maps. Delegation IDs accumulated for the session, and a retained tracker kept its conversation buffers and routing history after shutdown: ```text closed / model_closed llm_finished / processing_finished ``` Represent session and response lifecycles with explicit phases: ```text session: active -> draining -> closed response: streaming -> processing -> waiting for tools -> removed ``` Keep each turn with its transcript and each pending tool's two completion signals in one record. Combine delegation routing into one recent-history cache using the existing retention limit, keeping active routes pinned. Clear tracker-owned history and routing caches when pipeline cleanup ends. Extend the existing recorded pipeline cases to check cleanup while retaining the tracker. Full conversation transcripts and unfinished work still consume memory until completion or shutdown; captured content is not truncated.
> 🤖 **This text was AI-generated.** The follow-up commits on #1136 exposed gaps in our guidance for agents writing SDK integrations. This PR adds a pre-review check in `AGENTS.md` and documents the expected practices in the SDK integrations skill. - Reuse the underlying provider’s usage parser and check the recorded metrics. - Gate optional hooks together, skip incompatible hooks quietly, and rely on the shared patcher for idempotency. - Run real framework pipelines with recorded provider traffic and assert on exported spans. Check caller context after exceptions and cancellation. - Make session and response phases explicit, bound routing history, and verify that teardown clears tracker-owned state. The skill validator, `git diff --check`, and commit hooks passed.
Adds GPT-Live support to Braintrust's Pipecat integration. Capture transcripts, user and agent audio, backend model calls, and tool execution in one trace. Usage metrics distinguish the Live model's billed duration from backend token usage, helping explain which models did the work and estimate the call's cost.
What it looks like
Usage
With
BRAINTRUST_API_KEYset, call setup before creating your Pipecat worker:Install
braintrust[audio]for Opus/Ogg recordings. Recording can also be enabled separately withcapture_user_audio_attachmentsandcapture_agent_audio_attachments.Supports one
OpenAILiveLLMServicesession per pipeline with hosted Responses delegation. Client delegation and session resets are not covered.Design