build(deps): bump inspect-ai from 0.3.263 to 0.3.265

[dependabot]

Bumps inspect-ai from 0.3.263 to 0.3.265.

Changelog

Sourced from inspect-ai's changelog.

0.3.265 (17 September 2026)

• Agent Bridge: Bridged host tools are no longer denied under an approval policy when the sandboxed agent presents them to its model under a different name.

0.3.264 (16 September 2026)

• Agent Bridge: Sandboxed agents using the Responses API no longer stall after a single model turn when the model calls a tool; function_call and custom_tool_call output items now carry a non-null item id, and streamed custom tool calls now report completed status so client SDKs dispatch them.
• Bugfix: Closing cached S3 sessions after an eval no longer leaves s3fs to close them again at garbage collection, which raised a stray AssertionError: Session was never entered in unrelated code.
• Bugfix: Task failures no longer report an internal "no running event loop" error in place of the original exception.
• Anthropic: cache_ttl now defaults to "auto", which switches a sample's prompt-cache TTL from 5 minutes to 1 hour after a >5 minute gap between its requests; pass "5m" or "1h" to pin.
• Hugging Face literal: task targets now keep the rest of the value when it contains additional colons.
• Review: human_reviewer() lets an operator review a tool call together with its result and continue or terminate the sample, on the same surfaces as the human approver.
• Agents: react() accepts review policies, which apply to the agent's tool calls in place of any eval-level or task-level reviewers, as approval does for approvers.
• OpenAI-compatible token-counting and compaction endpoints that return 405 are now handled the same as those that return 404.
• Bedrock and SageMaker now require aiobotocore instead of aioboto3, which is no longer installed, and Inspect no longer holds botocore back to an old release.
• Bugfix: eval_retry now reuses the model roles recorded in the original log, including roles the task set itself in Task(...).
• Review: New Reviewer protocol and review policies (Task(review=), eval(review=), --review) run after a tool call executes and before the model sees its result, and can continue, terminate, or escalate; each decision is recorded as a ReviewEvent.
• Cancelling an unfinished tool result review stops the sample and preserves the completed tool output in the transcript.
• Tool result reviewers now inspect parsing and approval errors raised by tools that executed.
• OpenRouter: Gemini reasoning now replays as structured reasoning details (keeping the encrypted thought signature for multi-turn tool use) instead of a <think> tag, so reasoning no longer leaks into assistant output text. As in OpenRouter's own SDK, only signed text and encrypted entries are replayed: Gemini no longer sees its own readable prior thinking on later turns, only the thought signature. Reasoning replayed from another provider (no OpenRouter details) now goes into the <think> tag as readable text only, and is omitted entirely when it has none (e.g. a redacted block with no summary), so no signature or opaque payload enters the assistant text channel for any model family.
• OpenRouter: Gemini thoughts returned only as a signature (no readable text) no longer log a warning or surface raw JSON as the reasoning content.
• Fixed Linux evaluations slowing down as model clients open more HTTPS connections.
• Fixed RuntimeError: Event loop is closed when a memoized model is used across multiple eval() calls or event loops.
• Bugfix: The OpenAI Responses provider no longer raises ValueError("Unexpected output type: ResponseToolSearchOutputItem") when an agent uses native OpenAI deferred tool search; the response-item handler now recognises the tool_search_output item without overwriting the cached tool_search_call. (#4968)
• Sample selection: --sample-id now accepts ids containing colons (e.g. user:cybergym/arvo_6008); a task: prefix is stripped only when it names a task in the run.
• Multiple choice: A dataset target of 0 now raises an error instead of being interpreted as option Z on tasks with 26 or more choices.
• Agent Bridge: Bare model names now resolve using the provider of the bridge endpoint, so clients can send names without a provider prefix.
• Agent Bridge: Web search and code execution items from Google and Mistral models now reach Responses API clients with a unique item id instead of an empty one.
• Agent Bridge: Custom tool calls returned to Responses API clients now preserve their registered namespace.
• Scoring: multiple_choice() now recognizes answer letters wrapped in LaTeX or markdown ($B$, **B**, (B)), which previously scored INCORRECT.
• Scoring: perplexity() and target_perplexity() now record infinite perplexity for a sample whose NLL is too large to exponentiate instead of losing the sample to an OverflowError.
• Datasets: csv_dataset() now honors the dialect's delimiter when no explicit delimiter is supplied, including tab-separated and registered custom dialects.
• Datasets: csv_dataset() now loads UTF-8 files with a byte-order mark, including Excel CSV exports, without requiring an explicit encoding.
• Datasets: file_dataset() now reads .tsv and .tab files as tab-delimited instead of rejecting them.
• Documents: Data URIs without parameters now retain their declared media type and receive the corresponding default filename.
• Elicitation: long lines in ask_user prompts are no longer hard-wrapped by the console, so long commands copy out of the terminal intact.
• Compaction: summary compaction now produces a more detailed, structured summary that preserves code snippets, user messages, and any security-relevant constraints stated earlier in the conversation.
• Control Channel: inspect ctl sample cancel now works on a sample that is still initializing (e.g. waiting on sandbox provisioning) — the cancel applies the moment the sample starts, and inspect ctl sample list marks the pending cancel.
• Control Channel: Starting and stopping the control server no longer adds ~200ms to every eval(), which dominated the wall time of very small evals during tests.
• Control Channel: INSPECT_EVAL_CTL_SERVER is now honored by eval() and eval_set() called from Python, not only by the CLI.
• Sample and Task Sources: sample_complete() now fires for a running sample cancelled individually, so a source waiting on that sample no longer stalls; a blocking callback can no longer hang a task cancel.
• Agent Bridge: Google clients now receive token log probabilities and top candidates returned by the host model.
• Google: OAuth/ADC requests to the Gemini Developer API no longer send the placeholder API-key header alongside bearer authentication.
• Scoring: math() now records reason="invalid_response_format" when no answer can be extracted, so format failures are distinguishable from wrong answers.
• Scoring: math() now gives a symbolic answer the same verdict whether it and the target are written in LaTeX or plain notation (e.g. \frac{x}{2} vs x/2), plain-notation symbols match case-insensitively like LaTeX ones, and an answer is parsed under its target's symbol assumptions so cancelling imaginary terms (e.g. x+i-i vs x) still match.
• Scoring: choice() now tags empty completions as NOANSWER with reason="no_response" and records reason="invalid_response_format" when no choice can be parsed, so format failures are distinguishable from wrong answers.
• Scorer: metrics that own a degenerate shape (e.g. grouped()) now report it on an all-unscored run instead of collapsing to a synthesized flat NaN, on both the list and dict metric paths; metrics that raise on empty input still report NaN, with a one-time warning. (#5150)
• Approval: Policy files given as percent-encoded file:// URIs (e.g. paths with spaces, as Path.as_uri() produces) are now accepted by eval(), Task(), and --approval.
• Checkpoints: Invalidating a sample now re-runs it from scratch on retry (its checkpoints are discarded) instead of resuming from its last checkpoint.
• Bugfix: Interrupting a checkpointed eval's retry (Ctrl-C, crash, OOM) no longer loses checkpointed progress, including for samples the retry never reached.

... (truncated)

Commits

• 615e284 update changelog for release
• 7fdb13a Stop denying bridged host tool calls under an approval policy (stopgap until ...
• 957363e regen docs
• 456d982 update changelog for release
• 127d5f3 Fix Krippendorff reference docstring indentation (#5315)
• 854a3b6 Sandbox agent bridge: block cross-origin proxy access and fix stalled tool-ca...
• 036591c Run model data sync only in the upstream repository (#5388)
• 828c615 Preserve custom tool namespaces in agent bridge responses (#5450)
• 64358f1 build(deps): bump astral-sh/setup-uv in the github-actions group (#5431)
• 84dbaa9 docs: fix typos in approval and providers docs (#5437)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[amrit110]

Automated fix applied and PR merged

The agentic fix loop successfully fixed this PR and merged it.

✓ Successfully fixed merge_only failures - Modified 0 files - Executed 114 agent actions - (82 info, 16 tool_call, 3 error, 11 tool_result, 2 reasoning)

View PR history on dashboard

AI Engineering Maintenance Bot