[STG-2515] feat(evals): anthropic_sdk bare-loop harness adapter

[shrey150]

What

Part 3 of 5 — stacked on #2338 (vercel_ai_sdk) (base: shrey/evals-harness-vercel-ai-sdk; GitHub shows only this PR's delta). When the parent merges, retarget this PR's base and rebase.

Registers --harness anthropic_sdk: a hand-rolled tool_use loop via @anthropic-ai/sdk — the TS twin of the Python while stop_reason == "tool_use" loop that dominates real-world Anthropic usage (37.5M PyPI dl/wk) and of the Modal sandbox template's agent.mjs. No retries, no planning, no memory — deliberately: the bareness is the instrument.

• Anthropic models only (anthropic/ prefix stripped; other providers rejected loudly).
• Step cap: shared 40 default, EVAL_ANTHROPIC_SDK_MAX_STEPS override; when the cap binds it's surfaced as the stop reason.
• Assistant text buffered per turn folds into the next tool call's reasoning; usage summed across turns.
• Adds the @anthropic-ai/sdk dependency (pinned 0.39.0, matching packages/core) — this PR carries its own lockfile delta.
• Testability: client injection seam (messages.create mock).

External harnesses design + usage: packages/evals/README.md#external-harnesses (in #2337). Linear: STG-2515.

Comparability fix (2026-07-09 audit)

Wired the caller's AbortSignal into @anthropic-ai/sdk's native abort option (messages.create(body, { signal })) instead of only polling signal?.aborted between loop iterations — previously an abort mid-loop left an already-in-flight messages.create() call running to completion instead of cancelling the HTTP request. One new test asserts the signal is forwarded as-is to the client.

(anthropic_sdk's step cap reached (N) stop-reason shape was already correct here — it's the reference shape vercel_ai_sdk (#2338) and openai_agents_sdk (#2340) were fixed to match, per the same audit.)

E2E Test Matrix

Live smoke ran on the pre-split combined branch (content identical to this stack's tip — verified by diff); results transplanted, not rerun.

Command / flow	Observed output	Confidence / sufficiency
evals run b:webtailbench --harness anthropic_sdk -m anthropic/claude-haiku-4-5-20251001 -l 1 -t 1 -c 1 (local build, real ANTHROPIC + GEMINI keys, local Chrome via browse CLI)	Run completed (exit 0) at the 40-step cap (stop reason step cap reached (40)). 40 browse tool calls recorded; trajectory persisted; verifier graded: outcomeSuccess=false, processScore=0.29, 5 rubric criteria, no verifierError. Usage: 172k in / 2.6k out tokens	Bar met: harness completes, trajectory captured, graded outcome, no verifierError. Task pass/fail is NOT the bar (united.com flights). Cap-binding at 40 with haiku on a hard task is exactly the instrument behavior the design doc predicts
pnpm exec vitest run (this branch)	Test Files 51 passed, Tests 376 passed — adds anthropicSdkRunner.test.ts (multi-turn loop threading assistant/tool_result messages, tool_use → browse execution, step-cap stop reason, usage summing, model-prefix validation, SDK-error folding)	Mocked-model loop coverage; live behavior proven by the smoke row above
tsc --noEmit + prettier + eslint	clean	Types/style only
pnpm exec vitest run (this branch, post-fix)	Test Files 51 passed, Tests 379 passed — adds the AbortSignal-forwarding test	Proves the signal reaches messages.create's options arg unchanged
pnpm -w exec tsc --noEmit + prettier + eslint (post-fix)	clean	Types/style only

No changeset: packages/evals is private.

🤖 Generated with Claude Code

[changeset-bot]

⚠️ No Changeset found

Latest commit: 7d73972

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

No issues found across 7 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

_{Re-trigger cubic}