fix: silence AI SDK "system message in messages" warning in act/extract/observe

[shrey150]

What & why

#2305 silenced the AI SDK v5 "system message in messages" warning, but only for agent.execute()'s outer loop (v3AgentHandler.ts). act(), extract(), and observe() build their system prompt the same way — a { role: "system", ... } message inside the messages array — and route through a separate call path that was never patched: AISdkClient.createChatCompletion() in packages/core/lib/v3/llm/aisdk.ts (the default client for any "provider/model" string), plus the identical pattern in the public BYOC client packages/core/lib/v3/external_clients/aisdk.ts.

Since the hybrid/DOM agent's own act/extract/observe tools call these primitives internally on every step, an agent.execute() run that clicks/types repeatedly still spammed the warning even after upgrading past the first fix — a customer reported exactly this.

Fix

Add allowSystemInMessages: true to the 4 remaining generateObject/generateText call sites (2 in llm/aisdk.ts, 2 in external_clients/aisdk.ts), same pattern as the original fix.

E2E Test Matrix

Command / flow	Observed output	Confidence / sufficiency
BEFORE — published @browserbasehq/stagehand@alpha (already includes #2305's fix), act() → extract() → observe() chain on a real Browserbase session	AI SDK system-in-messages warnings emitted: YES (4)	Reproduces the customer's exact symptom on the current alpha — confirms #2305 left this path unpatched.
AFTER — local patched build (this branch), identical act() → extract() → observe() chain, same model, real Browserbase session	AI SDK system-in-messages warnings emitted: NO (0)	Proves the fix removes the warning from all three primitives.
AFTER — local patched build, full chain: act() + extract() + observe() + hybrid agent.execute() (multi-step form fill, real Browserbase session)	AI SDK system-in-messages warnings emitted: NO (0 total)	Confirms no regression on the already-fixed agent.execute() outer loop, and that the agent's internal act/extract/observe tool calls are also silenced end-to-end.
pnpm turbo run build --filter @browserbasehq/stagehand	Tasks: 3 successful, 3 total	Typecheck + emit green.
eslint + prettier --check on changed files	Both clean	Style/lint gates pass.

A/B model: anthropic/claude-haiku-4-5-20251001, experimental: true, hybrid agent mode — same harness shape as #2305's own E2E matrix.

Closes STG-2573.

---

Summary by cubic

Silences the noisy "system message in messages" warning in act(), extract(), and observe(), including internal tool calls. Sets allowSystemInMessages: true at 4 generateObject/generateText sites and adds unit tests for both AISDK clients; closes STG-2573.

^{Written for commit d001bd5. Summary will update on new commits.}

[cubic-dev-ai]

All reported issues were addressed across 3 files

Confidence score: 5/5

• Safe to merge after the addressed issues were fixed.

Architecture diagram

sequenceDiagram
    participant Client as External Caller
    participant Agent as Hybrid Agent
    participant ActTool as act() tool
    participant ExtractTool as extract() tool
    participant ObserveTool as observe() tool
    participant AISdkClient as AISdkClient (llm/aisdk.ts)
    participant BYOCClient as BYOC AISdkClient (external_clients/aisdk.ts)
    participant OpenAI as AI SDK / LLM Backend

    Note over Client,OpenAI: System prompt is built as { role: "system", ... } message inside messages array

    Client->>Agent: agent.execute() (multi-step loop)
    Agent->>ActTool: internal tool call (click/type)
    ActTool->>AISdkClient: createChatCompletion() with system message

    Note over AISdkClient: allowSystemInMessages: true (generation call)
    AISdkClient->>OpenAI: generateText() with system in messages

    Agent->>ExtractTool: internal tool call
    ExtractTool->>AISdkClient: createChatCompletion() with system message
    Note over AISdkClient: allowSystemInMessages: true (generation call)
    AISdkClient->>OpenAI: generateObject() with system in messages

    Agent->>ObserveTool: internal tool call
    ObserveTool->>AISdkClient: createChatCompletion() with system message
    Note over AISdkClient: allowSystemInMessages: true (generation call)
    AISdkClient->>OpenAI: generateText() with system in messages

    Note over AISdkClient: Two call sites: generateObject (response_model) and generateText (tools)

    alt External caller uses BYOC (Bring Your Own Client)
        Client->>BYOCClient: createChatCompletion() with system message
        Note over BYOCClient: allowSystemInMessages: true (generation call)
        BYOCClient->>OpenAI: generateObject() with system in messages

        Client->>BYOCClient: createChatCompletion() with system message
        Note over BYOCClient: allowSystemInMessages: true (generation call)
        BYOCClient->>OpenAI: generateText() with system in messages

        Note over BYOCClient: Two call sites: generateObject (response_model) and generateText (tools)
    end

    Note over Client,OpenAI: Total 4 call sites patched (2 in llm/aisdk.ts, 2 in external_clients/aisdk.ts)
    Note over OpenAI: AI SDK v5 warning suppressed for all system-in-messages calls

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[changeset-bot]

🦋 Changeset detected

Latest commit: d001bd5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/stagehand-evals	Patch
@browserbasehq/stagehand-server-v3	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pirate]

Approved assuming the warning is actually spurious.

If we are indeed repeating the system prompt in the message that seems like it could be wasting tokens / context window unnecessarily?

[shrey150]

@pirate good question, but no duplication here — the system prompt is only ever passed once per call. It's never passed as a separate top-level system: param and a messages entry; it's built as a single role: "system" message that gets prepended to messages, and that's it (see prependSystemMessage() in v3AgentHandler.ts, and the equivalent mapping in llm/aisdk.ts/external_clients/aisdk.ts).

allowSystemInMessages: true doesn't change what gets sent to the model at all — it only tells the AI SDK "don't warn/throw about this pattern." Same request body before and after this PR.

On the token cost question: every multi-step agent call is a fresh, stateless completion request, so the system prompt has to be resent on every step regardless of whether you use the system: param or a role: "system" message in messages — that's inherent to the chat completions API, not something this PR introduces or changes.

The reason it's deliberately kept as a messages entry (rather than moved to the "textbook" top-level system: string) is the providerOptions.anthropic.cacheControl: { type: "ephemeral" } attached to it — that only works on a message-array entry, not the top-level system string. That's Anthropic prompt caching, which makes repeated identical system prompts across agent steps cheaper, not more expensive. Moving to system: would actually regress token cost on Anthropic models by losing that cache hit. So this pattern is intentional and net cost-reducing, not a mistake — the original PR (#2305) called out this exact tradeoff.