Skip to content

fix(aiops): final answers no longer dropped, fragmented, or duplicated - #100

Open
OmarHussain09 wants to merge 3 commits into
offloadmemory:master-v1from
OmarHussain09:fix/ai-ops
Open

OmarHussain09 wants to merge 3 commits into
offloadmemory:master-v1from
OmarHussain09:fix/ai-ops

Conversation

@OmarHussain09

@OmarHussain09 OmarHussain09 commented Aug 4, 2026 •

Copy link
Copy Markdown

Three fixes for the same end-to-end failure: a finished AI Ops report was thrown away and replaced with "I wasn't able to produce the analysis you requested", and what survived to history came back shattered or duplicated. All three are triggered by reasoning models, which return content-block arrays instead of strings and keep every streamed delta as its own block.

1. fix(aiops): promote a reasoning-model deliverable instead of re-synthesizing it

finalNode promotes an already-composed report verbatim to avoid paying for a closing LLM call, but gated that on typeof m.content === 'string'. A reasoning model never returns a string — one 7k-char report arrived as 544 separate text blocks. The gate missed it, so the finished report was dropped and the fallback re-synthesized from toolResults.slice(-3); with only three SSM shell results in view the model said, correctly for what it could see, that it had failed — moments after producing the analysis. The UI then elevated that into the Report card.

Extracted the scan as findRenderedDeliverable() and routed content through contentToText(), which this file already applies in six other places for the same reason.

2. fix(chat): coalesce streamed content blocks when replaying history

reconstructAiContentParts emitted one UI part per persisted block, so a reloaded answer shattered into hundreds of rows split mid-word ('ikes', ' latency <0', '.3', 'ms, ') — 644 for one report. Consecutive same-kind blocks now accumulate into a single part, and whitespace is judged per run rather than per block: Bedrock emits bare-space deltas between mid-word splits, and discarding those individually fused halves together ("no" + " " + "blocking" → "noblocking"). Runs are still trimmed at their ends, so existing leading-whitespace behaviour holds. Applies to every stored thread on reload, not just new runs.

3. fix(history): drop the duplicated promoted answer on reload

Promotion re-emits the deliverable verbatim as a final-phase message. Live, that copy never reaches the client — promotion makes no model call and the route only emits text from on_chat_model_* events. Persistence has no such filter, so both rows were stored and history replayed the report twice: once as execution prose, once inside the Report card.

The later copy is now dropped so a reloaded thread matches the live view. Matching is on text, not phase — a fragmented reasoning-model turn is persisted as a raw content-block array with no phase marker (route.ts only prefixes the marker when content is a string), so the execution copy often has no phase to match on. The two stored copies are byte-identical once the marker is stripped (6420 chars each in the thread this was found in), so exact comparison suffices. A genuine finalNode fallback answer differs from the execution text and is never dropped, keeping "did promotion fire?" readable from the transcript. The 800-char floor mirrors finalNode's own, so ordinary repeated narration is left alone. Wired into both history paths (chat-history and the checkpoint fallback).

Tests

26 new/existing tests across the three touched areas pass:

cd apps/web-ui && bun run test -- tests/history-dedupe-answers.test.ts \
  lib/agent-chat/__tests__/ai-content-parts.test.ts \
  lib/agent/__tests__/rendered-deliverable.test.ts

Test Files  3 passed (3)
     Tests  26 passed (26)

🤖 Generated with Claude Code

Omar Hussain and others added 3 commits August 4, 2026 07:45
…esizing it

finalNode promotes an already-composed report verbatim rather than paying for a
closing LLM call, but it gated that on `typeof m.content === 'string'`. A
reasoning model never returns a string: a turn that emits reasoning alongside the
answer arrives as a content-block array, and Sonnet 5 keeps every streamed delta
as its own text block — 544 blocks for one 7k-char report in the run this was
found in.

The gate silently missed those, so a finished report was dropped and the fallback
re-synthesized an answer from `toolResults.slice(-3)`. With only three SSM shell
results in view the model said, correctly for what it could see, "I wasn't able
to produce the Aurora Serverless V2 analysis you requested" — after having
produced exactly that analysis moments earlier. The UI then elevated that into
the Report card, since REPORT_PHASES covers the final phase.

Extract the scan as findRenderedDeliverable() and route content through
contentToText(), which this file already applies in six other places for the same
reason. Whether the answer promotes no longer depends on which turn the model
happened to attach its reasoning to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
reconstructAiContentParts emitted one UI part per persisted content block. A
reasoning model keeps every streamed delta as its own block, so a reloaded answer
shattered into hundreds of rows — 644 for one report — each a fragment of a
sentence, split mid-word ('ikes', ' latency <0', '.3', 'ms, ').

Two changes, both needed:

  - consecutive blocks of the same kind now accumulate into a single part, so a
    645-block message reloads as one text part;
  - whitespace is judged per RUN rather than per block. Bedrock emits bare-space
    deltas between mid-word splits, and discarding those individually fused the
    halves together ("no" + " " + "blocking" -> "noblocking"). The run is still
    trimmed at its ends, so the existing leading-whitespace behaviour holds.

Applies to every stored thread on reload, not just new runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
finalNode promotes a deliverable by re-emitting it verbatim as a final-phase
message. Live, that copy never reaches the client: promotion makes no model call,
and the route only emits text from on_chat_model_* events. Persistence has no
such filter, so both rows are stored and history replayed the report twice —
once as execution prose, once inside the Report card.

Drop the later copy so a reloaded thread matches the live view. Matching is on
text, not phase: a fragmented reasoning-model turn is persisted as a raw
content-block array with no phase marker at all (route.ts only prefixes the
marker when content is a string), so the execution copy often has no phase to
match on. The two stored copies are byte-identical once the marker is stripped —
6420 chars each in the thread this was found in — so exact comparison suffices.

A genuine finalNode fallback answer differs from the execution text and is
therefore never dropped, which keeps "did promotion fire?" readable from the
transcript. The 800-char floor mirrors finalNode's own, so ordinary repeated
narration is left alone.

Wired into both history paths (chat-history and the checkpoint fallback).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@OmarHussain09 OmarHussain09 changed the title fix(aiops): stop dropping reasoning-model reports, and replay them intact fix(aiops): final answers no longer dropped, fragmented, or duplicated Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant