Skip to content

CODAP-1408: Coalesce streamed create-item requests into batched model changes#2631

Open
kswenson wants to merge 17 commits into
CODAP-1403-streaming-points-animationfrom
CODAP-1408-coalesce-streamed-creates
Open

CODAP-1408: Coalesce streamed create-item requests into batched model changes#2631
kswenson wants to merge 17 commits into
CODAP-1403-streaming-points-animationfrom
CODAP-1408-coalesce-streamed-creates

Conversation

@kswenson

@kswenson kswenson commented Jun 10, 2026

Copy link
Copy Markdown
Member

Summary

When a plugin streams cases one at a time (e.g. Simmer emitting ~200 items), each create request triggered its own model change and full tile-reactivity cascade — O(N) per add, O(N²) per stream — and the plugin's ~2s request timers started timing out around n≈118 with an open graph.

The request-queue processor now:

  • Defers its drain by a macrotask so already-delivered postMessage requests accumulate in the queue (previously each request was fully processed within its own message event, so the queue never held more than one).
  • Coalesces runs of consecutive single-item create … item requests into one batched create — one applyModelChange, one addCases, one notification round per batch — responding to each original request with its own per-segment result (positional itemIDs; new cases attributed to the segment of their earliest contributing item, matching sequential semantics).
  • Processes one work unit per drain tick, leaving the rest in the queue so the browser paints between batches (preserving the visible sense of streaming) and late-arriving requests merge into later batches.
  • Adapts the batch size to the backlog (each tick takes up to 1/10 of the queue, floor 5) and to the head request's queue wait (as it approaches 1s, the batch ramps up to the entire backlog) — keeping up with large/fast streams and slow machines while streaming visibly otherwise.
  • Suppresses point animation only for coalesced batches (addresses CODAP-1403: Graph points respond appropriately to streamed data #2628 review feedback): user-entered rows and paced plugin creates animate as usual; only high-speed streams snap.

If a batched create cannot proceed, the run's members are processed individually (exact sequential semantics). The noCoalesce debug flag disables coalescing for A/B comparison.

Measured (Simmer streams, graph + table open)

  • Before: per-request cost grew 9→20+ ms; timeouts from n≈118 of 200.
  • After: 200 one-at-a-time creates process in ~20 batches, ~60 ms total processing (was ~2,900 ms); no timeouts. Nine successive 500-case streams (4,500 cases) completed without timeouts; practical ceiling on the test machine ≈5,000 cases, where the residual cost is the per-batch graph re-render at large N (CODAP-1404/1405/1406 territory).
  • Sampler is unaffected: its rAF-paced create/select alternation never coalesces and never backs up — and its creates animate again.

Semantics notes

  • Plugin acks are delivered per batch; every ack arrives earlier in wall-clock terms at scale.
  • Observing plugins receive fewer, larger createCases notifications.
  • All plugin requests gain one macrotask (~0–4 ms) of latency from the deferred drain.
  • The drain is now strictly serial (previously all queued processors launched without awaiting).
  • Latent fix: tableModified was checked before the async processors could set it, so incrementInterruptionCount never fired from this path; it now works.
  • Unchanged: undo (plugin creates aren't undoable) and request ordering (only consecutive runs coalesce).

Post-review hardening (high-effort code review)

  • Data integrity — no re-add on a failed coalesced create. A batched create mutates the dataset inside applyModelChange, which is not transactional on throw, so a failure after addCases can leave cases committed. The fallback that re-processed members individually would then add those cases a second time (silent duplication). Now the individual fallback runs only when the dataContext was unresolvable (nothing mutated); any post-resolve failure responds with an error to each member instead. Covered by a new regression test.
  • Cleanup — single-segment attribution. createItemsInSegments no longer builds the itemId→segment map or runs the per-case reduce on the ordinary (non-coalesced) create path, where every new case trivially belongs to segment 0. Behavior-preserving.
  • Cleanup — lazy debug logging. The per-member coalesced-processing log no longer eagerly JSON.stringifys each request on the streaming hot path when DEBUG_PLUGINS is off.

Stacked on #2628 (CODAP-1403); retarget to main after it merges.

Fixes CODAP-1408.

🤖 Generated with Claude Code

kswenson and others added 5 commits June 10, 2026 08:32
…to remove before PR)

Restores the request-latency logging ([REQ-DBG]) and prf.measure blocks from
292809a (stripped from the CODAP-1404 branch in 9111557), plus the
syncMs/asyncMs split in the REQ-DBG log used for the graph-open/closed
bisection. Establishes the measurement baseline for coalescing streamed
create-item requests. Not for merge — strip before the PR is ready.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… changes

Plugin requests arrive one per postMessage event and were drained synchronously,
so each streamed create triggered its own model change and full tile-reactivity
cascade — O(N) per add, O(N²) per stream — tripping the plugin's ~2s request
timer at scale (a 200-case Simmer stream with an open graph timed out from
n≈118). The request queue processor now defers its drain by a macrotask so
already-delivered requests accumulate, then processes ONE work unit per drain
tick: a run of consecutive single-item creates (capped at 10) executes as one
batched create — one applyModelChange, one addCases, one notification round —
while the browser paints between ticks, preserving the visible sense of
streaming. Measured: 200 streamed creates ≈ 60ms total processing (was ~2,900ms
and climbing), zero plugin timeouts.

- request-coalescer.ts: takeNextWorkUnit() identifies the coalescable run (or
  single request) at the head of the queue, up to a max run length
- item-handler.ts: createItemsInSegments() executes a multi-segment batch and
  returns per-segment results (positional itemIDs; new cases attributed to the
  segment of their earliest contributing item, matching sequential semantics);
  handler create() is the single-segment case
- request processor: deferred one-unit-per-tick drain with fallback to
  individual processing if a batched create cannot proceed; tableModified is
  now checked after processing completes (previously it was checked before the
  async processors could set it, so interruption counting never fired here)
- request-queue.ts: adds items getter and takeItems(); processItems() removed;
  RequestPair exported
- DEBUG_NO_COALESCE ("noCoalesce" debug flag) processes requests individually
  for A/B comparison

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the fixed batch cap of 10 with adaptive sizing driven by two signals:

- Backlog-proportional: each drain tick takes up to 1/10 of the queued
  requests (floor of 5), so small backlogs stream in fine-grained batches
  while large ones grow the batch to keep up. Machine speed is adapted to
  implicitly: slower ticks let more requests accumulate, growing the batch.
- Queue-wait urgency: as the head request's wait approaches 1s, the batch
  ramps up to the entire backlog — timely acknowledgment (plugin request
  timers are conventionally ~2s) outranks streaming granularity. Since a
  flood's tail requests have been waiting since the flood began, this also
  accelerates the tail of a large drain instead of dribbling it out at the
  minimum batch size (500-case streams into a large dataset timed out with
  backlog-proportional sizing alone).

Promotes the request queue's enqueuedAt timestamp from temporary diagnostic
to a real field consumed by the processor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CODAP-1403 (#2628) suppressed point animation on every addCases to fix the
streaming animation freeze, but as review noted, animation is desirable for
individual case additions — user-entered rows, plugin creates at user-visible
rates. Whether the request processor coalesced multiple create requests into
one batch is exactly the "high-speed stream" discriminator, so suppression is
now opt-in via a new addCases suppressAnimation option that the item handler
sets for multi-segment (coalesced) batches only. A single create request —
even one carrying many items — animates as an ordinary bulk add. removeCases
suppression is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Strip the temporary request-latency logging ([REQ-DBG]) and prf.measure
blocks re-added under 4042f0d to measure the coalescing work. The request
queue's enqueuedAt timestamp remains — it now drives queue-wait-aware batch
sizing. Also drops an unnecessary optional call on the handler function in
processAction, which is guarded above.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kswenson kswenson added the v3 CODAP v3 label Jun 10, 2026
@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.74011% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.37%. Comparing base (2e6decc) to head (09a78b8).

Files with missing lines Patch % Lines
...-interactive/data-interactive-request-processor.ts 96.55% 3 Missing ⚠️
v3/src/lib/embedded-mode/embedded-server.ts 0.00% 1 Missing ⚠️
Additional details and impacted files
@@                            Coverage Diff                            @@
##           CODAP-1403-streaming-points-animation    #2631      +/-   ##
=========================================================================
+ Coverage                                  87.33%   87.37%   +0.04%     
=========================================================================
  Files                                        798      799       +1     
  Lines                                      45468    45577     +109     
  Branches                                   11523    11545      +22     
=========================================================================
+ Hits                                       39708    39825     +117     
+ Misses                                      5747     5741       -6     
+ Partials                                      13       11       -2     
Flag Coverage Δ
cypress 70.29% <73.96%> (-0.01%) ⬇️
jest 61.13% <96.61%> (+0.24%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cypress

cypress Bot commented Jun 10, 2026

Copy link
Copy Markdown

codap-v3    Run #11916

Run Properties:  status check passed Passed #11916  •  git commit 09a78b8958: null
Project codap-v3
Branch Review CODAP-1408-coalesce-streamed-creates
Run status status check passed Passed #11916
Run duration 08m 51s
Commit git commit 09a78b8958: null
Committer null
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 1
Tests that did not run due to a developer annotating a test with .skip  Pending 82
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 380
View all changes introduced in this branch ↗︎

Adds doc/plugin-request-processing.md describing the request pipeline
(queueing, deferred drain, coalescing, adaptive batch sizing, response
slicing), plugin-visible semantics, the plugin traffic patterns that shaped
the design, and known limits/future extensions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes CODAP v3’s data-interactive/plugin request pipeline to prevent O(N²) slowdown and timeouts when plugins stream many single-item create … item requests by batching/coalescing them into fewer model changes while preserving sequential semantics.

Changes:

  • Defer request-queue draining by a macrotask and process one “work unit” per tick, enabling consecutive create-item requests to coalesce into batched creates with adaptive batch sizing.
  • Add suppressAnimation plumbing so high-speed coalesced adds snap (no point animation restart), while ordinary adds continue to animate.
  • Add targeted unit tests and documentation describing the new queue/coalescing behavior and semantics.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
v3/src/models/data/data-set-types.ts Adds suppressAnimation option to addCases options for downstream consumers (graph/data-display).
v3/src/lib/embedded-mode/embedded-server.ts Updates disposer typing to match new request-processor return type.
v3/src/lib/debug.ts Adds DEBUG_NO_COALESCE debug flag for A/B comparison.
v3/src/data-interactive/request-coalescer.ts Introduces logic to form “single” vs “coalesced” work units from queued requests.
v3/src/data-interactive/request-coalescer.test.ts Adds unit tests for coalescing rules and max-run-length behavior.
v3/src/data-interactive/handlers/item-handler.ts Adds createItemsInSegments() to execute coalesced creates as one model change and slice per-request results.
v3/src/data-interactive/handlers/item-handler.test.ts Adds tests for segment slicing, case attribution semantics, and animation suppression behavior.
v3/src/data-interactive/data-interactive-request-processor.ts Reworks queue draining to be deferred/serial and to execute coalesced create runs via createItemsInSegments().
v3/src/data-interactive/data-interactive-request-processor.test.ts Adds tests for deferred draining, batching size behavior, ordering, and fallback-to-individual processing.
v3/src/components/web-view/request-queue.ts Adds enqueue timestamps and replaces processItems() with items + takeItems() to support the new drain model.
v3/src/components/graph/models/graph-data-configuration-model.test.ts Updates expectations: suppress-on-add is now opt-in via suppressAnimation, remove always suppresses.
v3/src/components/data-display/models/data-configuration-model.ts Applies suppression on remove always, and on add only when requested via addCases option.
v3/doc/plugin-request-processing.md New internal documentation for the end-to-end request queuing/coalescing pipeline and semantics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread v3/src/data-interactive/data-interactive-request-processor.ts Outdated
Comment thread v3/src/data-interactive/data-interactive-request-processor.ts Outdated
Comment thread v3/src/data-interactive/handlers/item-handler.ts Outdated
…iew)

- A throwing handler no longer drops the request: processSingleRequest
  catches the exception and responds with an errorResult, so the plugin
  gets an error instead of waiting out its request timer (new
  V3.DI.Error.exceptionProcessingRequest string).
- Response callbacks are invoked through a guard so one throwing consumer
  can't prevent later responses (e.g. the remaining members of a coalesced
  batch) from being delivered.
- New-case attribution now uses a precomputed itemIndex→segmentIndex
  lookup instead of a per-case linear scan of the segment ranges, avoiding
  O(segments × cases) behavior in large coalesced batches.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Comment thread v3/src/data-interactive/data-interactive-request-processor.test.ts
Comment thread v3/src/data-interactive/data-interactive-request-processor.test.ts Outdated
Comment thread v3/src/data-interactive/data-interactive-request-processor.test.ts
…Copilot review)

Captures the dataset's name after creation and uses it in the test request
resource strings, so the tests stay correct if createDataSet ever renames
the dataset for uniqueness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kswenson and others added 5 commits June 10, 2026 13:35
createItemsInSegments unwrapped the Collaborative `{ values: {...} }`
shape via `typeof item.values === "object"`, which also matched an item
whose own `values` attribute held an object (dropping its other fields)
or null (`typeof null === "object"` -> crash on the __id__ access). Only
unwrap a non-null, non-array object so a malformed or "values"-named
attribute can't corrupt the item or throw.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Explain why the request-queue processor needs no "disposed" flag: at most
one drain timer is ever pending (always held in drainTimer), takeItems
re-fires the reaction to schedule the next timer before the await, and the
disposer clears that one timer -- so a mid-await drain can't leak a
follow-up timer. Comment only; no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
createItemsInSegments identifies new cases by an identity-based before/
after diff, which correctly excludes pre-existing parent/middle cases that
a streamed item merely joins. Add a hierarchical case guarding that
invariant: one segment joining an existing (a1,a2) group reports only its
new leaf case, while a segment introducing a fresh hierarchy reports its
new parent, middle, and leaf cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapse the three parallel structures used to attribute new cases back
to their originating request -- segmentIndexByItemIndex, itemIndexMap, and
segmentRanges -- down to segmentRanges plus a single id->segment map. Since
items are appended in segment order, a new case's earliest contributing
item is simply the one in its lowest-numbered segment, so the reduce finds
the min segment directly rather than min item-index then an index->segment
lookup.

Add a characterization test covering attribution to a non-zero earliest
segment (segment 2 joins a parent created by segment 1): the prior tests
all had segment 0 as the earliest contributor, so this guards the exact
behavior the simplification touches.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The new-case attribution snippet wrote caseInfoMap.get(caseId).childItemIds,
but the code uses optional chaining (caseInfoMap.get(caseId)?.childItemIds
?? []) to handle the no-contributing-item edge case the same paragraph
describes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kswenson kswenson marked this pull request as ready for review June 10, 2026 23:55
@kswenson kswenson requested a review from bfinzer June 10, 2026 23:55

@bfinzer bfinzer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻LGTM
Wow! Quite a subtle and important piece of work. Claude did not come up with any blockers.

processCoalescedRun fell back to processing members individually whenever
the batched createItemsInSegments did not cleanly succeed. Because
applyModelChange is not transactional on throw, a failure after addCases
leaves cases committed, so the fallback re-added them — silent data
duplication. Only fall back when the dataContext was unresolvable (nothing
mutated); otherwise respond with an error to each member. Also make the
per-member debug log lazy instead of eagerly stringifying on the hot path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… path

createItemsInSegments built an itemId->segment Map and ran a per-case
childItemIds.reduce even for an ordinary (non-coalesced) create, where every
new case trivially belongs to segment 0. Guard the attribution machinery
behind segments.length > 1 and take a direct segment-0 path otherwise.
Behavior-preserving: a single segment always attributed every case to
segment 0 anyway.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kswenson

Copy link
Copy Markdown
Member Author

Code review — remaining items (non-blocking)

A high-effort review (8 finder angles, independent verification) produced the fixes already landed on this branch (see the PR description's Post-review hardening section). The items below are deliberate-design tradeoffs and judgment calls, not bugs — recording them here so reviewers can weigh in and so the risk surface is documented.

Behavior changes worth a conscious call

  • Removal animation suppressed globally, not just for streams. removeCases now unconditionally sets suppressAnimation=true. Note this does not animate the deleted point out — that sprite was always destroyed instantly. What's lost is the ~1s glide of the surviving points/bars settling/rescaling after a deletion (visible in dot plots that re-pack, bar charts that resize, and any plot whose axis range was pinned by the deleted value). Applies to every deletion (table row delete, plugin delete item/delete case, selection delete), not only the streaming case this PR targets. No data/correctness impact. If we want to keep the glide for ordinary deletes, gating suppression on a streaming/bulk signal would be a deliberate follow-up (the removal path doesn't currently carry that signal).

  • suppressAnimation can latch on an off-screen plot. The flag is set when a coalesced/removal action is forwarded but only cleared inside matchCirclesToData. For a graph/map that isn't rendering when a coalesced batch arrives, the flag stays set until the next matchCirclesToData, so the first later add on that backgrounded plot draws without its enter animation. Cosmetic, one-time, self-correcting.

Tradeoffs already noted in the PR / docs

  • Per-request latency. Every plugin request (not just streamed creates) now waits one macrotask before processing. Worth confirming a chatty, round-trip-heavy non-streaming plugin doesn't regress noticeably.
  • Batched notifications. Streamed creates now emit one createCases notification per collection per batch instead of one per request. A plugin that counts notifications or pairs one-per-item could under-count — validate against real plugins (Simmer/Collaborative).
  • Coalescing keys on exact resource-string equality. Two streamed creates targeting the same dataContext via different selector forms (name vs id vs #default) won't coalesce, so that stretch loses the batching/animation-suppression benefit. Degraded perf only, never wrong data.

Suggested manual test plan before merge

  1. Streaming: run Simmer/Collaborative pushing a large burst; confirm case + point counts match exactly (no duplicates), points don't freeze, no console errors.
  2. Stream + mid-stream error: feed one malformed item mid-stream; confirm one error response and no duplicated good rows around it (exercises the no-re-add fix).
  3. Chatty non-streaming plugin: compare wall-clock completion vs main (macrotask latency).
  4. Notification-listening plugin: confirm correct totals with batched notifications.
  5. Deletions: single table-row delete, plugin delete item, selection delete — confirm the instant (no-glide) removal is acceptable.
  6. Latch check: start a streaming burst with the graph in a backgrounded tile, bring it forward, add one case — confirm the first add's missing animation self-corrects.
  7. Regression label for the full Cypress suite (data-interactive + graph-animation specs).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

Comment thread v3/src/data-interactive/data-interactive-request-processor.ts
Comment thread v3/src/data-interactive/data-interactive-request-processor.ts Outdated
Comment thread v3/doc/plugin-request-processing.md Outdated
…g, doc)

- processSingleRequest debug log no longer eagerly JSON.stringifies the request
  when DEBUG_PLUGINS is off (matches the coalesced path).
- onProcessingStart fires once before the batched create rather than per-member
  after it, so it accurately signals when processing starts.
- Update plugin-request-processing.md "Failure handling" to match the implemented
  semantics: only an unresolvable dataContext falls back to per-request processing;
  a post-resolve failure responds with per-request errors without retrying (avoids
  re-adding cases already committed by the non-transactional applyModelChange).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Comment thread v3/src/data-interactive/data-interactive-request-processor.ts
The single-timer invariant alone did not make teardown safe. When takeItems()
empties the queue, the reaction does not re-arm a timer before drain()'s await,
so drainTimer is null at that point. If the processor is disposed mid-await and a
request then arrives (e.g. an in-flight postMessage before the endpoint
disconnects), the trailing scheduleDrain() in drain()'s finally would arm a new
timer and process that request after disposal.

Add an explicit `disposed` flag, checked in scheduleDrain() and at the top of
drain() and set by the disposer (which also nulls drainTimer), so no drain runs
after teardown. Correct the now-inaccurate "no disposed guard needed" comments.
Add a regression test that holds a drain mid-await, disposes, pushes a request,
and asserts it is not processed (verified red before the fix).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Comment thread v3/src/utilities/translation/lang/en-US.json5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants