perf(datasource): stop O(N) GUI-thread scans on pan/zoom/autoscale

[jeandet]

Stacked on #33 (← #32) — merge those first; this PR's diff then collapses to its own commit (f3653cd).

Three hot paths scanned entire datasets despite the sorted-keys contract:

• keyRange: the multi-column and 2D sources used the linear-scan qcp::algo::keyRange while keyRangeSorted (O(1)/O(log n)) existed and was already used by the single-column source. QCPColorMap2's pipeline calls xRange on every viewport change (each pan step) and selectTest per hit-test. Switched QCPSoAMultiDataSource, QCPRowMajorMultiDataSource and qcp::algo2d::xRange to keyRangeSorted (2D yRange stays linear: variable-Y layouts are not globally sorted).

• valueRange with a key restriction iterated all N points testing each key against the window: a zoomed-in autoscale on a 10M×4 multigraph read 40M elements on the GUI thread per data_changed. The scan is now bounded by findBegin/findEnd. Histogram keys are scattered (sorted contract does not hold), so QCPHistogram2D::getValueRange now does its own linear scan for restricted ranges instead of inheriting the windowed source path.

• QCPHistogram2D::selectTest ran finiteKeyValueBounds — a full O(N) scan through virtual accessors — on every mouse press. Bounds are now memoized per (keyPositiveOnly, valuePositiveOnly), invalidated by setDataSource/dataChanged; getKeyRange shares the cache.

Measured on the SciQLopPlots side (10M×3 multigraph, 0.1% visible window, together with the companion windowed-percentile change there): min/max rescale ~17 ms → ~0.04 ms (~400×), percentile rescale ~7 ms → ~0.5 ms (~13×) per call.

Tests: algoValueRangeRestrictionMatchesBruteForce (windowed == brute force across boundary-exact/outside/degenerate windows and sign domains), histogram2dValueRangeRestrictedUnsortedKeys (guards the scattered-keys path against the sorted-window optimization), histogram2dSelectTestReflectsNewData (cache invalidation).

Part of the 2026-06-09 audit (M1/M2/M4). Stack: #32 → #33 → this → small sweep.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR removes multiple O(N) dataset scans from GUI-thread hot paths (pan/zoom/autoscale/select) by switching to sorted-key range helpers and by bounding key-restricted value-range scans via binary search; it also adds targeted concurrency-safety and regression tests around the async pipeline and plottable lifetimes.

Changes:

• Switch multi/2D sources to keyRangeSorted and optimize key-restricted valueRange to scan only [findBegin, findEnd) rather than all points.
• Add QCPHistogram2D bounds memoization for selectTest/getKeyRange, and force a linear scan for restricted getValueRange when keys are scattered (histogram contract).
• Add tests for the optimized range behavior and for several prior ASan-identified lifetime/race scenarios (scheduler shutdown, queued job after delete, waterfall snapshot semantics).

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/auto/test-waterfall/test-waterfall.h	Adds new waterfall snapshot/race regression test declarations.
tests/auto/test-waterfall/test-waterfall.cpp	Adds tests ensuring parameter changes apply immediately and concurrent changes are safe.
tests/auto/test-scatter-rhi/test-scatter-rhi.cpp	Avoids dangling spans by switching helper to owning setData(std::move(...)).
tests/auto/test-pipeline/test-pipeline.h	Declares new pipeline/scheduler/Histogram2D regression tests.
tests/auto/test-pipeline/test-pipeline.cpp	Implements scheduler shutdown + colormap queued-job-after-delete tests; adds Histogram2D cache/value-range tests.
tests/auto/test-datasource2d/test-datasource2d.cpp	Drains colormap pipeline to avoid async use-after-return in tests.
tests/auto/test-datasource/test-datasource.h	Declares new valueRange key-restriction equivalence test.
tests/auto/test-datasource/test-datasource.cpp	Adds brute-force vs windowed valueRange equivalence guard test.
src/plottables/plottable-waterfall.h	Makes the waterfall adapter immutable and switches graph to rebuild adapters on changes.
src/plottables/plottable-waterfall.cpp	Implements adapter snapshot rebuild path and removes draw-time mutation.
src/plottables/plottable-histogram2d.h	Introduces memoized finite-bounds cache for histogram select/range queries.
src/plottables/plottable-histogram2d.cpp	Implements bounds caching + correct restricted getValueRange for scattered keys.
src/plottables/plottable-colormap2.h	Changes gap-threshold handling to support transform rebake on updates.
src/plottables/plottable-colormap2.cpp	Captures transform parameters by value; rebakes transform on setGapThreshold.
src/datasource/soa-multi-datasource.h	Switches keyRange to keyRangeSorted for sorted-key sources.
src/datasource/row-major-multi-datasource.h	Switches keyRange to keyRangeSorted for sorted-key sources.
src/datasource/pipeline-scheduler.h	Adds mShuttingDown flag to prevent scheduling during destruction.
src/datasource/pipeline-scheduler.cpp	Drops queued work on shutdown and rejects submissions while destructing.
src/datasource/async-pipeline.h	Replaces atomic destroyed flag with a mutex-guarded destroy gate across check+invoke.
src/datasource/async-pipeline.cpp	Sets the destroy gate under mutex; removes redundant destroyed check in deliverResult.
src/datasource/algorithms.h	Bounds key-restricted valueRange scans using findBegin/findEnd.
src/datasource/algorithms-2d.h	Switches 2D xRange to keyRangeSorted.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jeandet]

Already addressed — this is the same finding as on #33, where it was fixed by cfa000b ("clearing the source uninstalls the adapter too", reproducer clearingSourceEmptiesTheGraph), now merged on main. This PR is rebased on top, so the flagged code is no longer part of its diff.