Scenario layers: deferred census values + flex areas

[drewda]

Stacked on #381 (deferred-buffer-demographics). Retarget to main after #381 merges.

Summary

Extends the progressive-loading pattern established in #381 to the two remaining loosely-coupled scenario layers: census values and flex areas. The scenario pipeline has a tightly-coupled core (stops → routes → departures, which stays combined in /api/scenario) and independent layers that each need only small client-known inputs. Each layer now follows one pattern: a pure pass function shared by the inline pipeline and a standalone NDJSON streaming endpoint, a config flag with !== false semantics for CLI/WSDOT parity, and idempotent client-side ensureX() loaders. Census values and flex areas are deferred by default and auto-load implicitly the first time a user action needs them — no new buttons. Census values are additionally cached per aggregate layer, which also fixes a pre-existing stale-data gap where changing the aggregate layer after load re-sliced the old layer's values.

User-facing changes

Faster base queries

• The default browse query no longer loads census values or flex areas (in addition to Deferred / opt-in stop buffer demographics #381's deferred buffer demographics) — it fetches only stops, routes, departures, and feed versions.
• "Include Flex Service Areas" in Advanced Settings now defaults off and means "load with the scenario"; a tooltip explains flex loads on demand otherwise. (The Flex Services display toggle is off by default, so the inline flex load was usually wasted.)

Implicit on-demand loading

• Census values auto-load (with the loading modal) the first time any of these happens: the Stops (Aggregated) report tab is opened, Show Agg. Areas is enabled, the aggregate layer is changed, or buffer demographics are loaded.
• Flex areas auto-load the first time the Flex Services display toggle is enabled.
• Shared links arriving with showAggAreas=true or flexServicesEnabled=true load their layers automatically after the scenario arrives.

Per-layer census caching + stale-data fix

• Census values are cached per aggregate layer for the life of the scenario: switching State → County → State refetches nothing.
• Changing the aggregate layer after load now fetches that layer's actual ACS values (previously the table re-sliced the originally-loaded layer's data, leaving demographic columns stale or missing).
• Toggling any display off and back on never refetches — loaded data persists until a new query run.

Implementation details

Shared pass functions + endpoints

• src/scenario/census-values-pass.ts: runCensusValuesPass() (extracted from fetchCensusValues, bbox padding still tied to the stop buffer radius) and resolveScenarioArea() (extracted from fetchFeedVersions; resolves bbox/within from either an explicit bbox or admin-boundary geographyIds — single source of truth for the pipeline and the endpoint).
• src/scenario/flex-areas-pass.ts: runFlexAreasPass() with per-feed-version error isolation via the same TaskQueue pattern; FlexStopTimesQueryVars, FlexDepartureTuple, getSelectedDateRange, and MAX_FLEX_LOCATIONS_PER_FEED_VERSION move here (re-exported through the src/scenario barrel, so external imports are unchanged).
• server/api/census-values.post.ts and server/api/flex-areas.post.ts mirror buffer-geographies.post.ts: validate, stream NDJSON via streamCensusValues/streamFlexAreas into the same wire format.

Per-layer census cache

• ScenarioData.censusGeographies → censusGeographiesByLayer: Map<layer, Map<geoid, CensusGeographyData>>. The receiver routes each streamed entry by its .layer (already on the wire — examples and pre-feature captures replay correctly), and applyScenarioResultFilter selects the current config.aggregateLayer's map into ScenarioFilterResult.censusGeographies, so every UI consumer (report tables, choropleth, census modal, CSV) is untouched.
• tne.vue's filter watch adds aggregateLayer as a dep so cached-layer switches re-run the filter without a data change.

Config flags

• includeCensusValues?: boolean joins includeStopBufferDemographics; both plus includeFlexAreas use !== false semantics, so CLI/WSDOT callers (which never set them) keep inline behavior. The SPA sends includeCensusValues: includeStopBufferDemographics (demographics-with-scenario needs census values inline for aggregation row seeding) — derived, no new URL param.
• WSDOT analyses launched from the SPA spread the scenario config and therefore now skip the inline flex/census/buffer scenario passes — all of which they never consumed (WSDOT fetches its own census data via getGeographyData; the skipped inline census fetch is the duplicate flagged in the TODO at src/analysis/wsdot/index.ts). wsdot-cli output is unchanged.

Client composable

• useBufferRefetch generalizes to useScenarioRefetch: adds ensureCensusValues() (per-layer in-flight dedupe, abort-and-reissue on layer change mid-flight), ensureFlexAreas() (one-shot with a loaded flag so legitimately-empty results aren't refetched), and resetLayerState() (called at fetch start and completion; seeds loaded-layer bookkeeping from the data itself, covering inline loads, examples, and pre-feature captures uniformly).
• Buffer loadNow() runs census + buffer concurrently into the shared receiver (disjoint accumulator fields) and owns the modal around the pair so the faster stream doesn't hide it early.
• The loading modal's progress percentage guards against a zero total (census/flex-only sessions emit no numeric progress counters).

Docs

• docs/data-flow.md gains a "Scenario layers" section: the core-vs-layers dependency graph, the pass/endpoint/flag/ensure pattern, the trigger table, and the caching semantics.

Tests

• census-values-pass.test.ts: emit shape, bbox padded iff radius > 0, layer stamping, resolveScenarioArea bbox/geographyIds/MultiPolygon paths.
• flex-areas-pass.test.ts: empty-input no-op, per-feed-version error isolation.
• scenario.test.ts: census pass runs when the flag is undefined (CLI parity) and skips when false; receiver per-layer merge independence; filter layer selection.

User test plan

• Run a default browse query with the network tab open: the /api/scenario stream contains no census or flex data and completes faster.
• Open Reports → Stops (Aggregated): the loading modal appears, one /api/census-values request fires, and the table populates including stop-less geographies. Then enable Show Agg. Areas: the choropleth renders with no additional request.
• Change Aggregate by to County: one fetch, demographic columns show county ACS values. Switch back to Tract: no request (cache). Change layers twice quickly: only the final layer's data lands.
• Click "Load stop buffer demographics" on a freshly loaded scenario: one modal session issues both /api/census-values and /api/buffer-geographies; the aggregation table seeds full rows with apportioned columns.
• Enable the Flex Services toggle in a region with flex service (e.g. Bend, OR): the modal appears, /api/flex-areas fires once, areas render on the map and the Flex Areas report tab; toggling off and on again does not refetch.
• Check both Advanced Settings checkboxes and re-run: census, flex, and buffer data all ride the main fetch; no on-demand requests fire afterwards.
• Open a shared link with showAggAreas=true: census values load automatically once the scenario arrives.
• Run a WSDOT analysis from the Analysis tab: results match pre-PR output.

🤖 Generated with Claude Code

[cloudflare-workers-and-pages]

Deploying calact-network-analysis-tool with    Cloudflare Pages

Latest commit:	be3efb3
Status:	✅  Deploy successful!
Preview URL:	https://3466434d.calact-network-analysis-tool.pages.dev
Branch Preview URL:	https://scenario-layers.calact-network-analysis-tool.pages.dev

View logs