Skip to content

feat(config): add dynamic typed-config router for prefix filter#1919

Draft
webern wants to merge 4 commits into
m/pr-5-arch-testfrom
m/pr-6-dynamic
Draft

feat(config): add dynamic typed-config router for prefix filter#1919
webern wants to merge 4 commits into
m/pr-5-arch-testfrom
m/pr-6-dynamic

Conversation

@webern

@webern webern commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Human Summary

This migrates a second configuration struct, DogStatsDPrefixFilterConfiguration to the typed configuration system. This one is dynamic and serves as a forcing function to get typed dynamic config updates working.

AI Summary

Adds PrefixFilterConfig as a source-agnostic native struct in saluki-component-config and
implements a dynamic config router in agent-data-plane-config-system that subscribes to raw-map
update events, re-translates the full Datadog config, atomically swaps the current
SalukiConfiguration via ArcSwap, and publishes changed slices through DynamicConfig<T> watch
handles.

Migrates the DogStatsD prefix filter component from raw-key watch_for_updates() watchers +
GenericConfiguration to a single DynamicConfig<PrefixFilterConfig> handle. The 4-arm select!
loop (one per watched key) becomes a single changed() arm that rebuilds the EffectiveFilterlist
from a typed snapshot.

Fills the 6 no-op witness consume_* methods for the prefix filter keys so translation populates
PrefixFilterConfig fields.

Change Type

  • New feature

How did you test this PR?

  • 6 unit tests for the dynamic router (update wake, duplicate suppression, unrelated key, malformed
    recovery, fixed mode, ArcSwap visibility)
  • 2 translation tests (all 6 prefix filter fields, legacy _blacklist alias)
  • Rewritten component tests using DynamicConfig::fixed() and DynamicConfig::live()
  • Panoramic integration test: set metric_filterlist via control plane, verify on
    /config/internal
  • make check-all (clippy, Vale, unused-deps)

References

@webern webern changed the title chore(config): add PrefixFilterConfig native struct feat(config): add dynamic typed-config router for prefix filter Jun 24, 2026
@dd-octo-sts dd-octo-sts Bot added the area/test All things testing: unit/integration, correctness, SMP regression, etc. label Jun 24, 2026

webern commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

@datadog-official

datadog-official Bot commented Jun 24, 2026

Copy link
Copy Markdown

Pipelines

Fix all issues with BitsAI

⚠️ Warnings

🚦 10 Pipeline jobs failed

DataDog/saluki | binary-size-analysis   View in Datadog   GitLab

DataDog/saluki | test-integration   View in Datadog   GitLab

DataDog/saluki | test-integration-macos-amd64   View in Datadog   GitLab

View all 10 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 97e505d | Docs | Datadog PR Page | Give us feedback!

webern added 4 commits June 24, 2026 17:18
Add the source-agnostic PrefixFilterConfig leaf type in
saluki-component-config and embed it in the DogStatsD domain
group. Fill the six Datadog witness methods that were no-ops
and add translation tests.
Introduce ArcSwap<SalukiConfiguration> in ConfigurationSystem so
the dynamic router can atomically swap retranslated configs.
Add DynamicConfigHandles with a ScopedConfig<PrefixFilterConfig>
handle that wakes on accepted runtime updates.
Rename DogStatsDPrefixFilterConfiguration to
DogStatsDPrefixFilterBuilder. Replace the GenericConfiguration
raw-key watchers with a single ScopedConfig<PrefixFilterConfig>
handle. Wire through add_dsd_pipeline_to_blueprint using
system.dynamic_handles().prefix_filter.
Panoramic test that exercises the full dynamic typed-config
pipeline: Core Agent config set -> GenericConfiguration ->
router retranslation -> /config/internal serves the updated
prefix_filter slice.
@pr-commenter

pr-commenter Bot commented Jun 24, 2026

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: cf8aca1f-9916-492c-b85b-eb92a877a756
Baseline: 9c1abdeb · Comparison: 97e505de · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (35)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
dsd_uds_1mb_3k_contexts_cpu (erratic) cpu ⚪ +18.56 metrics profiles logs
dsd_uds_512kb_3k_contexts_cpu (erratic) cpu ⚪ +13.28 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_cpu (erratic) cpu ⚪ +4.67 metrics profiles logs
quality_gates_rss_idle memory ⚪ +3.89 metrics profiles logs
dsd_uds_10mb_3k_contexts_cpu (erratic) cpu ⚪ +3.59 metrics profiles logs
dsd_uds_1mb_3k_contexts_memory memory ⚪ +3.19 metrics profiles logs
dsd_uds_100mb_3k_contexts_memory memory ⚪ +2.81 metrics profiles logs
dsd_uds_10mb_3k_contexts_memory memory ⚪ +2.72 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ +2.53 metrics profiles logs
dsd_uds_500mb_3k_contexts_memory memory ⚪ +2.31 metrics profiles logs
dsd_uds_512kb_3k_contexts_memory memory ⚪ +2.22 metrics profiles logs
dsd_uds_500mb_3k_contexts_throughput throughput ⚪ -1.81 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ +1.78 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_memory memory ⚪ +1.45 metrics profiles logs
otlp_ingest_traces_5mb_memory memory ⚪ +1.39 metrics profiles logs
dsd_uds_100mb_3k_contexts_cpu (erratic) cpu ⚪ +1.32 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_memory memory ⚪ +1.24 metrics profiles logs
otlp_ingest_logs_5mb_memory (ignored) memory ⚪ +0.96 metrics profiles logs
otlp_ingest_metrics_5mb_memory memory ⚪ +0.90 metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory ⚪ +0.71 metrics profiles logs
quality_gates_rss_dsd_heavy memory ⚪ +0.51 metrics profiles logs
otlp_ingest_traces_5mb_cpu (erratic) cpu ⚪ +0.40 metrics profiles logs
otlp_ingest_traces_ottl_transform_5mb_throughput throughput ⚪ -0.11 metrics profiles logs
dsd_uds_512kb_3k_contexts_throughput throughput ⚪ -0.01 metrics profiles logs
dsd_uds_1mb_3k_contexts_throughput throughput ⚪ -0.00 metrics profiles logs
dsd_uds_10mb_3k_contexts_throughput throughput ⚪ +0.00 metrics profiles logs
otlp_ingest_metrics_5mb_throughput throughput ⚪ +0.00 metrics profiles logs
otlp_ingest_metrics_5mb_cpu (erratic) cpu ⚪ -0.02 metrics profiles logs
dsd_uds_100mb_3k_contexts_throughput throughput ⚪ +0.02 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_cpu (erratic) cpu ⚪ -0.04 metrics profiles logs
otlp_ingest_logs_5mb_throughput (ignored) throughput ⚪ +0.05 metrics profiles logs
otlp_ingest_traces_5mb_throughput throughput ⚪ +0.05 metrics profiles logs
otlp_ingest_traces_ottl_filtering_5mb_throughput throughput ⚪ +0.06 metrics profiles logs
dsd_uds_500mb_3k_contexts_cpu (erratic) cpu ⚪ -0.51 metrics profiles logs
otlp_ingest_logs_5mb_cpu (ignored) cpu ⚪ -0.91 metrics profiles logs
Bounds Checks: ✅ Passed (5)
experiment check replicates observed links
quality_gates_rss_dsd_heavy memory_usage 10/10 ✅ 131 MiB ≤ 140 MiB metrics profiles logs
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 43.4 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 65.2 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_dsd_ultraheavy memory_usage 10/10 ✅ 191 MiB ≤ 200 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 29.3 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/test All things testing: unit/integration, correctness, SMP regression, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant