feat(agent-data-plane): added support for telemetry.dogstatsd_origin config key and document dogstatsd.* as not planned

[lucastemb]

Summary

telemetry.dogstatsd.* config keys are not applicable to ADP, and thus, won't be implemented. These config keys define static bucket bounds for histograms intended to capture latency; however, ADP employs dynamic, logarithmically calculated bounds.

telemetry.dogstatsd_origin was implemented by threading the resolved origin as the dogstatsd.processed metric. If there is no origin, it is represented as "", like the Core Agent, but never omitted.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

How did you test this PR?

Unit testing.

References

• Closes: Investigate DogStatsD internal telemetry config: telemetry.dogstatsd.* and telemetry.dogstatsd_origin. #1679

[pr-commenter]

Regression Detector (Agent Data Plane)

Run ID: 0d7f5a1d-a5be-4755-96ac-a4212e762f39
Baseline: 2dd3eabd · Comparison: 9bb344da · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (5)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment	goal	Δ mean %	links
quality_gates_rss_dsd_ultraheavy	memory	⚪ +0.04	metrics profiles logs
quality_gates_rss_dsd_heavy	memory	⚪ -0.02	metrics profiles logs
quality_gates_rss_idle	memory	⚪ -0.18	metrics profiles logs
quality_gates_rss_dsd_low	memory	⚪ -0.27	metrics profiles logs
quality_gates_rss_dsd_medium	memory	⚪ -0.58	metrics profiles logs

Bounds Checks: ✅ Passed (5)

experiment	check	replicates	observed	links
quality_gates_rss_dsd_heavy	memory_usage	10/10	✅ 132 MiB ≤ 140 MiB	metrics profiles logs
quality_gates_rss_dsd_low	memory_usage	10/10	✅ 42.1 MiB ≤ 50 MiB	metrics profiles logs
quality_gates_rss_dsd_medium	memory_usage	10/10	✅ 65.5 MiB ≤ 75 MiB	metrics profiles logs
quality_gates_rss_dsd_ultraheavy	memory_usage	10/10	✅ 192 MiB ≤ 200 MiB	metrics profiles logs
quality_gates_rss_idle	memory_usage	10/10	✅ 28.1 MiB ≤ 40 MiB	metrics profiles logs

Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

[pr-commenter]

Binary Size Analysis (Agent Data Plane)

Baseline: 2dd3eab · Comparison: 9bb344d · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 40.54 MiB (baseline) vs 40.54 MiB (comparison)
Size Change: -2.92 KiB (-0.01%)

✅ Binary size difference within threshold

Changes by Module

Module	File Size	Symbols
figment	+89.32 KiB	146
core	-52.40 KiB	1877
tokio	-35.83 KiB	799
anon.c631d4f28c2b2db28eff0b5e986a13c9.248.llvm.11665572444932606208	+16.78 KiB	1
anon.db13c838dff0ff8ee9d7f038b638fedf.281.llvm.9875580372017412076	-16.69 KiB	1
saluki_components::common::datadog	-16.50 KiB	75
anon.b6d0d2e272a8b09c819cd1fb068ea72c.19.llvm.5667897768182752762	+15.33 KiB	1
anon.661ef608d0e9e2423c9230617f2bc873.62.llvm.10199967635082326453	-15.25 KiB	1
anyhow	-15.07 KiB	201
anon.a9030c24ed0837a8b572f3c7e56188a8.26.llvm.9410433398480454732	+13.90 KiB	1
anon.cd318cd9cfe68e9e5ea889ef80898e31.51.llvm.2198595051969070510	-13.82 KiB	1
saluki_components::sources::dogstatsd	+12.44 KiB	81
chrono	+10.87 KiB	4
alloc	-10.62 KiB	167
rmp	+9.92 KiB	21
&mut rmp_serde	-8.99 KiB	8
hashbrown	+8.67 KiB	106
std	-8.51 KiB	84
anon.77d0d1ecb3d3a439681fa998b99095e3.101.llvm.14984887614861076458	-8.15 KiB	1
anon.77d0d1ecb3d3a439681fa998b99095e3.100.llvm.372543674685518203	+8.10 KiB	1

Detailed Symbol Changes

    FILE SIZE        VM SIZE    
 --------------  -------------- 
  +0.0% +2.30Ki  +0.8% +52.0Ki    [11673 Others]
  [NEW] +24.6Ki  [NEW] +24.3Ki    _<saluki_components::sources::dogstatsd::_::<impl serde_core::de::Deserialize for saluki_components::sources::dogstatsd::DogStatsDConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::ha62e8990e89ad88a
  [NEW] +22.0Ki  [NEW] +21.7Ki    _<saluki_components::sources::dogstatsd::DogStatsDConfiguration as saluki_core::components::sources::builder::SourceBuilder>::build::_{{closure}}::h9a8612fad792fd76
  [NEW] +16.8Ki  [NEW]     +76    anon.c631d4f28c2b2db28eff0b5e986a13c9.248.llvm.11665572444932606208
  [NEW] +15.3Ki  [NEW]     +81    anon.b6d0d2e272a8b09c819cd1fb068ea72c.19.llvm.5667897768182752762
  [NEW] +15.1Ki  [NEW] +15.0Ki    _<tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::hcdc826e9d2a9c03b
  [NEW] +13.9Ki  [NEW]     +82    anon.a9030c24ed0837a8b572f3c7e56188a8.26.llvm.9410433398480454732
  [NEW] +11.6Ki  [NEW] +11.5Ki    saluki_components::sources::dogstatsd::replay::writer::run_capture_loop::h7b3257eb6d549e40
  [NEW] +11.2Ki  [NEW] +11.1Ki    _<figment::value::magic::RelativePathBuf as figment::value::magic::Magic>::deserialize_from::he69c481f2172058f
  [NEW] +10.7Ki  [NEW] +10.5Ki    _<chrono::format::formatting::DelayedFormat<I> as core::fmt::Display>::fmt::h99918175052c1f93
  [NEW] +10.5Ki  [NEW] +10.4Ki    saluki_components::sources::dogstatsd::handle_frame::hc0a943a6ec661213
 -68.6% -11.6Ki -69.1% -11.6Ki    saluki_components::transforms::trace_obfuscation::sql::obfuscate_sql_string::h0498423d6323202d
 -98.1% -12.0Ki -98.9% -12.0Ki    std::sys::backtrace::__rust_begin_short_backtrace::hf89fc19d327aa0da
  [DEL] -12.0Ki  [DEL] -11.9Ki    _<T as alloc::string::SpecToString>::spec_to_string::hba7a93f2df0bbb14
  [DEL] -13.8Ki  [DEL]     -82    anon.cd318cd9cfe68e9e5ea889ef80898e31.51.llvm.2198595051969070510
  [DEL] -14.7Ki  [DEL] -14.5Ki    saluki_components::common::datadog::apm::_::_<impl serde_core::de::Deserialize for saluki_components::common::datadog::apm::ApmConfiguration>::deserialize::h12bf12573cebd9fa
  [DEL] -15.0Ki  [DEL] -14.9Ki    _<tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::hbbd3eed83490b00f
  [DEL] -15.2Ki  [DEL]     -81    anon.661ef608d0e9e2423c9230617f2bc873.62.llvm.10199967635082326453
  [DEL] -16.7Ki  [DEL]     -76    anon.db13c838dff0ff8ee9d7f038b638fedf.281.llvm.9875580372017412076
  [DEL] -20.8Ki  [DEL] -20.6Ki    _<saluki_components::sources::dogstatsd::DogStatsDConfiguration as saluki_core::components::sources::builder::SourceBuilder>::build::_{{closure}}::h956cbcd87e19f905
  [DEL] -24.9Ki  [DEL] -24.6Ki    _<saluki_components::sources::dogstatsd::_::<impl serde_core::de::Deserialize for saluki_components::sources::dogstatsd::DogStatsDConfiguration>::deserialize::__Visitor as serde_core::de::Visitor>::visit_map::h6b8cf9810393c85f
  -0.0% -2.92Ki  +0.1% +46.4Ki    TOTAL

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a2c38265a5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[webern]

AI aided, seems like maybe there's a bug with evicting from the cache. Otherwise looks good.

[webern]

Below is what sonnet considers a bug. I have to admit I don't know this domain at all, so I hope it's not just wrong or something, but it's saying that the deletion method you added misses the key and the cache grows unbounded. This probably wouldn't be an issue in most environments but if there are 10's of thousands of container IDs churning, perhaps it could be:

---

origin_processed_metric_key builds a key with only 3 tags (message_type, listener_type, origin), but the counters were registered through MetricsBuilder, which prepends component_id and component_type as default tags. The keys don't match, so delete_counter always misses and the registry grows unbounded.

Fix: store the correct key at registration time rather than reconstructing it at eviction. Add a make_key method to MetricsBuilder that mirrors register_counter_with_tags but returns the Key:

// in saluki-metrics MetricsBuilder
pub fn make_key<I, T>(&self, metric_name: &'static str, additional_tags: I) -> Key
where I: IntoIterator<Item = T>, T: MetricTag,
{
    let mut tags = (*self.default_tags).clone();
    tags.extend(additional_tags.into_iter().map(MetricTag::into_label));
    Key::from_parts(metric_name, tags)
}

Then extend OriginMetricCounters to carry ok_key: Key and error_key: Key, populate them alongside the counters in get_or_create, and replace these two delete_counter calls with delete_counter(&evicted.ok_key) / delete_counter(&evicted.error_key). The two free functions (origin_processed_metric_key, origin_processed_error_key) can then be removed.

[lucastemb]

Good catch. Seems like Sonnet was right. I just put up the fix in my newest commit.

[webern]

Sonnet: Verified parity with Core Agent: dogstatsd.processed for events and service checks is declared with three label dimensions (message_type, state, origin) in server.go:283-284, and both paths always call tlmProcessed.Inc("events"/"service_checks", "ok"/"error", "") with a hardcoded empty string for origin. So Core Agent also always emits origin:"" on these series — origin:"" here is correct and consistent. 👍