FIx for BSP benchmark aux counters (exportedSpans/droppedSpans) always reporting zero#8539
Open
EvgeniiR wants to merge 4 commits into
Open
FIx for BSP benchmark aux counters (exportedSpans/droppedSpans) always reporting zero#8539EvgeniiR wants to merge 4 commits into
EvgeniiR wants to merge 4 commits into
Conversation
BatchSpanProcessorMetrics.getMetric() used stringKey("dropped") to
filter metric points, but BatchSpanProcessor records the attribute with
booleanKey("dropped"). The filter never matched, so exportedSpans and
droppedSpans always returned 0 in all BSP benchmarks since open-telemetry#3017.
- Fix stringKey → booleanKey in BatchSpanProcessorMetrics
- Remove dropRatio() — ratio does not aggregate correctly across JMH
thread states
- Switch BatchSpanProcessorDroppedSpansBenchmark to Type.EVENTS to
avoid integer truncation to zero in tight-loop (millions of ops/iter
collapse per-op long counters even after the key-type fix)
- Derive numThreads from BenchmarkParams.getThreads() in @setup instead
of hardcoding in each benchmark method body
… method BenchmarkParams.getThreads() injected into @setup(Level.Iteration) correctly returns the per-method @threads(N) value. Verified empirically: ratio (exportedSpans + droppedSpans) / (primary_ops_per_sec × iteration_time) ≈ 1.0 with 5 threads. Removes the fragile manual coupling between @threads(N) and numThreads = N scattered across each benchmark method body.
…chmark BatchSpanProcessorMultiThreadBenchmark had the same two issues as the other benchmarks fixed in the previous commit: - numThreads hardcoded in each method body (fragile; same BenchmarkParams fix) - Type.OPERATIONS reports spans/s (same unit as primary, drop rate not obvious); Type.EVENTS reports absolute iteration totals with clear drop rates
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8539 +/- ##
============================================
+ Coverage 90.93% 90.97% +0.03%
+ Complexity 10209 10208 -1
============================================
Files 1013 1013
Lines 27175 27170 -5
Branches 3184 3182 -2
============================================
+ Hits 24712 24717 +5
+ Misses 1735 1729 -6
+ Partials 728 724 -4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Add BatchSpanProcessorMetrics.dropRatio(int numIterations), adjusting AggregationPolicy.SUM (numThreads * numIterations) of Type.EVENTS. Use it in DroppedSpansBenchmark and MultiThreadBenchmark, restoring the dropRatio aux counter. CpuBenchmark stays on Type.OPERATIONS, where the drop rate is read directly and this method does not apply.
EvgeniiR
force-pushed
the
bsp-benchmark-aux-counters
branch
from
89e693c to
47db02e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Closes issue #8538 (BatchSpanProcessorDroppedSpansBenchmark, BatchSpanProcessorMultiThreadBenchmark, and BatchSpanProcessorCpuBenchmark were reporting exportedSpans and droppedSpans auxiliary counters as 0)
AuxCounters.TypefromOPERATIONStoEVENTSinDroppedSpansBenchmarkandMultiThreadBenchmarkto fix misleading throughput-normalized counters and make drop rates and totals directly interpretable.numThreadsvariable)BatchSpanProcessorDroppedSpansBenchmark impact
Running
BatchSpanProcessorDroppedSpansBenchmarkon main produces (measured with-PjmhIterations=1 -PjmhTime=5; the benchmark default is 20s):With the fix applied (measured with
-PjmhIterations=1 -PjmhTime=5):Type.EVENTSaux counters report the total event count for the measurement iteration,not a per-second rate.
BatchSpanProcessorMultiThreadBenchmark impact
MultiThreadBenchmarkshares the sameBatchSpanProcessorMetricshelper, so it carried thesame stringKey bug — all counters reported
≈ 0on main.Before fix (all thread variants, measured with
-PjmhIterations=1 -PjmhTime=5):After fix —
Type.EVENTS+BenchmarkParams(measured with-PjmhIterations=1 -PjmhTime=5).With
Type.EVENTS, aux counters report absolute totals per iteration rather than a per-secondrate, so
dropRatio = droppedSpans / (exportedSpans + droppedSpans)is directly readable:BatchSpanProcessorCpuBenchmark impact
CpuBenchmarkkeepsType.OPERATIONS— each operation submits exactly one span andLockSupport.parkNanos(100_000)limits throughput to ~10k spans/sec per thread, so per-secondvalues are readable integers. The stringKey bug caused all counters to report
≈ 0:Before fix (all thread variants):
After fix (
Type.OPERATIONSin throughput mode reports the counter rate in the same units asthe primary metric — spans/s, not spans/op. Since each operation submits exactly one span,
exportedSpans ≈ primary_thrpt, scaling linearly with thread count):No drops are expected —
CpuBenchmarkmeasures export throughput at a controlled low rate(~10k spans/s per thread via
parkNanos), not queue saturation.Type.OPERATIONSis thenatural unit here: the per-second rate is what a CPU profiler run reports.
Details
stringKey("dropped")→booleanKey("dropped")inBatchSpanProcessorMetrics— fixes the main issueAuxCounters.Type.OPERATIONS→Type.EVENTSinBatchSpanProcessorDroppedSpansBenchmarkand
BatchSpanProcessorMultiThreadBenchmark—Type.OPERATIONSin throughput mode reportscounter / time, soexportedSpansanddroppedSpansappear inops/s. This makes it impossible to read absolute totals.Type.EVENTSreports raw iteration totals, from whichdrop ratio = droppedSpans / (exportedSpans + droppedSpans).numThreadsderived fromBenchmarkParams.getThreads()injected into@Setup(Level.Iteration)in all three benchmarks instead of being hardcoded in each benchmark method body — removes
the fragile manual coupling between
@Threads(N)andnumThreads = N(verified:getThreads()correctly returns the per-method thread count, not the JMH global default)