Skip to content

fix(diagnostics): read the warmup advisory's trend criterion from the autocorrelation-corrected t-statistic - #228

Merged
MaartenMarsman merged 4 commits into
developfrom
fix/warmup-advisory-corrected-slope
Nov 7, 2026
Merged

MaartenMarsman merged 4 commits into
developfrom
fix/warmup-advisory-corrected-slope

Conversation

@MaartenMarsman

@MaartenMarsman MaartenMarsman commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

The warmup advisory now uses the energy trend statistic adjusted for autocorrelation. The previous standard error assumed independent draws and could trigger warnings from ordinary sampling variation.

slope_significant_corrected records the trend criterion used by warmup_incomplete. The existing slope_significant remains available for comparison and is used if the corrected statistic is unavailable. The E-BFMI and variance-ratio criteria and the printed advisory are unchanged.

The vignette and NEWS explain the adjustment briefly. A large energy_tau indicates strong autocorrelation; it does not establish incomplete warmup. The documentation recommends checking R-hat, ESS, and the other sampling diagnostics together.

Before and after

Eleven NUTS fits (~/bgms-review/advisory-fits.R, table in advisory-fits.txt): the companion tutorial's worked example (Wenchuan complete cases, 4 chains, 10000 sampling and 5000 warmup iterations) and its 0.5x refit, three Wenchuan fits at the default run length with seeds 1 to 3, the vignette's six-variable fit, two ADHD fits, two Boredom fits, one bgmCompare fit on Wenchuan.

rule fits with the advisory
ordinary least-squares (before) 8 of 11
corrected (after) 1 of 11

The fit that keeps it is Boredom at the default run length, seed 2: corrected t of 3.66 in chain 4, with largest R-hat 1.085 and smallest ESS 78. The worked example loses it: chain 1 has an uncorrected t of 4.5 and a corrected t of 1.99, with energy_tau 5.1, E-BFMI 0.86 and variance ratio 1.08.

Four fits at the default run length (the three Wenchuan seeds and Boredom seed 1) have largest R-hat between 1.027 and 1.032 and smallest ESS between 172 and 347, and lose the advisory. Their corrected t-statistics are all below 1.6: their energy traces show no trend. What they show is slow mixing, which R-hat and ESS report. The advisory now speaks to warmup only.

Verification

The three tests added for the rule change were removed again; the existing field-contract tests list the new field. tests/testthat/test-warmup-check.R passes with 41 expectations, test-mcmc-diagnostics.R 175, test-warmup-schedule.R 15, test-prior-sensitivity.R 101. R CMD check --as-cran on the implementation commit: 0 errors, 0 warnings, 0 notes; the later commits change tests, comments and documentation only. lintr and styler clean. Behavioural comparison develop vs branch: dev/review-2026-08/repro/advisory-demos.R and advisory-compare.R (run by run-advisory.sh), output in ~/bgms-review/advisory-comparison.txt: on the worked example's stored energy traces develop flags chain 1 and the branch flags none, with identical corrected statistics underneath.

… autocorrelation-corrected t-statistic

check_warmup_complete() flags a chain's energy trend when the t-statistic
with its standard error widened by sqrt(energy_tau) exceeds 2.58 in
absolute value; where energy_tau cannot be estimated it uses the ordinary
least-squares statistic as before. The new field slope_significant_corrected
is that criterion; slope_significant keeps the ordinary least-squares
statistic for comparison. The E-BFMI and variance-ratio criteria are
unchanged.

The diagnostics vignette describes the corrected criterion and no longer
presents the uncorrected one as the flag's rule.
…erion; the field contract tests cover the new field
@MaartenMarsman
MaartenMarsman merged commit 511cc0d into develop Nov 7, 2026
0 of 8 checks passed
@MaartenMarsman
MaartenMarsman deleted the fix/warmup-advisory-corrected-slope branch November 7, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant