Skip to content

fix(sensitivity): use verdicts() flags and clarify diagnostic reporting - #227

Merged
MaartenMarsman merged 3 commits into
developfrom
fix/sensitivity-flag-and-saturation
Nov 7, 2026
Merged

MaartenMarsman merged 3 commits into
developfrom
fix/sensitivity-flag-and-saturation

Conversation

@MaartenMarsman

@MaartenMarsman MaartenMarsman commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

The sensitivity check now uses verdicts() to flag near-threshold verdicts in the original fit. Its fragile column replaces insufficient and the two subcause columns. The fourth printed count also includes changed verdicts whose Monte Carlo error is unavailable; the report states how many additional edges it includes.

The saturated column now records an unavailable chosen-scale MCSE. This does not necessarily mean the inclusion probability is exactly 0 or 1. Summary notes and documentation explain blank diagnostics without treating them as evidence of zero error.

Fits with infinite split R-hat are named, with advice to check convergence before interpreting their verdicts. The help page explains how to increase warmup iterations. The R-hat calculation is unchanged from develop; regression tests protect against losing convergence warnings when parameter units change.

The printed notes, help pages, NEWS, and vignettes use shorter explanations. The Refits: line capitalizes NUTS and describes default initial values without “cold start” terminology.

Validation: test-mcmc-diagnostics.R, test-prior-sensitivity.R (including slow and certification tiers), test-summary-marking.R, and test-verdicts.R pass. Lintr, package styler, and git diff --check pass. Documentation regenerated and the summary snapshot updated.

… one saturation rule for the summary cells and the check

prior_sensitivity_check() takes its chosen-scale flag from verdicts() on
the original fit, so the two reports count the same edges. The
between-chain band and the per-chain verdict test are gone, as are the
$edges columns insufficient, insufficient_noisy and insufficient_disagree;
the flag is the fragile column. The printed count labels are "verdict held
at every scale", "changed within Monte Carlo error", "changed beyond Monte
Carlo error" and "could change on a rerun"; the sentence under them names
the flagged edges and states that their log Bayes factors lie within two
standard errors of an evidence threshold.

The split-R-hat kernel applies the ESS kernel's variance floor: a
sub-chain with biased variance below 1e-15 counts as constant, so a
Rao-Blackwellized inclusion series constant to numerical precision has NA
for mcse, n_eff and Rhat together. The check's saturated column is the
same criterion (the chosen-scale MCSE is NA). The note under the summary
tables and in the check says "0 or 1 in every draw of every chain, or
constant to numerical precision"; the Details of summary.bgms,
summary.bgmCompare and the diagnostics vignette explain why the series is
constant.

The sentence for a fit with an infinite split-R-hat says the sampler did
not move and that the fit's verdicts do not describe the posterior
distribution at that scale; the help page states that rhat_max covers the
continuous parameters only and what such a refit needs. The Refits line
prints NUTS. ?verdicts states that an indicator whose standard errors are
undefined is not flagged. The vignette's saved check object is regenerated.
…t the blank-cell and infinite-R-hat wording

The split-R-hat kernel is unchanged from develop: a sub-chain counts as
constant only when its draws are exactly equal, so the Rhat of an inclusion
series that is constant to numerical precision is computed, as before. The
saturated column of prior_sensitivity_check() stays the edges whose
chosen-scale Monte Carlo standard error is NA.

The Details of summary.bgms and summary.bgmCompare and the diagnostics
vignette say that mcse and n_eff are NA when the series has no usable
variation and that Rhat is NA only for exactly constant chains; the recorded
value for an included edge is the probability of retaining it; the cutoff
is a variation of about 3e-8, not a removal probability of 1e-8.

The infinite-R-hat sentence states what the diagnostic shows: a parameter
constant within every split chain at values that differ between split
chains, which the sampler did not explore. The help page says such a refit
may need a longer warmup or a cold start. NEWS states that the fourth count
is the verdicts() flag plus the edges whose verdict change could not be
compared with an error.
@MaartenMarsman MaartenMarsman changed the title fix(sensitivity): the check's fourth count is verdicts()' flag; one saturation rule for the summary cells and the check fix(sensitivity): use verdicts() flags and clarify diagnostic reporting Nov 7, 2026
@MaartenMarsman
MaartenMarsman merged commit 31a8373 into develop Nov 7, 2026
0 of 8 checks passed
@MaartenMarsman
MaartenMarsman deleted the fix/sensitivity-flag-and-saturation branch November 7, 2026 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant