diff --git a/NEWS.md b/NEWS.md index 5c86bb56..3b43d3fd 100644 --- a/NEWS.md +++ b/NEWS.md @@ -165,9 +165,17 @@ released from this line yet. ## Diagnostics -* The printed `summary()` of a `bgm()` or `bgmCompare()` fit no longer shows - the directional flip counts `n0->1` and `n1->0` in its inclusion table. The - columns stay in `summary(fit)$indicator`. +* The pairwise summary table for a `bgm()` fit and the difference summary + tables for a `bgmCompare()` fit no longer include `share_incl`, the proportion + of Monte Carlo variance attributed to uncertainty in inclusion. The reported + effective sample sizes (`n_eff`) are unchanged. + +* The inclusion tables of `summary()` for `bgm()` and `bgmCompare()` fits no + longer include the directional transition counts `n0->1` and `n1->0`, either + in the returned object or in the printed output. Their diagnostic columns + are `mean`, `mcse`, `sd`, `n_eff`, and `Rhat`. The co-clustering table under + the stochastic block prior also drops all four transition counts and + `n_eff_mixt`; its diagnostic columns are `mean`, `mcse`, `sd`, and `Rhat`. * The printed `verdicts()` table closes with the number of verdicts that appear robust to Monte Carlo error, followed by the number flagged because @@ -230,6 +238,11 @@ released from this line yet. ## Deprecations +* `extract_ess(estimator = "mixt")`, the transition-based effective sample size + deprecated in 0.2.0.0, is defunct: the call errors. The transition-based + estimate is no longer computed; `estimator = "rb"`, the default, returns the + effective sample size of the Rao-Blackwellized inclusion draws. + * The `step_size` argument of `sample_ggm_prior()` is deprecated and ignored: the NUTS warmup sets its own initial step size. The returned `step_size` field has been removed. diff --git a/R/extractor_functions.R b/R/extractor_functions.R index 4da42b6f..ac21fc7d 100644 --- a/R/extractor_functions.R +++ b/R/extractor_functions.R @@ -225,14 +225,13 @@ extract_indicators.bgmCompare = function(bgms_object) { #' continuous, the standard MCSE/ESS/split-R-hat machinery applies to it, and #' the fit summary's inclusion table reports `mcse`, `n_eff`, and `Rhat` on the #' RB draws. They quantify precision *conditional on exploration*: a stuck chain -#' can show a beautifully converged `J` chain with a high `n_eff`, so read them -#' beside the per-direction flip counts (`n0->1`, `n1->0`), which record the -#' exploration itself and whose asymmetry no symmetric summary recovers. The RB -#' draws vary on almost every edge, including edges whose indicator never -#' flipped; the three columns are `NA` only where the RB draws are constant to -#' double precision, which places the inclusion probability at its numerical -#' bound and the verdict beyond any threshold. All of this is in the fit -#' summary's inclusion table, `summary(fit)$indicator`. +#' can produce apparently converged Rao-Blackwellized inclusion draws with a +#' high `n_eff`. The RB draws vary on +#' almost every edge, including edges whose indicator never flipped; the three +#' columns are `NA` only where the RB draws are constant to double precision, +#' which places the inclusion probability at its numerical bound and the +#' verdict beyond any threshold. All of this is in the fit summary's inclusion +#' table, `summary(fit)$indicator`. #' #' @param bgms_object A fitted model object of class `bgms` (from [bgm()]) #' or `bgmCompare` (from [bgmCompare()]). @@ -1359,9 +1358,7 @@ extract_pairwise_thresholds = function(bgms_object) { #' #' @details #' The `indicator` element is the split-R-hat of the Rao-Blackwellized -#' inclusion draws, matching the default of [extract_ess()]; the indicator -#' chain's transition-based effective sample size remains available as -#' `extract_ess(fit, estimator = "mixt")` (deprecated). +#' inclusion draws, matching [extract_ess()]. #' #' One failure mode these diagnostics may miss: chains can agree that an edge #' is included while disagreeing about the magnitude of its weight, for @@ -1469,76 +1466,27 @@ extract_rhat.bgmCompare = function(bgms_object) { # ------------------------------------------------------------------------------ # indicator_ess_column # ------------------------------------------------------------------------------ -# Resolve the indicator ESS that extract_ess() returns. The default "rb" is the -# continuous ESS of the Rao-Blackwellized inclusion draws, read from the summary -# table, so the NA masking summarize_rb_inclusion() applies carries over -# unchanged. The deprecated "mixt" is the indicator chain's transition-based -# ESS, no longer a summary column; it is recomputed from the raw indicator draws -# for as long as it is accepted. The deprecation warning is raised in the -# generic, where the caller is the user rather than bgms itself. +# The indicator ESS that extract_ess() returns: the continuous ESS of the +# Rao-Blackwellized inclusion draws, read from the summary table, so the NA +# masking summarize_rb_inclusion() applies carries over unchanged. Fits without +# Rao-Blackwellized draws (bgms < 0.2.0.0) carry no n_eff column and error, as +# in extract_posterior_inclusion_probabilities(). # -# Fits without Rao-Blackwellized draws (bgms < 0.2.0.0) carry no n_eff column, -# so a default call falls back to the transition ESS while an explicit -# estimator = "rb" errors, as in -# extract_posterior_inclusion_probabilities(). -# -# @param fit The fit object (for the raw indicator draws). # @param summary_indicator The fit's posterior_summary_indicator table. -# @param estimator Character: "rb" or "mixt" (or the unevaluated -# default vector). -# @param estimator_missing Logical: TRUE if the caller did not supply one. # # Returns: numeric vector of ESS values, one per indicator. # ------------------------------------------------------------------------------ -indicator_ess_column = function(fit, summary_indicator, estimator, estimator_missing) { +indicator_ess_column = function(summary_indicator) { # names(), not $, because $ partial-matches n_eff to a legacy table's # n_eff_mixt. - has_rb = "n_eff" %in% names(summary_indicator) - estimator = if(estimator_missing && !has_rb) { - "mixt" - } else { - match.arg(estimator, choices = c("rb", "mixt")) - } - if(estimator == "rb") { - if(!has_rb) { - stop( - "This fit carries no Rao-Blackwellized inclusion draws, so the ", - "Rao-Blackwellized effective sample size is unavailable. Refit with ", - "bgms >= 0.2.0.0." - ) - } - return(summary_indicator$n_eff) - } - - if("n_eff_mixt" %in% names(summary_indicator)) { - return(summary_indicator$n_eff_mixt) - } - indicator_transition_ess(fit) -} - - -# ------------------------------------------------------------------------------ -# indicator_transition_ess -# ------------------------------------------------------------------------------ -# Transition-based ESS of the binary indicator chains, recomputed from the raw -# draws. Retained only to keep the deprecated extract_ess(estimator = "mixt") -# working; the quantity is no longer a summary column. -# -# @param fit A bgms or bgmCompare object. -# -# Returns: numeric vector of transition ESS values, one per indicator. -# ------------------------------------------------------------------------------ -indicator_transition_ess = function(fit) { - cache = get_fit_cache(fit) - raw = if(is.null(cache)) NULL else cache$raw - if(is.null(raw) || is.null(raw[[1]][["indicator_samples"]])) { + if(!"n_eff" %in% names(summary_indicator)) { stop( - "This fit carries no raw indicator draws, so the transition-based ", - "effective sample size cannot be recomputed." + "This fit carries no Rao-Blackwellized inclusion draws, so the ", + "Rao-Blackwellized effective sample size is unavailable. Refit with ", + "bgms >= 0.2.0.0." ) } - ind_stats = .compute_indicator_ess_cpp(combine_chains(raw, "indicator_samples")) - unname(ind_stats[, "n_eff_mixt"]) + summary_indicator$n_eff } @@ -1550,11 +1498,11 @@ indicator_transition_ess = function(fit) { #' #' @param bgms_object A fitted model object of class `bgms` (from [bgm()]) #' or `bgmCompare` (from [bgmCompare()]). -#' @param estimator Character; which effective sample size to return for the -#' edge (or difference) indicators. `"rb"` (default) returns the ESS of the -#' Rao-Blackwellized inclusion draws. `"mixt"` -#' `r lifecycle::badge("deprecated")` returns the indicator chain's -#' transition-based ESS. Ignored for all other parameter types. +#' @param estimator Character; `"rb"`, the only accepted value, returns the ESS +#' of the Rao-Blackwellized inclusion draws for the edge (or difference) +#' indicators. `"mixt"` `r lifecycle::badge("deprecated")`, the indicator +#' chain's transition-based ESS, was deprecated in bgms 0.2.0.0 and now +#' errors. Ignored for all other parameter types. #' #' @return A named list with ESS values for each parameter type present in #' the model (e.g., `main`, `pairwise`, `indicator`). @@ -1569,36 +1517,23 @@ indicator_transition_ess = function(fit) { #' Rao-Blackwellized draws are constant to double precision, where the #' inclusion probability is at its numerical bound. #' -#' `estimator = "mixt"`, the transition-based ESS of the binary indicator chain, -#' is deprecated and no longer a summary column; a call recomputes it from the -#' raw indicator draws and warns. It converts flip counts to an effective sample -#' size through a two-state first-order Markov model that a substantial share of -#' edge chains violate, and the precision of the inclusion probability and its -#' Bayes factor is carried by the Rao-Blackwellized ESS and Monte Carlo standard -#' error. The directional flip counts themselves (`n0->1`, `n1->0`) remain in -#' the inclusion table. -#' -#' Fits made with bgms < 0.2.0.0 carry no Rao-Blackwellized draws; a default -#' call falls back to the transition ESS without warning, while an explicit -#' `estimator = "rb"` errors. +#' For older fits without Rao-Blackwellized inclusion draws, `extract_ess()` +#' stops when extracting indicator ESS. Refit with bgms 0.2.0.0 or later. #' #' @seealso [bgm()], [bgmCompare()], [extract_rhat()] for the #' Rao-Blackwellized indicator R-hat, #' [extract_posterior_inclusion_probabilities()] #' @family extractors #' @export -extract_ess = function(bgms_object, estimator = c("rb", "mixt")) { - if(!missing(estimator) && identical(estimator, "mixt")) { - lifecycle::deprecate_warn( +extract_ess = function(bgms_object, estimator = "rb") { + estimator = match.arg(estimator, choices = c("rb", "mixt")) + if(identical(estimator, "mixt")) { + lifecycle::deprecate_stop( "0.2.0.0", "extract_ess(estimator = 'no longer supports \"mixt\"')", details = paste( - "The transition-based effective sample size no longer bears on any", - "inclusion verdict: the two-state model behind it is rejected on a", - "substantial share of edge chains, and the precision of the inclusion", - "probability and its Bayes factor is carried by the Rao-Blackwellized", - "effective sample size and Monte Carlo standard error. Use the default", - "estimator = \"rb\"; the directional flip counts remain in", - "summary(fit)$indicator." + "The transition-based effective sample size is no longer available.", + "Use extract_ess(fit) to obtain the effective sample size of the", + "Rao-Blackwellized inclusion draws." ) ) } @@ -1608,8 +1543,7 @@ extract_ess = function(bgms_object, estimator = c("rb", "mixt")) { #' @inheritParams extract_ess #' @exportS3Method #' @noRd -extract_ess.bgms = function(bgms_object, estimator = c("rb", "mixt")) { - estimator_missing = missing(estimator) +extract_ess.bgms = function(bgms_object, estimator = "rb") { ensure_summaries(bgms_object) result = list() @@ -1633,10 +1567,7 @@ extract_ess.bgms = function(bgms_object, estimator = c("rb", "mixt")) { # Indicator ESS (if edge selection was used) if(!is.null(bgms_object$posterior_summary_indicator)) { - result$indicator = indicator_ess_column( - bgms_object, bgms_object$posterior_summary_indicator, - estimator, estimator_missing - ) + result$indicator = indicator_ess_column(bgms_object$posterior_summary_indicator) names(result$indicator) = rownames(bgms_object$posterior_summary_indicator) } @@ -1650,8 +1581,7 @@ extract_ess.bgms = function(bgms_object, estimator = c("rb", "mixt")) { #' @inheritParams extract_ess #' @exportS3Method #' @noRd -extract_ess.bgmCompare = function(bgms_object, estimator = c("rb", "mixt")) { - estimator_missing = missing(estimator) +extract_ess.bgmCompare = function(bgms_object, estimator = "rb") { ensure_summaries(bgms_object) result = list() @@ -1681,10 +1611,7 @@ extract_ess.bgmCompare = function(bgms_object, estimator = c("rb", "mixt")) { # Indicator ESS (if difference selection was used) if(!is.null(bgms_object$posterior_summary_indicator)) { - result$indicator = indicator_ess_column( - bgms_object, bgms_object$posterior_summary_indicator, - estimator, estimator_missing - ) + result$indicator = indicator_ess_column(bgms_object$posterior_summary_indicator) names(result$indicator) = rownames(bgms_object$posterior_summary_indicator) } diff --git a/R/mcmc_summary.R b/R/mcmc_summary.R index 98ccea99..4c06d588 100644 --- a/R/mcmc_summary.R +++ b/R/mcmc_summary.R @@ -233,12 +233,11 @@ summarize_indicator = function(fit, component = c("indicator_samples"), param_na if(is.null(array3d)) array3d = combine_chains(fit, component) nparam = dim(array3d)[3] - # Batch indicator ESS + transition counts via C++ + # Batch indicator mean and MCSE via C++ ind_stats = .compute_indicator_ess_cpp(array3d) batch_rhat = .compute_rhat_cpp(split_chains(array3d)) - result = cbind(ind_stats[, c("mean", "mcse", "sd", "n00", "n01", "n10", "n11", "n_eff_mixt"), drop = FALSE], Rhat = batch_rhat) - colnames(result)[4:7] = c("n0->0", "n0->1", "n1->0", "n1->1") + result = cbind(ind_stats[, c("mean", "mcse", "sd"), drop = FALSE], Rhat = batch_rhat) if(is.null(param_names)) { data.frame(parameter = paste0("indicator [", seq_len(nparam), "]"), result, check.names = FALSE) @@ -277,21 +276,10 @@ summarize_rb_inclusion = function(raw, param_names, keep_parameter_col = FALSE) mat[keep, ] = as.matrix(sub[, cols, drop = FALSE]) } - ind_stats = .compute_indicator_ess_cpp(combine_chains(raw, "indicator_samples")) - - full = cbind( - mat[, c("mean", "mcse", "sd", "n_eff", "Rhat"), drop = FALSE], - ind_stats[, c("n01", "n10"), drop = FALSE] - ) - # Keep the directional transition counts whole; their asymmetry is - # decision-relevant and no symmetric summary recovers it. - colnames(full)[colnames(full) == "n01"] = "n0->1" - colnames(full)[colnames(full) == "n10"] = "n1->0" - if(keep_parameter_col) { - data.frame(parameter = param_names, full, check.names = FALSE, row.names = NULL) + data.frame(parameter = param_names, mat, check.names = FALSE, row.names = NULL) } else { - out = as.data.frame(full, check.names = FALSE) + out = as.data.frame(mat, check.names = FALSE) rownames(out) = param_names out } @@ -345,9 +333,7 @@ summarize_slab = function(fit, component = c("pairwise_samples"), param_names = # inclusion-dominated cells). The composite MCSE^2 splits into an inclusion part # (mu_cond^2 * var_p, with var_p the RB J-chain MCSE^2 of the inclusion # probability) and a slab part (p_hat^2 * var_mu, from the included-only draws). -# share_incl is the inclusion part's share of the composite MCSE^2 -- the -# bottleneck: whether the Monte Carlo error is dominated by inclusion or slab -# uncertainty. Inputs may be vectors (one entry per edge). +# Inputs may be vectors (one entry per edge). derived_weight_ess = function(post_var, p_hat, var_p, mu_cond, var_mu) { # A constant / never-updated inclusion chain contributes no inclusion error. var_p[!is.finite(var_p)] = 0 @@ -357,8 +343,7 @@ derived_weight_ess = function(post_var, p_hat, var_p, mu_cond, var_mu) { ok = is.finite(mcse2) & mcse2 > 0 list( mcse = ifelse(ok, sqrt(mcse2), NA_real_), - n_eff = ifelse(ok & is.finite(post_var), post_var / mcse2, NA_real_), - share_incl = ifelse(ok, part_incl / mcse2, NA_real_) + n_eff = ifelse(ok & is.finite(post_var), post_var / mcse2, NA_real_) ) } @@ -413,7 +398,7 @@ summarize_pair = function(fit, data.frame( parameter = names_out, mean = eap, mcse = comp$mcse, sd = sqrt(post_var), - n_eff = comp$n_eff, share_incl = comp$share_incl, Rhat = rhat, + n_eff = comp$n_eff, Rhat = rhat, check.names = FALSE ) } @@ -495,12 +480,11 @@ summarize_indicator_compare = function(fit, component = "indicator_samples", par array3d = combine_chains(fit, component) nparam = dim(array3d)[3] - # Batch indicator ESS + transition counts via C++ + # Batch indicator mean and MCSE via C++ ind_stats = .compute_indicator_ess_cpp(array3d) batch_rhat = .compute_rhat_cpp(split_chains(array3d)) - result = cbind(ind_stats[, c("mean", "mcse", "sd", "n00", "n01", "n10", "n11", "n_eff_mixt"), drop = FALSE], Rhat = batch_rhat) - colnames(result)[4:7] = c("n0->0", "n0->1", "n1->0", "n1->1") + result = cbind(ind_stats[, c("mean", "mcse", "sd"), drop = FALSE], Rhat = batch_rhat) if(is.null(param_names)) { data.frame(parameter = paste0("indicator [", seq_len(nparam), "]"), result, check.names = FALSE) @@ -511,10 +495,9 @@ summarize_indicator_compare = function(fit, component = "indicator_samples", par # Summarize one effect with spike-and-slab draws. n_eff is the derived composite -# ESS for the model-averaged weight (see derived_weight_ess); share_incl is the -# inclusion part's share of the composite MCSE^2. The inclusion Monte Carlo error -# comes from the RB J-chain (draws_rb) when available, else from the binary -# indicator's transition-based MCSE. +# ESS for the model-averaged weight (see derived_weight_ess). The inclusion +# Monte Carlo error comes from the RB J-chain (draws_rb) when available, else +# from the binary indicator's transition-based MCSE. summarize_mixture_effect = function(draws_pw, draws_id, name, draws_rb = NULL) { # Handle case where single-chain extraction returns a vector # (dimension gets dropped when extracting [, , idx] from array with nchains=1) @@ -573,7 +556,6 @@ summarize_mixture_effect = function(draws_pw, draws_id, name, draws_rb = NULL) { mcse = comp$mcse, sd = sqrt(post_var), n_eff = comp$n_eff, - share_incl = comp$share_incl, Rhat = rhat, check.names = FALSE ) diff --git a/R/mcmc_summary_sbm.R b/R/mcmc_summary_sbm.R index 0954d276..ed1360b1 100644 --- a/R/mcmc_summary_sbm.R +++ b/R/mcmc_summary_sbm.R @@ -42,10 +42,9 @@ summarize_alloc_pairs = function(allocations, node_names = NULL) { batch_rhat = .compute_rhat_cpp(split_chains(array3d)) result = cbind( - ind_stats[, c("mean", "mcse", "sd", "n00", "n01", "n10", "n11", "n_eff_mixt"), drop = FALSE], + ind_stats[, c("mean", "mcse", "sd"), drop = FALSE], Rhat = batch_rhat ) - colnames(result)[4:7] = c("n0->0", "n0->1", "n1->0", "n1->1") if(is.null(node_names)) { rn = paste0(Pairs[, 1], "-", Pairs[, 2]) dimn = as.character(seq_len(no_variables)) diff --git a/R/methods_bgmcompare.R b/R/methods_bgmcompare.R index aec82921..3c4d3313 100644 --- a/R/methods_bgmcompare.R +++ b/R/methods_bgmcompare.R @@ -327,11 +327,8 @@ print.summary.bgmCompare = function(x, digits = 3, ...) { if(!is.null(x$indicator)) { cat("Inclusion probabilities:\n") - # mean/mcse/sd/n_eff/Rhat are the Rao-Blackwellized inclusion estimate. The - # directional flip counts n0->1 / n1->0 stay in summary(fit)$indicator and - # are not printed. + # mean/mcse/sd/n_eff/Rhat are the Rao-Blackwellized inclusion estimate. ind = head(x$indicator, 6) - ind = ind[, setdiff(names(ind), c("n0->1", "n1->0")), drop = FALSE] ind_has_na = anyNA(ind) @@ -407,7 +404,7 @@ print.summary.bgmCompare = function(x, digits = 3, ...) { if(!is.null(x$indicator) && maind_has_na) { cat("Note: NA values are suppressed in the print table. They occur for differences\n") - cat("that were never selected, so the composite ESS and share are undefined;\n") + cat("that were never selected, so the composite ESS is undefined;\n") cat("`summary(fit)$main_diff` still contains the NA values.\n") } cat("\n") @@ -434,7 +431,7 @@ print.summary.bgmCompare = function(x, digits = 3, ...) { if(!is.null(x$indicator) && pairwised_has_na) { cat("Note: NA values are suppressed in the print table. They occur for differences\n") - cat("that were never selected, so the composite ESS and share are undefined;\n") + cat("that were never selected, so the composite ESS is undefined;\n") cat("`summary(fit)$pairwise_diff` still contains the NA values.\n") } cat("\n") diff --git a/R/methods_bgms.R b/R/methods_bgms.R index e3d86efe..9d4bc681 100644 --- a/R/methods_bgms.R +++ b/R/methods_bgms.R @@ -171,7 +171,7 @@ print.summary.bgms = function(x, digits = 3, ...) { if(nrow(x$pairwise) > .summary_preview_rows) cat("... (use `summary(fit)$pairwise` to see full output)\n") if(!is.null(x$indicator) && pair_has_na) { cat("Note: NA values are suppressed in the print table. They occur for edges that\n") - cat("were never selected, so the composite ESS and share are undefined;\n") + cat("were never selected, so the composite ESS is undefined;\n") cat("`summary(fit)$pairwise` still contains the NA values.\n") } cat("\n") @@ -179,11 +179,8 @@ print.summary.bgms = function(x, digits = 3, ...) { if(!is.null(x$indicator)) { cat("Inclusion probabilities:\n") - # mean/mcse/sd/n_eff/Rhat are the Rao-Blackwellized inclusion estimate. The - # directional flip counts n0->1 / n1->0 stay in summary(fit)$indicator and - # are not printed. + # mean/mcse/sd/n_eff/Rhat are the Rao-Blackwellized inclusion estimate. ind = head(x$indicator, .summary_preview_rows) - ind = ind[, setdiff(names(ind), c("n0->1", "n1->0")), drop = FALSE] ind_has_na = anyNA(ind) ind[] = lapply(ind, function(col) ifelse(is.na(col), "", round(col, digits))) print(ind) diff --git a/man/extract_ess.Rd b/man/extract_ess.Rd index 2bf5b89e..0205ea27 100644 --- a/man/extract_ess.Rd +++ b/man/extract_ess.Rd @@ -4,17 +4,17 @@ \alias{extract_ess} \title{Extract Effective Sample Size} \usage{ -extract_ess(bgms_object, estimator = c("rb", "mixt")) +extract_ess(bgms_object, estimator = "rb") } \arguments{ \item{bgms_object}{A fitted model object of class \code{bgms} (from \code{\link[=bgm]{bgm()}}) or \code{bgmCompare} (from \code{\link[=bgmCompare]{bgmCompare()}}).} -\item{estimator}{Character; which effective sample size to return for the -edge (or difference) indicators. \code{"rb"} (default) returns the ESS of the -Rao-Blackwellized inclusion draws. \code{"mixt"} -\ifelse{html}{\href{https://lifecycle.r-lib.org/articles/stages.html#deprecated}{\figure{lifecycle-deprecated.svg}{options: alt='[Deprecated]'}}}{\strong{[Deprecated]}} returns the indicator chain's -transition-based ESS. Ignored for all other parameter types.} +\item{estimator}{Character; \code{"rb"}, the only accepted value, returns the ESS +of the Rao-Blackwellized inclusion draws for the edge (or difference) +indicators. \code{"mixt"} \ifelse{html}{\href{https://lifecycle.r-lib.org/articles/stages.html#deprecated}{\figure{lifecycle-deprecated.svg}{options: alt='[Deprecated]'}}}{\strong{[Deprecated]}}, the indicator +chain's transition-based ESS, was deprecated in bgms 0.2.0.0 and now +errors. Ignored for all other parameter types.} } \value{ A named list with ESS values for each parameter type present in @@ -34,18 +34,8 @@ Rao-Blackwellized by default). It is \code{NA} for indicators whose Rao-Blackwellized draws are constant to double precision, where the inclusion probability is at its numerical bound. -\code{estimator = "mixt"}, the transition-based ESS of the binary indicator chain, -is deprecated and no longer a summary column; a call recomputes it from the -raw indicator draws and warns. It converts flip counts to an effective sample -size through a two-state first-order Markov model that a substantial share of -edge chains violate, and the precision of the inclusion probability and its -Bayes factor is carried by the Rao-Blackwellized ESS and Monte Carlo standard -error. The directional flip counts themselves (\code{n0->1}, \code{n1->0}) remain in -the inclusion table. - -Fits made with bgms < 0.2.0.0 carry no Rao-Blackwellized draws; a default -call falls back to the transition ESS without warning, while an explicit -\code{estimator = "rb"} errors. +For older fits without Rao-Blackwellized inclusion draws, \code{extract_ess()} +stops when extracting indicator ESS. Refit with bgms 0.2.0.0 or later. } \seealso{ \code{\link[=bgm]{bgm()}}, \code{\link[=bgmCompare]{bgmCompare()}}, \code{\link[=extract_rhat]{extract_rhat()}} for the diff --git a/man/extract_posterior_inclusion_probabilities.Rd b/man/extract_posterior_inclusion_probabilities.Rd index 37b4b2ab..acf60e56 100644 --- a/man/extract_posterior_inclusion_probabilities.Rd +++ b/man/extract_posterior_inclusion_probabilities.Rd @@ -58,14 +58,13 @@ space, and requires a fit from bgms >= 0.2.0.0. Because the RB draw is continuous, the standard MCSE/ESS/split-R-hat machinery applies to it, and the fit summary's inclusion table reports \code{mcse}, \code{n_eff}, and \code{Rhat} on the RB draws. They quantify precision \emph{conditional on exploration}: a stuck chain -can show a beautifully converged \code{J} chain with a high \code{n_eff}, so read them -beside the per-direction flip counts (\code{n0->1}, \code{n1->0}), which record the -exploration itself and whose asymmetry no symmetric summary recovers. The RB -draws vary on almost every edge, including edges whose indicator never -flipped; the three columns are \code{NA} only where the RB draws are constant to -double precision, which places the inclusion probability at its numerical -bound and the verdict beyond any threshold. All of this is in the fit -summary's inclusion table, \code{summary(fit)$indicator}. +can produce apparently converged Rao-Blackwellized inclusion draws with a +high \code{n_eff}. The RB draws vary on +almost every edge, including edges whose indicator never flipped; the three +columns are \code{NA} only where the RB draws are constant to double precision, +which places the inclusion probability at its numerical bound and the +verdict beyond any threshold. All of this is in the fit summary's inclusion +table, \code{summary(fit)$indicator}. } \seealso{ \code{\link[=extract_inclusion_bf]{extract_inclusion_bf()}}, \code{\link[=bgm]{bgm()}}, \code{\link[=bgmCompare]{bgmCompare()}}, diff --git a/man/extract_rhat.Rd b/man/extract_rhat.Rd index 269a2fb0..c4391f47 100644 --- a/man/extract_rhat.Rd +++ b/man/extract_rhat.Rd @@ -20,9 +20,7 @@ model fitted with \code{\link[=bgm]{bgm()}} or \code{\link[=bgmCompare]{bgmCompa } \details{ The \code{indicator} element is the split-R-hat of the Rao-Blackwellized -inclusion draws, matching the default of \code{\link[=extract_ess]{extract_ess()}}; the indicator -chain's transition-based effective sample size remains available as -\code{extract_ess(fit, estimator = "mixt")} (deprecated). +inclusion draws, matching \code{\link[=extract_ess]{extract_ess()}}. One failure mode these diagnostics may miss: chains can agree that an edge is included while disagreeing about the magnitude of its weight, for diff --git a/tests/testthat/_snaps/summary-marking.md b/tests/testthat/_snaps/summary-marking.md index e666f460..d994f4cf 100644 --- a/tests/testthat/_snaps/summary-marking.md +++ b/tests/testthat/_snaps/summary-marking.md @@ -38,7 +38,7 @@ left their indicators unsampled. Group differences (main effects): - parameter mean mcse sd n_eff share_incl Rhat + parameter mean mcse sd n_eff Rhat * A (diff1; 1) * A (diff1; 2) * A (diff1; 3) @@ -48,16 +48,16 @@ ... (use `summary(fit)$main_diff` to see full output) * a group lacks observations in this category or in the reference category; the estimate reflects the prior, not the data Note: NA values are suppressed in the print table. They occur for differences - that were never selected, so the composite ESS and share are undefined; + that were never selected, so the composite ESS is undefined; `summary(fit)$main_diff` still contains the NA values. Group differences (pairwise effects): - parameter mean mcse sd n_eff share_incl Rhat + parameter mean mcse sd n_eff Rhat A-B (diff1) A-C (diff1) B-C (diff1) Note: NA values are suppressed in the print table. They occur for differences - that were never selected, so the composite ESS and share are undefined; + that were never selected, so the composite ESS is undefined; `summary(fit)$pairwise_diff` still contains the NA values. Use `summary(fit)$` to access full results. diff --git a/tests/testthat/test-extractor-functions.R b/tests/testthat/test-extractor-functions.R index 654d0d2f..8ce0c762 100644 --- a/tests/testthat/test-extractor-functions.R +++ b/tests/testthat/test-extractor-functions.R @@ -348,58 +348,33 @@ test_that("extract_ess indicators default to the RB n_eff and honour estimator", # The default is the RB n_eff column, matching summary() and the R-hat that # extract_rhat() reports. expect_equal(unname(rb), ind$n_eff, info = paste(ctx, "default is RB n_eff")) - expect_equal(unname(extract_ess(fit, estimator = "rb")$indicator), ind$n_eff, - info = paste(ctx, "explicit rb is RB n_eff") - ) # NA masking is inherited from the summary table. expect_identical(is.na(unname(rb)), is.na(ind$n_eff), info = paste(ctx, "RB NA positions") ) - # The transition ESS is deprecated and no longer a summary column; the - # estimator still resolves, warns, and recomputes it from the raw draws. - expect_false("n_eff_mixt" %in% names(ind), info = paste(ctx, "no transition column")) - expect_warning( - mixt <- extract_ess(fit, estimator = "mixt")$indicator, - class = "lifecycle_warning_deprecated" + expect_equal(unname(extract_ess(fit, estimator = "rb")$indicator), ind$n_eff, + info = paste(ctx, "explicit rb is RB n_eff") ) - raw = bgms:::get_fit_cache(fit)$raw - ref = bgms:::.compute_indicator_ess_cpp( - bgms:::combine_chains(raw, "indicator_samples") - )[, "n_eff_mixt"] - expect_equal(unname(mixt), unname(ref), info = paste(ctx, "mixt is the transition ESS")) + # The transition ESS is not a summary column; its estimator is defunct. + expect_false("n_eff_mixt" %in% names(ind), info = paste(ctx, "no transition column")) + expect_error(extract_ess(fit, estimator = "mixt"), class = "lifecycle_error_deprecated") expect_error(extract_ess(fit, estimator = "nonsense")) } expect_gt(checked, 0L) }) -test_that("extract_ess indicators fall back to the transition ESS without RB draws", { +test_that("extract_ess indicators error on summary tables without RB draws", { # Summary tables from fits predating the RB regime (bgms < 0.2.0.0) carry no - # n_eff column: a default call falls back, an explicit "rb" errors, and an - # explicit "mixt" reads the legacy column. + # n_eff column. rb_table = data.frame(n_eff = c(100, NA)) legacy_table = data.frame(n_eff_mixt = c(80, NA)) - default_arg = c("rb", "mixt") - - expect_equal(indicator_ess_column(NULL, rb_table, default_arg, TRUE), c(100, NA)) - expect_equal(indicator_ess_column(NULL, legacy_table, default_arg, TRUE), c(80, NA)) - expect_equal(indicator_ess_column(NULL, legacy_table, "mixt", FALSE), c(80, NA)) - expect_error(indicator_ess_column(NULL, legacy_table, "rb", FALSE), "Rao-Blackwellized") -}) - -test_that("extract_ess indicator names and other elements ignore estimator", { - fit = get_bgms_fit() - rb = extract_ess(fit) - expect_warning(mixt <- extract_ess(fit, estimator = "mixt"), - class = "lifecycle_warning_deprecated" - ) - expect_identical(names(rb), names(mixt)) - expect_identical(names(rb$indicator), names(mixt$indicator)) - expect_equal(rb$pairwise, mixt$pairwise) + expect_equal(indicator_ess_column(rb_table), c(100, NA)) + expect_error(indicator_ess_column(legacy_table), "Rao-Blackwellized") }) test_that("extract_rhat and extract_ess error on non-bgms objects", { diff --git a/tests/testthat/test-mcmc-diagnostics.R b/tests/testthat/test-mcmc-diagnostics.R index c9f2a04d..ca924256 100644 --- a/tests/testthat/test-mcmc-diagnostics.R +++ b/tests/testthat/test-mcmc-diagnostics.R @@ -421,7 +421,11 @@ test_that("bgm RB inclusion Rhat is the classic split-Rhat on J draws, masked on # Edges whose indicator never flipped keep their numbers as long as J varies: # those are the near-boundary short-run edges where the MCSE is needed. - zero_flip = summ[["n0->1"]] + summ[["n1->0"]] == 0 + ind_chains = fit$raw_samples$indicator + ind_arr = array(NA_real_, dim = c(niter, nchains, nparam)) + for(c in seq_len(nchains)) ind_arr[, c, ] = ind_chains[[c]] + counts = bgms:::.compute_indicator_ess_cpp(ind_arr) + zero_flip = counts[, "n01"] + counts[, "n10"] == 0 varying = zero_flip & sds > 1e-6 expect_true(any(varying)) expect_true(all(is.finite(reported[varying]))) @@ -523,7 +527,6 @@ test_that("the indicator summary keeps Rhat Inf for indicators constant at diffe draws = array(0, dim = c(100, 2, 1)) draws[, 2, 1] = 1 summ = bgms:::summarize_indicator(NULL, array3d = draws) - expect_true(is.na(summ$n_eff_mixt)) expect_identical(summ$Rhat, Inf) same = array(1, dim = c(100, 2, 1)) expect_true(is.na(bgms:::summarize_indicator(NULL, array3d = same)$Rhat)) diff --git a/tests/testthat/test-rb-inclusion-probabilities.R b/tests/testthat/test-rb-inclusion-probabilities.R index 1544f1c9..6243f11d 100644 --- a/tests/testthat/test-rb-inclusion-probabilities.R +++ b/tests/testthat/test-rb-inclusion-probabilities.R @@ -400,16 +400,12 @@ test_that("the printed bgm summary reports the RB inclusion estimate", { fit = rb_omrf_fit() ind = summary(fit)$indicator - # The RB precision columns sit beside the raw directional flip counts, which - # stay retrievable so their asymmetry survives. The transition ESS is gone. - expect_true(all(c("mean", "mcse", "sd", "n_eff", "Rhat", "n0->1", "n1->0") %in% - colnames(ind))) - expect_false("n_eff_mixt" %in% colnames(ind)) + # The table holds the RB precision columns only: no transition ESS and no + # directional flip counts. + expect_identical(colnames(ind), c("mean", "mcse", "sd", "n_eff", "Rhat")) # ESS/Rhat are finite or NA (constant columns), never NaN. expect_false(any(is.nan(ind$n_eff))) expect_false(any(is.nan(ind$Rhat))) - # Directional flip counts are non-negative integers, retrievable per edge. - expect_true(all(ind[["n0->1"]] >= 0 & ind[["n1->0"]] >= 0)) # The summary mean equals the RB inclusion probability, edge for edge. rb = extract_posterior_inclusion_probabilities(fit, estimator = "rb") @@ -429,9 +425,8 @@ test_that("the bgmCompare summary RB inclusion has NA rows for unselected mains" ) ind = summary(fit)$indicator - expect_true(all(c("mean", "mcse", "sd", "n_eff", "Rhat", "n0->1", "n1->0") %in% - colnames(ind))) - expect_false("n_eff_mixt" %in% colnames(ind)) + expect_true(all(c("mean", "mcse", "sd", "n_eff", "Rhat") %in% colnames(ind))) + expect_false(any(c("n0->1", "n1->0", "n_eff_mixt") %in% colnames(ind))) expect_false(any(is.nan(ind$n_eff))) # Unselected main-effect differences were never updated, so some rows are NA; # the selected pairwise differences are finite. diff --git a/tests/testthat/test-regressions-2.R b/tests/testthat/test-regressions-2.R index a519b8cb..1dfb57fe 100644 --- a/tests/testthat/test-regressions-2.R +++ b/tests/testthat/test-regressions-2.R @@ -274,9 +274,9 @@ test_that("GGM selection summary is a mixture summary on the association scale", ) pw = fit$posterior_summary_pairwise - # Same summarizer as the discrete models: the derived composite ESS and its - # inclusion-share (bottleneck) columns are present for the weight row. - expect_true(all(c("n_eff", "share_incl") %in% colnames(pw))) + # Same summarizer as the discrete models: the derived composite ESS column is + # present for the weight row. + expect_true("n_eff" %in% colnames(pw)) # And the mean is on the association scale coef() reports. cf = coef(fit)$pairwise expect_equal(sort(pw$mean), sort(cf[upper.tri(cf)]), tolerance = 1e-8) diff --git a/vignettes/diagnostics.Rmd b/vignettes/diagnostics.Rmd index fefbd9ca..3648661d 100644 --- a/vignettes/diagnostics.Rmd +++ b/vignettes/diagnostics.Rmd @@ -55,7 +55,11 @@ summary(fit)$pairwise - The effective sample size (ESS) reflects the number of independent samples that would provide equivalent precision. Larger ESS values indicate more reliable estimates. - The Monte Carlo standard error (MCSE) measures the additional variability introduced by using a finite number of MCMC draws. A small MCSE relative to the posterior standard deviation indicates stable estimates, whereas a large MCSE suggests that more samples are needed. -The 1.01 rule is a guideline for continuous parameters. It should **not** be applied directly to the binary edge (or difference) **indicators**. A decisive edge produces a nearly saturated indicator chain — the inclusion probability sits close to 0 or 1 and the indicator only rarely flips — so R-hat carries little information and can be `NA` (all chains identical) or large (chains stuck in different structures). Read the indicator R-hat together with the directional transition counts (`n0->1` and `n1->0`, reported in the indicator summary) and the posterior inclusion probability: for a saturated or barely-flipping indicator the statistic is uninformative rather than a sign of trouble. The indicator R-hat is most useful when it is large while the inclusion probability is near 0.5 and the transition counts show the chains disagreeing on inclusion. +The inclusion table reports R-hat for the Rao-Blackwellized inclusion draws, +rather than the binary inclusion indicators. An indicator that rarely changes +can still have varying Rao-Blackwellized draws and informative diagnostics. +Interpret these diagnostics alongside the posterior inclusion probabilities +and the parameter diagnostics. ## Diagnostics for edge-selected parameters @@ -73,9 +77,6 @@ errors that actually limit the model-averaged weight. - **n_eff** in the pairwise table is the composite ESS of the model-averaged weight: it combines the Monte Carlo error of the conditional (included-only) weight with the Monte Carlo error of the inclusion probability. -- **share_incl** is the share of that composite Monte Carlo variance - contributed by the inclusion part, so it says which of the two limits the - precision of the weight. The inclusion table carries its own diagnostics: @@ -97,12 +98,6 @@ summary(fit)$indicator second route — an `sd` of about `2e-11` with `NA` `mcse` and `n_eff` and a finite `Rhat`. Either way the reading is the same: the inclusion probability is pinned, not poorly estimated. -- **n0->1** and **n1->0** are the directional flip counts of the binary - indicator chain. They record how much the chain explored the two states, which - the Rao-Blackwellized columns cannot see: those measure precision conditional - on exploration, so a chain stuck in one structure can show a large `n_eff`. - Few flips are expected on an edge whose inclusion probability sits near 0 or - 1, and the asymmetry between the two counts is itself informative. ## Traceplots