Skip to content

New example: Parameter sensitivity analysis and scenario sweeps #55

@smjenness

Description

@smjenness

Motivation

Users building models with EpiModel inevitably need to explore how results change as parameters vary. The CompetingStrains example has a rudimentary parameter sweep (over concurrency), but no example demonstrates systematic sensitivity analysis as its primary focus. This is among the most common tasks users face after building a model, and there's no clear template for how to structure it efficiently in EpiModel.

Suggested Design

Disease Model: SIR (Simple, Familiar)

Use a deliberately simple SIR model so the disease mechanics don't distract from the sensitivity analysis workflow. No custom modules needed — use EpiModel's built-in SIR or a minimal custom set.

Focus: Workflow and Analysis Patterns

This example is about the analysis framework around the simulation, not the disease model itself. It should demonstrate:

  1. One-way sensitivity analysis: Vary a single parameter while holding others fixed
  2. Two-way sensitivity analysis: Vary two parameters simultaneously (heat map)
  3. Scenario comparison: Run a defined set of parameter combinations and compare outcomes
  4. Efficient batch simulation: How to structure loops/lapply over parameter grids

Parameters to Sweep

Parameter Range Purpose
inf.prob 0.05–0.50 (10 values) Demonstrates sensitivity to transmissibility
rec.rate 0.01–0.10 (5 values) Recovery rate — combined with inf.prob gives effective R0
mean.degree 0.5–2.0 (4 values) Network structure — requires re-estimating the ERGM for each value

ERGM Parameterization

  • ~edges: Varied as part of the sweep (mean degree from 0.5 to 2.0)
  • ~concurrent (optional): Could add a second sweep dimension over concurrency to demonstrate how network structure interacts with disease parameters
  • Dissolution ~offset(edges): Fixed duration ~50 weeks

The key teaching point for ERGM: when sweeping over network parameters (like mean degree), you must re-estimate the ERGM for each value. This is different from sweeping over epidemic parameters, which only require re-running netsim(). The example should make this distinction explicit.

Scenarios

Not scenarios in the traditional sense — instead, the example demonstrates:

  1. One-way sweep over inf.prob: 10 simulations at different values, extract final prevalence and cumulative incidence from each
  2. Two-way sweep over inf.prob × rec.rate: 50-cell grid (10 × 5), produce a heat map of attack rate
  3. Network parameter sweep: Re-estimate ERGM at 4 mean degree values, run epidemics on each — shows the full workflow including netest() inside the loop
  4. Threshold analysis: Find the critical inf.prob at which the epidemic takes off (R0 > 1) for a given network

Analyses

  • Tornado diagram: One-way sensitivity of cumulative infections to each parameter — which parameter matters most?
  • Heat map: Two-way sensitivity (inf.prob × rec.rate) with cumulative attack rate as the outcome
  • Epidemic probability curve: Fraction of simulations producing a major epidemic vs. inf.prob — demonstrates stochastic extinction vs. takeoff
  • Threshold identification: Plot cumulative infections vs. inf.prob to visually identify the epidemic threshold
  • Efficient simulation strategy: Demonstrate using nsims > 1 within each parameter set to capture stochastic variability, then summarize across simulations before comparing across parameters

Implementation Notes

The example should demonstrate a clean, reusable pattern:

# Parameter grid
param_grid <- expand.grid(
  inf.prob = seq(0.05, 0.50, length.out = 10),
  rec.rate = seq(0.01, 0.10, length.out = 5)
)

# Batch simulation
results <- lapply(seq_len(nrow(param_grid)), function(i) {
  param <- param.net(inf.prob = param_grid$inf.prob[i],
                     rec.rate = param_grid$rec.rate[i], ...)
  sim <- netsim(est, param, init, control)
  # Extract summary statistics
  ...
})

And for network parameter sweeps where re-estimation is needed:

for (md in mean_degrees) {
  target.stats <- md * (n / 2)
  est <- netest(nw, formation, target.stats, coef.diss)
  sim <- netsim(est, param, init, control)
  ...
}

Relationship to Existing Examples

  • Builds on any simple model (could use AddingAnExposedState's SEIR or built-in SIR)
  • Extends CompetingStrains' concurrency sweep into a general-purpose methodology
  • Teaches a workflow pattern that applies to every other gallery example
  • Complements CostEffectivenessAnalysis by providing the parameter exploration that would precede a formal CEA

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions