Skip to content

Add adaptive sampling parameters and early stopping for optimization#12

Open
xiki-tempula wants to merge 2 commits into
DingGroup:mainfrom
xiki-tempula:feat_optimise
Open

Add adaptive sampling parameters and early stopping for optimization#12
xiki-tempula wants to merge 2 commits into
DingGroup:mainfrom
xiki-tempula:feat_optimise

Conversation

@xiki-tempula

@xiki-tempula xiki-tempula commented Feb 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add adaptive defaults for sample_size and warmup_steps that scale with problem dimensionality
  • Implement early stopping for hyperparameter optimization to improve efficiency
  • Add comprehensive tests for the new adaptive parameter functionality

Changes

Adaptive sample_size

  • Changed default from fixed 1000 to None (adaptive)
  • When None, automatically computed as max(1000, 100 * (m - 1)) where m is the number of states
  • Ensures adequate posterior sampling for high-dimensional problems

Adaptive warmup_steps

  • Changed default from fixed 500 to None (adaptive)
  • When None, automatically computed as max(500, sample_size // 2) following Stan-like heuristics
  • Scales appropriately with sample size for better NUTS adaptation

Early Stopping for Optimization

  • Renamed optimize_steps to max_optimize_steps (breaking change for keyword argument users)
  • Added optimize_patience (default: 500) - steps without improvement before stopping
  • Added optimize_min_delta (default: 1e-4) - minimum loss improvement threshold
  • Tracks best parameters during optimization and restores them at convergence
  • Prints early stopping message when triggered (if verbose=True)

New Parameters

Parameter Default Description
sample_size None Adaptive: max(1000, 100 * (m - 1))
warmup_steps None Adaptive: max(500, sample_size // 2)
max_optimize_steps 10000 Maximum optimization iterations
optimize_patience 500 Early stopping patience
optimize_min_delta 1e-4 Minimum improvement threshold

Breaking Changes

  • optimize_steps parameter renamed to max_optimize_steps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant