Skip to content

Parallelize examples with joblib (closes #5)#65

Open
endolith wants to merge 4 commits into
masterfrom
cursor/add-joblib-parallelization-to-files-5679
Open

Parallelize examples with joblib (closes #5)#65
endolith wants to merge 4 commits into
masterfrom
cursor/add-joblib-parallelization-to-files-5679

Conversation

@endolith

Copy link
Copy Markdown
Owner

Summary

Closes #5. Adds joblib parallelization to the remaining Monte Carlo example scripts that still ran serial election loops, and documents the shared parallelism pattern in examples/README.md.

Changes

Newly parallelized (10 scripts): Merrill tables/figures and Weber effectiveness/table scripts now use the same batch_size / Parallel(n_jobs=-3) pattern as the examples that already had joblib.

Documentation: examples/README.md explains batching, worker count, result aggregation, and seed behavior (Monte Carlo scripts intentionally omit fixed seeds; parallelism preserves the same statistical intent as a serial loop).

Scope: examples/ only — no library or test changes.

Reproducibility review

  • Workers return partial Counter / defaultdict totals merged on the main process; completion order does not affect aggregates.
  • Election generation still uses the same RNG entry points as before (random_utilities, impartial_culture, normal_electorate, np.random, tiebreaker='random').
  • Examples remain stochastic verification runs (no global seed); this matches pre-parallel behavior.

CI

  • ruff check examples/ --select=E9,F63,F7,F82 — pass
  • pytest — pass

Overlap with PR #52

PR #52 (elsim.studies API) also refactors the same example files to use JoblibBackend instead of direct joblib.Parallel calls. Only one of these PRs should merge as-is — whichever lands second will need a rebase and either adoption of elsim.studies or manual conflict resolution. This PR is intentionally scoped to examples/ with direct joblib usage to close #5 without depending on the studies API.

Open in Web Open in Cursor 

Bumps [https://github.com/astral-sh/ruff-pre-commit](https://github.com/astral-sh/ruff-pre-commit) from v0.15.14 to 0.15.15.
- [Release notes](https://github.com/astral-sh/ruff-pre-commit/releases)
- [Commits](astral-sh/ruff-pre-commit@v0.15.14...v0.15.15)

---
updated-dependencies:
- dependency-name: https://github.com/astral-sh/ruff-pre-commit
  dependency-version: 0.15.15
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
@what-the-diff

what-the-diff Bot commented Jun 12, 2026

Copy link
Copy Markdown

PR Summary

  • Enhanced README with Parallel Monte Carlo Information: Enhanced the README documentation with guidance about implementing parallel processing for Monte Carlo simulations. It now includes instructions on beneficially utilizing joblib, an integral tool for parallel computing.

  • Adoption of Batch Processing in Simulation Scripts: The adoption of batch processing has enhanced various simulation scripts. By defining a batch_size, it's now possible to simulate multiple elections per worker, reducing scheduling overhead. The approach enhances the efficiency by concentrating the loop iterations on the batch_size rather than the total number of elections.

  • Parallel Processing Integration: By integrating Parallel and delayed from joblib, we've parallelized the execution of worker tasks. This development affects various scripts, facilitating faster execution of tasks across multiple CPU cores.

  • Refactoring of Simulation Logic: The simulation logic has undergone significant improvements. We created a new function - simulate_batch(), which neatly encapsulates the logic for processing elections in batches. Also, the method of aggregating results from individual simulations has been adjusted, providing an optimized process.

  • Updated Required Library Imports: We introduced necessary joblib imports in each of the affected scripts, supporting the efficient parallel execution of tasks.

  • Improved Initialization of utility_sums: We revised the initialization process of utility_sums to support both ranked and rated methods for utility accumulation, promoting flexibility.

This update delivers an improved, efficient, and parallel execution process that promises increased productivity and faster task execution. It streamlines the simulation process, allowing us to handle more tasks concurrently.

@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.37%. Comparing base (14b8d55) to head (b4436ae).

Additional details and impacted files
@@           Coverage Diff           @@
##           master      #65   +/-   ##
=======================================
  Coverage   96.37%   96.37%           
=======================================
  Files          17       17           
  Lines         496      496           
=======================================
  Hits          478      478           
  Misses         18       18           
Flag Coverage Δ
no-numba 95.76% <ø> (ø)
numba 88.10% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

dependabot Bot and others added 3 commits June 12, 2026 23:48
Bumps [https://github.com/astral-sh/ruff-pre-commit](https://github.com/astral-sh/ruff-pre-commit) from v0.15.15 to 0.15.16.
- [Release notes](https://github.com/astral-sh/ruff-pre-commit/releases)
- [Commits](astral-sh/ruff-pre-commit@v0.15.15...v0.15.16)

---
updated-dependencies:
- dependency-name: https://github.com/astral-sh/ruff-pre-commit
  dependency-version: 0.15.16
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: endolith <endolith@gmail.com>
Add a short section explaining batching, n_jobs=-3, result aggregation,
and that Monte Carlo scripts intentionally omit fixed seeds.

Remove parallelization_summary.md (development artifact).

Co-authored-by: endolith <endolith@gmail.com>
@cursor cursor Bot force-pushed the cursor/add-joblib-parallelization-to-files-5679 branch from 947aef5 to b4436ae Compare June 12, 2026 23:48
@cursor cursor Bot marked this pull request as ready for review June 12, 2026 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallelize all examples

2 participants