build(deps-dev): Bump benchee from 1.1.0 to 1.5.0

[dependabot]

Bumps benchee from 1.1.0 to 1.5.0.

Release notes

Sourced from benchee's releases.

1.5.0 (2025-10-21)

Finally max_sample_size is here to alleviate a variety of woes resulting from gathering too many samples! Also finally, Benchee lets you exclude outliers!

Features (User Facing)

• Introduce max_sample_size which guides how many samples will be gathered at most for a given scenario. This avoids a variety of issues when scenarios gather too many samples (memory consumption, statistics taking long to calculate, formatters hanging/not working). Defaults to 1_000_000, setting it to nil gathers unlimited samples again (behavior before this version).
• Introduce exclude_outliers option which when set to true will automatically exclude outliers from the samples gathered. Especially important for run time, you can remove samples caused by garbage collection or external factors. Defaults to false. Shout out to @​NickNeck who implemented this long wished for feature over in Statistex.
• Display input_name entries in Livebook/Table.Reader protocol. Thanks @​madlep!

Bugfixes (User Facing)

• fixed a bug where if times were supplied as 0 instead of 0.0 we'd sometimes gather a single measurement
• elixir 1.19 compilation warnings have been fixed

Features (Plugins)

• The %Benchee.Statistics{} struct now comes with values to accompany the outlier exclusion feature:
  • outliers - if outlier exclusion was enabled, may include any samples of outliers that were found, empty list otherwise
  • lower_outlier_bound - value below which values are considered an outlier
  • upper_outlier_bound - value above which values are considered an outlier

1.4.0

Some nice features (pre_check: :all_same is cool) along with adding support for some new stuff (tprof) and fixing some bugs.

Features (User Facing)

• pre_check now accepts the option :all_same - which raises if any job returns a value different from another for the same input. Useful when benchmarking multiple variants of the same function. Thanks to @​sabiwara!
• Enable the tprof profiler that shipped with OTP 27, requires OTP 27+ and elixir 1.17+.

Bugfixes (User Facing)

• Make gathering of system data more resilient, if the commands we ran didn't exist it may have crashed previously. Most notably, occurring on Windows 11 due to te removal of the tool we use to gather processor information. Now "N/A" is returned.
• An incompatibility with the upcoming OTP 28 that'd emit a warning on newer elixir versions was removed. Thanks TBK145!

1.3.1

Small release to remove warnings for the upcoming elixir 1.17 as well as add proper solaris support.

Features (User Facing)

• Solaris systems will now correctly be identified and have their CPU and memory parsed. Thanks to @​brianewell!

Bugfixes (User Facing)

• warnings on elixir-main/1.17-rc have been removed. Thanks to @​mhanberg & @​alco!

1.3.0

A big swath of bug fixes and improvements. The highlights certainly are fixes and conveniences around saved benchmarks and loading them again via the new Benchee.report/1. The other big one is saving a lot of memory (and time!) when processing big inputs. Sadly the latter comes with some breaking changes for plugins, but they are well justified and shouldn't actually affect any plugin in practice.

There's also a known issue for elixir 1.14.0 to 1.16.0-rc.0 - read up about it.

Features (User Facing)

... (truncated)

Changelog

Sourced from benchee's changelog.

1.5.0 (2025-10-21)

Finally max_sample_size is here to alleviate a variety of woes resulting from gathering too many samples! Also finally, Benchee lets you exclude outliers!

Features (User Facing)

• Introduce max_sample_size which guides how many samples will be gathered at most for a given scenario. This avoids a variety of issues when scenarios gather too many samples (memory consumption, statistics taking long to calculate, formatters hanging/not working). Defaults to 1_000_000, setting it to nil gathers unlimited samples again (behavior before this version).
• Introduce exclude_outliers option which when set to true will automatically exclude outliers from the samples gathered. Especially important for run time, you can remove samples caused by garbage collection or external factors. Defaults to false. Shout out to @​NickNeck who implemented this long wished for feature over in Statistex.
• Display input_name entries in Livebook/Table.Reader protocol. Thanks @​madlep!

Bugfixes (User Facing)

• fixed a bug where if times were supplied as 0 instead of 0.0 we'd sometimes gather a single measurement
• elixir 1.19 compilation warnings have been fixed

Features (Plugins)

• The %Benchee.Statistics{} struct now comes with values to accompany the outlier exclusion feature:
  • outliers - if outlier exclusion was enabled, may include any samples of outliers that were found, empty list otherwise
  • lower_outlier_bound - value below which values are considered an outlier
  • upper_outlier_bound - value above which values are considered an outlier

1.4.0 (2025-04-14)

Some nice features (pre_check: :all_same is cool) along with adding support for some new stuff (tprof) and fixing some bugs.

Features (User Facing)

• pre_check now accepts the option :all_same - which raises if any job returns a value different from another for the same input. Useful when benchmarking multiple variants of the same function. Thanks to @​sabiwara!
• Enable the tprof profiler that shipped with OTP 27, requires OTP 27+ and elixir 1.17+.

Bugfixes (User Facing)

• Make gathering of system data more resilient, if the commands we ran didn't exist it may have crashed previously. Most notably, occurring on Windows 11 due to te removal of the tool we use to gather processor information. Now "N/A" is returned.
• An incompatibility with the upcoming OTP 28 that'd emit a warning on newer elixir versions was removed. Thanks TBK145!

1.3.1 (2024-06-05)

Small release to remove warnings for the upcoming elixir 1.17 as well as add proper solaris support.

Features (User Facing)

• Solaris systems will now correctly be identified and have their CPU and memory parsed. Thanks to @​brianewell!

Bugfixes (User Facing)

• warnings on elixir-main/1.17-rc have been removed. Thanks to @​mhanberg & @​alco!

1.3.0 (2023-12-22)

A big swath of bug fixes and improvements. The highlights certainly are fixes and conveniences around saved benchmarks and loading them again via the new Benchee.report/1. The other big one is saving a lot of memory (and time!) when processing big inputs. Sadly the latter comes with some breaking changes for plugins, but they are well justified and shouldn't actually affect any plugin in practice.

There's also a known issue for elixir 1.14.0 to 1.16.0-rc.0 - read up about it.

... (truncated)

Commits

• 95c4808 prep 1.5.0 release
• 0edd4ca run type checks and lints on newest elixir & erlang
• f11ca4b Also ignore formatters from duplicated code for now, fixing them is... a long...
• bdbdeb0 Exclude tests from duplicated code check
• 38913ca Use fully released elixir 1.19 in CI
• 60d5dcd Add changelog entry for Table.Reader fix
• 56ff581 Fix scenario typo
• 376a981 Actually use name instead of job_name, which my past self says is the cor...
• ad343fe Be more explicit about no_input handling
• c05fcb3 include input_name in Table.Reader fields
• Additional commits viewable in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Note
Automatic rebases have been disabled on this pull request as it has been open for over 30 days.

[firestack]

@dependabot recreate