2.13.0

2.13.0

Highlights

• #1981 (Breaking): Drop Python 3.9 support; add Python 3.13 support; replace nox with uv.
• #2056: Add render-track command to render a track's Jinja2 templates to JSON
• #2047 / #2058: Add esql-profile operation for ES|QL queries with profiling and operator timing metrics
• #2037: Account for the number of clients in the ingest-doc-count option
• #2033: Add ingest-doc-count option to index an exact number of documents
• #1992: Add run-until runner
• #1991: Create CLI for Rally storage manager.
• #1971: Support API key authentication for reporting

Enhancements

• #2046: Add detailed results support for ES|QL queries
• #2039: Log warning on BadRequestError
• #1868: Bulk http status & 429 retries
• #2041: Track certain headers if present in error responses
• #2034: Disk usage stats timeout
• #2028: Set the pbzip2 -m flag to the highest supported limit of 2000MB
• #2020: Parse sampling value in DownsampleParamSource
• #2016: Use wait_for_completion:False instead of relying on timeouts in ForceMerge
• #2019: Always show mirror failures when present.
• #2015: Modify storage CLI to produce multiple objects for transference stats and mirror failures when using --filebeat option
• #2013: Read transfer URL from .status file instead of its relative path.
• #2008: Add esrally-storage prune subcommand.
• #1959: Capture statistics during data download so that we can profile mirrors configuration in CI.
• #2005: Adjust default IT folder
• #2004: Add sampling method parameter to downsampling
• #2000: Add Google Cloud Storage adapter.
• #2001: Sort files to make deterministic
• #1995: Initialize all loggers in rally daemon
• #1996: Speedup default s3 corpus downloader
• #1965: Enable new multipart transfer manager in track downloader.
• #1986: Enable a release build of an Elasticsearch revision from sources (source.build.release)
• #1984: Add ability to continue on network issues
• #1975: Allow configuring the process startup method to be used for creating thespian actors.
• #1964: Add TransferManager and S3Adapter classes.
• #1972: Add --user-tags to list races
• #1962: feat: Enable uv for venv management.
• #1960: Update storage tools with last changes from storage branch.
• #1967: Remove negative duration constraint (may be needed when comparing race results)
• #1957: Add skip-telemetry parameter
• #1952: Create module to implement multipart downloading using threads and previously submitted client.
• #1955: Update esrally.utils package: convert, pretty and threads.
• #1953: Reapply "Add a variable in the provisioner for the seed node count (#1647)" (#1680)
• #1951: Add missing JSON logs
• #1949: Create storage client that redirects downloads to mirror servers
• #1945: Add HTTP adapter for the storage client.
• #1943: Add some threading synchronization tools.
• #1942: Add tool to handle file parts.

Bug Fixes

• #2053: Fix including the chunk of the partition to calculate the amount of data to handle per partition
• #2050: Fix syntax
• #2038: Fix Docker build revisited
• #2035: Fix Docker build
• #2023: Fix esrally-storage put command target file names when multiple files are copied.
• #2017: Pin requirement: google-crc32c==1.7.1
• #2014: Fix optional version branch regex
• #2012: Fix cli_test
• #2009: Fix best_match versions algorithm in case of no major version
• #2006: Fix serverless ITs
• #1999: Fix parallel corpora files download.
• #1997: Address pytest deprecations
• #1998: Fix transfer finished condition when worker is stuck downloading.
• #1994: Fix: Respect --offline flag for all track operations
• #1988: Recover all loggers left orphan after thespian logging setup on actors startup.
• #1989: Fix Makefile
• #1987: Recover old silenced loggers instantiated before actor start is complete.
• #1977: Fix ITs after pytest-rally plugin modification
• #1973: Increase Docker image creation resilience
• #1966: Esrally/stop params/fix #1957
• #1956: Pin aiosignal
• #1946: Fix two typos.
• #1869: Use url_prefix from host dictionary in Elastic client
• #1937: Fix snapshot restore
• #1938: Stop opening for writes in MmapSource

Miscellaneous Changes

• #2049: Add shorter timeouts for dev docker builds
• #2062: Apply persist-credentials: false to GitHub Actions workflows
• #2027: Bump wheel from 0.45.1 to 0.46.2
• #2025: Bump pyasn1 from 0.6.1 to 0.6.2
• #2021: Bump aiohttp from 3.10.11 to 3.13.3
• #1985: Update GH macOS hosted runner image
• #1983: Remove pip caching from GH workflow
• #1979: Remove duplicate venv-destroy target in Makefile
• #1970: Remove pytest-rally dependency
• #1961: Update yappi to latest
• #1963: Clarify supported Python versions
• #1947: style: sort output in list tracks
• #1926: Bump jinja2 from 3.1.5 to 3.1.6

2.12.0

2.12.0

Highlights

• #1917: Upgrade thespian to 4.0.1
• #1912: Preserve pre-existing rally-metrics index templates by default
• #1893 (Breaking): feat: Drop support for Python 3.8 [ES-8947]

Enhancements

• #1932: issue #1931: create-track command supports serverless.
• #1924: Introduce tools for pretty printing JSON like objects and their differences.
• #1909: Add check if headers is not None before treating it like a dict
• #1908: Improve error messages regarding track syntax errors
• #1892: feat: Add a parallel JSON log by default
• #1885: feat: Add ability to run esrallyd inside docker [ES-9146]

Bug Fixes

• #1919: Suppress a flaky internal mypy error appearing randomly when executing make lint.
• #1913: Add ability to provide a list as input for disk-usage-stats
• #1904: Reduce log volume forwarded through Thespian framework
• #1896: Add read timeout for retry
• #1891: fix: Use named parameter for _to_datetime to pass default_tz
• #1888: fix: Release docker script fix - use echo not git

Miscellaneous Changes

• #1929: Upgrade google-auth to 2.38.0
• #1922: Implements a decorator to facilitate creating table-driven unit tests.
• #1911: Replace deprecate use of resources.read_text() function.
• #1899: Bump jinja2 from 3.1.4 to 3.1.5

2.11.1

Rally release 2.11.1

2.11.1

Highlights

• #1875: Switch to JDK 21

Enhancements

• #1884: Add Jinja filters for creating dynamic date ranges based on now()
• #1871: [ES-8436] feat: Change Dockerfile base images to Wolfi
• #1865: Add an option to filter by challange in list command
• #1859: Modify rally to allow multiple cars with complex configuration
• #1858: Bump urllib3 from 1.26.18 to 1.26.19

Bug Fixes

• #1882: fixes #1881 - composite operation failing when track_total_hits is false
• #1879: Pull non-default branch commits selectively
• #1877: Docs: fix level of (warmup-)interations
• #1876: Cast batch size as int
• #1863: Avoid logging ES credentials from running Rally processes
• #1861: Restore time measurement on request end
• #1848: Pin requests library until we fix docker-py

2.11.0

2.11.0

Highlights

• #1839: Add support for Python 3.12

Enhancements

• #1851: Assume no operator if security not enabled in serverless
• #1846: Update create track
• #1845: Allow rally to run ESQL operations on serverless for benchmarks
• #1831: Add client ID to metrics
• #1830: Add looped mode to bulk operation
• #1827: Track dataset size
• #1819: Override serverless cluster version in telemetry
• #1818: Override serverless cluster version
• #1816: Add FS metrics to node-stats telemetry device

Bug Fixes

• #1835: Avoid immense term error in metrics metadata fields
• #1825: Make Jit telelmetry compatible with JDK 9+
• #1823: Measure time until last response chunk
• #1805: Handle raw response exceptions

Doc Changes

• #1824: Fix telemetry docs
• #1811: Use composable index templates for metric indices

2.10.0

Highlights

• #1797: Document Rally use with Elastic Serverless

Enhancements

• #1791: Add ESQL operator
• #1789: Add serverless-aware runners
• #1787 (Breaking): Inject serverless_operator in track templates
• #1786: Add support for zstd-compression
• #1785: Fix serverless public use in trivial tracks

Bug Fixes

• #1790: Retrieve cluster info in serverless public mode
• #1784: Fix crash on too many consecutive errors

Miscellaneous Changes

• #1793: Bump urllib3 from 1.26.9 to 1.26.18

2.9.1

Enhancements

• #1778: Support API key authentication
• #1770: Exclude telemetry devices based on serverless status
• #1768: Detect serverless operator status automatically

Bug Fixes

• #1772: Fix "Could not checkout. Do you have uncommitted changes?" error

2.9.0

Highlights

• #1760: Exclude tasks based on serverless status
• #1750 (Breaking): Inject build_flavor in track templates

Enhancements

• #1756: Retrieve serverless build hash from nodes info API
• #1755: Add blob-store-stats telemetry device and tests
• #1743: Bulk action refresh param

Bug Fixes

• #1763: Fix running tracks across multiple load driver machines
• #1747: Fix rally-annotations index creation

Doc Changes

• #1748: Document actor system

Miscellaneous Changes

• #1759: Ensure serverless.mode/operator are always set

2.8.0

Highlights

• #1683: Upgrade Elasticsearch client to 8.6.1
• #1669: Upgrade ES Client to 8.x

Enhancements

• #1727: Allow configuring logging to use local time
• #1707: Evenly distribute client connections across hosts
• #1706: Allow specifying branch together with timestamp in revision
• #1702: Include 'refresh' and 'flush' stats by default when using 'node-stats-include-indices'
• #1663: Add operating system cgroup stats to node-stats telemetry

Bug Fixes

• #1722: Increase 'mapping.total_fields.limit' to '2000'
• #1719: Bump up minimum Python version
• #1710: Actually use client returned from '.options()'
• #1705: Refactor some runners to set transport level options
• #1703: Improve exception handling for metrics store errors
• #1698: Update progress after GCS download finished
• #1692: Allow to delete indices related to index templates
• #1691: Ignore HTTP 404 consistently in runners
• #1690: Use correct params for create component template
• #1687: Propagate request start/end lazily

Miscellaneous Changes

• #1725: Make async context manager synchronous

2.7.1

Highlights

• #1674: Allow reading metrics password from environment
• #1624: Skip building plugins moved to modules

Enhancements

• #1654: Add role metadata to node-stats telemetry device
• #1649: Allow specifying --installation-id in esrally install
• #1643: Skip response parsing for raw requests
• #1630: Add delay and duration parameters to jfr telmetry device
• #1619: Add shard data to search operation's detailed-results meta-data
• #1618: Support meta-data for composite tasks
• #1610: Support specifying user tags as json

Bug Fixes

• #1645: Fix --to-date option for list subcommand
• #1640: Pass plugin-params also for modules
• #1638: Don't use --short when retrieving revisions from git

Doc Changes

• #1666: Document bulk behavior with multiple clients/documents
• #1648: Fix broken links on the command line reference docs
• #1641: Mention jq is required for development
• #1613: Document Track Processor Interface
• #1605: Update list of available tracks

Miscellaneous Changes

• #1659: Simplify create_readers
• #1615: Print errors when --quiet is used
• #1611: Use newer ssl parameter in aiohttp client API
• #1607: Cut down on the amount of INFO-level logging

2.7.0

Highlights

• #1574: Include a new downsampling operation
• #1531: Support data streams in create-track
• #1517: Remove the check for internet connection

Enhancements

• #1599: add --user-tags as alias for --user-tag
• #1597: Add additional filter options to rally list races
• #1591: Support checking out remote branches
• #1588: Fail benchmark if assertion could not run
• #1576: Add build subcommand
• #1573: Add knn vectors to disk-usage-stats
• #1551: Switch from setuptools to hatchling
• #1542: Add new wait-for-current-snapshots-create operation
• #1541: Update opts.py to allow for 'None' timeout-value in client_options

Bug Fixes

• #1586: Fix supplier tests for aarch64
• #1580: Remove buggy indices.exists() calls from code
• #1578: Correctly retry HTTP downloads after incomplete reads
• #1563: Fix multi-cluster attribute check for wrapped runners
• #1540: Don't load libs path if not using it

Doc Changes

• #1600: Add docs for create-track from data streams
• #1572: Mention that create/restore-snapshot is an administrative operation
• #1568: Encourage users to read tracks documentation
• #1557: Remove duplicate property descriptions from corpora docs
• #1554: Link to the 7.14.0 ES Python client docs.
• #1552: Fix broken links on README.md

Miscellaneous Changes

• #1579: Use narrower exception to avoid pylint ignore
• #1571: Stop trying to test Elasticsearch 6.8.0 on ARM
• #1565: Fix make lint on Python 3.10
• #1564: Introduce rally-tracks compatibility testing
• #1555: Add an integration test for static responses.
• #1553: Run unit and integration tests in separate tox environments
• #1548: Move most packaging metadata to pyproject.toml
• #1546: Run lint with pre-commit
• #1539: Correctly log how long it took to close transports
• #1498: Bump upper tested version to 8.4.0
• #1497: Remove Elasticsearch 6.3 logic