Bump version to 2.13.0

Author: fressi-elastic

.dockerignore

@@ -0,0 +1,85 @@
+# Patterns aligned with .gitignore — reduces Docker build context size.
+# See .gitignore for original references and rationale.
+
+# OS
+.DS_Store
+.AppleDouble
+.LSOverride
+._*
+.DocumentRevisions-V100
+.fseventsd
+.Spotlight-V100
+.TemporaryItems
+.Trashes
+.VolumeIcon.icns
+.AppleDB
+.AppleDesktop
+.apdisk
+
+# IDE / editors
+*.iml
+.idea/
+*~
+/.project
+/.pydevproject
+/.vscode
+
+# Virtualenv / local env
+.venv*/
+env/
+.envrc
+.python-version
+
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Test / coverage
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*,cover
+.hypothesis/
+junit-*.xml
+.pytest_cache/
+
+# Translations / logs
+*.mo
+*.pot
+*.log
+
+# Docs build
+docs/_build/
+
+# PyBuilder
+target/
+
+# Pickles
+*.pk
+
+# Rally-specific
+.rally_it/
+NOTICE.txt
+recipes/ccr/ccr-target-hosts.json
+tracks/

.pre-commit-config.yaml

@@ -49,5 +49,15 @@ repos:
             "-j0",
             "--rcfile=.pylintrc",
           ]
-        exclude: ^(docs/|changelog.py)
+        exclude: ^(docs/|scripts/release/changelog\.py)
         require_serial: true
+
+  # Shell scripts: release tooling and repo githooks. Widen to all *.sh once legacy
+  # scripts under .buildkite/, recipes/, repo root, etc. pass shellcheck.
+  - repo: https://github.com/shellcheck-py/shellcheck-py
+    rev: v0.11.0.1
+    hooks:
+      - id: shellcheck
+        name: shellcheck
+        args: [-x]
+        files: (^scripts/release/.*\.sh$)|(^scripts/githooks/(pre-commit|post-commit)$)

AUTHORS

@@ -7,11 +7,14 @@ Aswin Murugesh
 Atara Avidan
 Brad Deam
 Brian Lesperance
+Carlos Delgado
 Christian Dahlqvist
 Christopher Manning
+Copilot
 Dale McDiarmid
 Daniel Mitterdorfer
 Dave Pifke
+David Kyle
 David Turner
 Debanjan Choudhury
 Dennis Lawler
@@ -36,7 +39,10 @@ Jason Bryan
 Jason Tedor
 Jim Ferenczi
 Jiří Holuša
+Joe Gallo
 John Sobanski
+John Wagster
+Jordan Powers
 Joshua Rich
 Karan Kaushik
 Kazuma Arimura
@@ -52,16 +58,20 @@ Lyndon Swan
 Maciek Sufa
 Magnus Kessler
 Martijn van Groningen
+Mary Gouseti
 Matt Veitas
 Max Jakob
 Michael Basnight
 Michael McCandless
+Michail Romaios
 Mike Baamonde
 MoBoo
 Nhat Nguyen
 Nicholas Knize
 nicholaskuechler
+Nick Dris
 Nik Everett
+Oliver Hansen
 Paul Coghlan
 Peter Dyson
 Przemko Robakowski
@@ -72,6 +82,7 @@ Rohit Nair
 Salvatore Campagna
 sstults
 Thomas Decaux
+Thomas Holmes
 Tobias Suckow
 tomcallahan
 Ugo Sangiorgi

CHANGELOG.md

@@ -1,3 +1,101 @@
+### 2.13.0
+
+#### Highlights
+
+* [#1981](https://github.com/elastic/rally/pull/1981) (Breaking): Drop Python 3.9 support; add Python 3.13 support; replace nox with uv.
+* [#2056](https://github.com/elastic/rally/pull/2056): Add ``render-track`` command to render a track's Jinja2 templates to JSON
+* [#2047](https://github.com/elastic/rally/pull/2047) / [#2058](https://github.com/elastic/rally/pull/2058): Add ``esql-profile`` operation for ES|QL queries with profiling and operator timing metrics
+* [#2037](https://github.com/elastic/rally/pull/2037): Account for the number of clients in the ingest-doc-count option
+* [#2033](https://github.com/elastic/rally/pull/2033): Add ingest-doc-count option to index an exact number of documents
+* [#1992](https://github.com/elastic/rally/pull/1992): Add run-until runner
+* [#1991](https://github.com/elastic/rally/pull/1991): Create CLI for Rally storage manager.
+* [#1971](https://github.com/elastic/rally/pull/1971): Support API key authentication for reporting
+
+#### Enhancements
+
+* [#2046](https://github.com/elastic/rally/pull/2046): Add detailed results support for ES|QL queries
+* [#2039](https://github.com/elastic/rally/pull/2039): Log warning on BadRequestError
+* [#1868](https://github.com/elastic/rally/pull/1868): Bulk http status & 429 retries
+* [#2041](https://github.com/elastic/rally/pull/2041): Track certain headers if present in error responses
+* [#2034](https://github.com/elastic/rally/pull/2034): Disk usage stats timeout
+* [#2028](https://github.com/elastic/rally/pull/2028): Set the pbzip2 -m flag to the highest supported limit of 2000MB
+* [#2020](https://github.com/elastic/rally/pull/2020): Parse sampling value in DownsampleParamSource
+* [#2016](https://github.com/elastic/rally/pull/2016): Use wait_for_completion:False instead of relying on timeouts in ForceMerge
+* [#2019](https://github.com/elastic/rally/pull/2019): Always show mirror failures when present.
+* [#2015](https://github.com/elastic/rally/pull/2015): Modify storage CLI to produce multiple objects for transference stats and mirror failures when using --filebeat option
+* [#2013](https://github.com/elastic/rally/pull/2013): Read transfer URL from `.status` file instead of its relative path.
+* [#2008](https://github.com/elastic/rally/pull/2008): Add `esrally-storage prune` subcommand.
+* [#1959](https://github.com/elastic/rally/pull/1959): Capture statistics during data download so that we can profile mirrors configuration in CI.
+* [#2005](https://github.com/elastic/rally/pull/2005): Adjust default IT folder
+* [#2004](https://github.com/elastic/rally/pull/2004): Add sampling method parameter to downsampling
+* [#2000](https://github.com/elastic/rally/pull/2000): Add Google Cloud Storage adapter.
+* [#2001](https://github.com/elastic/rally/pull/2001): Sort files to make deterministic
+* [#1995](https://github.com/elastic/rally/pull/1995): Initialize all loggers in rally daemon
+* [#1996](https://github.com/elastic/rally/pull/1996): Speedup default s3 corpus downloader
+* [#1965](https://github.com/elastic/rally/pull/1965): Enable new multipart transfer manager in track downloader.
+* [#1986](https://github.com/elastic/rally/pull/1986): Enable a release build of an Elasticsearch revision from sources (source.build.release)
+* [#1984](https://github.com/elastic/rally/pull/1984): Add ability to continue on network issues
+* [#1975](https://github.com/elastic/rally/pull/1975): Allow configuring the process startup method to be used for creating thespian actors.
+* [#1964](https://github.com/elastic/rally/pull/1964): Add `TransferManager` and `S3Adapter` classes.
+* [#1972](https://github.com/elastic/rally/pull/1972): Add --user-tags to list races
+* [#1962](https://github.com/elastic/rally/pull/1962): feat: Enable uv for venv management.
+* [#1960](https://github.com/elastic/rally/pull/1960): Update storage tools with last changes from `storage` branch.
+* [#1967](https://github.com/elastic/rally/pull/1967): Remove negative duration constraint (may be needed when comparing race results)
+* [#1957](https://github.com/elastic/rally/pull/1957): Add skip-telemetry parameter
+* [#1952](https://github.com/elastic/rally/pull/1952): Create module to implement multipart downloading using threads and previously submitted client.
+* [#1955](https://github.com/elastic/rally/pull/1955): Update `esrally.utils` package:  convert, pretty and threads.
+* [#1953](https://github.com/elastic/rally/pull/1953): Reapply "Add a variable in the provisioner for the seed node count (#1647)" (#1680)
+* [#1951](https://github.com/elastic/rally/pull/1951): Add missing JSON logs
+* [#1949](https://github.com/elastic/rally/pull/1949): Create storage client that redirects downloads to mirror servers
+* [#1945](https://github.com/elastic/rally/pull/1945): Add HTTP adapter for the storage client.
+* [#1943](https://github.com/elastic/rally/pull/1943): Add some threading synchronization tools.
+* [#1942](https://github.com/elastic/rally/pull/1942): Add tool to handle file parts.
+
+#### Bug Fixes
+
+* [#2053](https://github.com/elastic/rally/pull/2053): Fix including the chunk of the partition to calculate the amount of data to handle per partition
+* [#2050](https://github.com/elastic/rally/pull/2050): Fix syntax
+* [#2038](https://github.com/elastic/rally/pull/2038): Fix Docker build revisited
+* [#2035](https://github.com/elastic/rally/pull/2035): Fix Docker build
+* [#2023](https://github.com/elastic/rally/pull/2023): Fix `esrally-storage` put command target file names when multiple files are copied.
+* [#2017](https://github.com/elastic/rally/pull/2017): Pin requirement: google-crc32c==1.7.1
+* [#2014](https://github.com/elastic/rally/pull/2014): Fix optional version branch regex
+* [#2012](https://github.com/elastic/rally/pull/2012): Fix cli_test
+* [#2009](https://github.com/elastic/rally/pull/2009): Fix best_match versions algorithm in case of no major version
+* [#2006](https://github.com/elastic/rally/pull/2006): Fix serverless ITs
+* [#1999](https://github.com/elastic/rally/pull/1999): Fix parallel corpora files download.
+* [#1997](https://github.com/elastic/rally/pull/1997): Address pytest deprecations
+* [#1998](https://github.com/elastic/rally/pull/1998): Fix transfer finished condition when worker is stuck downloading.
+* [#1994](https://github.com/elastic/rally/pull/1994): Fix: Respect --offline flag for all track operations
+* [#1988](https://github.com/elastic/rally/pull/1988): Recover all loggers left orphan after thespian logging setup on actors startup.
+* [#1989](https://github.com/elastic/rally/pull/1989): Fix Makefile
+* [#1987](https://github.com/elastic/rally/pull/1987): Recover old silenced loggers instantiated before actor start is complete.
+* [#1977](https://github.com/elastic/rally/pull/1977): Fix ITs after pytest-rally plugin modification
+* [#1973](https://github.com/elastic/rally/pull/1973): Increase Docker image creation resilience
+* [#1966](https://github.com/elastic/rally/pull/1966): Esrally/stop params/fix #1957
+* [#1956](https://github.com/elastic/rally/pull/1956): Pin aiosignal
+* [#1946](https://github.com/elastic/rally/pull/1946): Fix two typos.
+* [#1869](https://github.com/elastic/rally/pull/1869): Use url_prefix from host dictionary in Elastic client
+* [#1937](https://github.com/elastic/rally/pull/1937): Fix snapshot restore
+* [#1938](https://github.com/elastic/rally/pull/1938): Stop opening for writes in MmapSource
+
+#### Miscellaneous Changes
+
+* [#2049](https://github.com/elastic/rally/pull/2049): Add shorter timeouts for dev docker builds
+* [#2062](https://github.com/elastic/rally/pull/2062): Apply ``persist-credentials: false`` to GitHub Actions workflows
+* Add optional Docker workflow for maintainers to run ``scripts/release/prepare.sh`` (``scripts/release/prepare-docker.sh``, ``scripts/release/Dockerfile``)
+* [#2027](https://github.com/elastic/rally/pull/2027): Bump wheel from 0.45.1 to 0.46.2
+* [#2025](https://github.com/elastic/rally/pull/2025): Bump pyasn1 from 0.6.1 to 0.6.2
+* [#2021](https://github.com/elastic/rally/pull/2021): Bump aiohttp from 3.10.11 to 3.13.3
+* [#1985](https://github.com/elastic/rally/pull/1985): Update GH macOS hosted runner image
+* [#1983](https://github.com/elastic/rally/pull/1983): Remove pip caching from GH workflow
+* [#1979](https://github.com/elastic/rally/pull/1979): Remove duplicate venv-destroy target in Makefile
+* [#1970](https://github.com/elastic/rally/pull/1970): Remove pytest-rally dependency
+* [#1961](https://github.com/elastic/rally/pull/1961): Update yappi to latest
+* [#1963](https://github.com/elastic/rally/pull/1963): Clarify supported Python versions
+* [#1947](https://github.com/elastic/rally/pull/1947): style: sort output in list tracks
+* [#1926](https://github.com/elastic/rally/pull/1926): Bump jinja2 from 3.1.5 to 3.1.6
+
 ### 2.12.0
 
 #### Highlights

CONTRIBUTING.md

@@ -38,6 +38,17 @@ If you want to get started in the project, a good idea is to check issues labele
 You will need to fork the Rally repository and clone it to your local machine. See 
 the [Github help page](https://help.github.com/articles/fork-a-repo) for help.
 
+Optional: use this repository’s hooks under `scripts/githooks` (instead of `.git/hooks`):
+
+```bash
+git config core.hooksPath scripts/githooks
+```
+
+* **`pre-commit`** — runs `uv run -- pre-commit run` (same as `make pre-commit`) on staged files before the commit is created. Requires [uv](https://docs.astral.sh/uv/) and a dev environment (`make install` or `make venv`). You do not need `make install-pre-commit` when using this path.
+* **`post-commit`** — strips `Co-authored-by` / `Made-with: Cursor` trailers Cursor may add; it runs `git commit --amend` when the message changes, so the latest commit’s hash can change.
+
+To stop using these hooks in this clone, run `git config --unset core.hooksPath`. The post-commit hook sets `GIT_CURSOR_HOOK_AMENDING` internally to avoid re-running while amending; you normally do not need to set that variable yourself.
+
 ### Importing the project into IntelliJ IDEA
 
 Rally builds using virtualenv. When importing into IntelliJ you will need to define an appropriate Python SDK, which is provided by virtualenv.
@@ -60,7 +71,7 @@ Once your changes and tests are ready to submit for review:
 
     Ensure that all tests pass by running `make check-all`. This runs sequentially lint checks, unit tests and integration tests. These can be executed in isolation using `make lint`, `make test` and `make it` respectively, in case you need to iterate over a subset of tests.
 
-    Note: Integration tests are much slower than unit tests and require `docker-compose`.
+    Note: Integration tests are much slower than unit tests and require `docker-compose`. They also require **Java 17 or 21**; set `JAVA_HOME` (or `JAVA21_HOME`) to match.
 
 3. Sign the Contributor License Agreement
 
@@ -79,6 +90,10 @@ Then sit back and wait. There will probably be discussion about the pull request
 
 Note: Contributors belonging to the "Elastic" organization on Github can merge PRs themselves after getting a "LGTM" (Looks good to me); this workflow is similar to the established one in the Elasticsearch project.
 
+## Release process (maintainers)
+
+Version bumps and changelog updates are automated with `scripts/release/prepare.sh` (often via `make release RELEASE_VERSION=X.Y.Z`, which uses Docker). Put the GitHub token at `~/.github/rally_release_changelog.token`, or set `RALLY_CHANGELOG_TOKEN` to that file's path so `changelog.py`, `make release-checks`, and `prepare-docker.sh` use the same location on the host. See the **Preparing a release** section in the [development documentation](https://esrally.readthedocs.io/en/latest/developing.html#preparing-a-release) for milestones, tokens, the release container, and how that differs from the published `elastic/rally` benchmark image.
+
 # Contributing to the Rally codebase
 
 **Repository:** [https://github.com/elastic/rally](https://github.com/elastic/rally)

Makefile

@@ -217,11 +217,15 @@ benchmark: venv
 # --- Release goals ---
 
 release-checks: venv
-	$(VENV_ACTIVATE); ./release-checks.sh $(release_version) $(next_version)
+	@if [ -z "$(RELEASE_VERSION)" ]; then echo "error: set RELEASE_VERSION (e.g. make release-checks RELEASE_VERSION=2.13.0)" >&2; exit 1; fi
+	$(VENV_ACTIVATE); ./scripts/release/checks.sh $(RELEASE_VERSION)
 
-# usage: e.g. make release release_version=0.9.2 next_version=0.9.3
-release: venv release-checks clean docs lint test it
-	$(VENV_ACTIVATE); ./release.sh $(release_version) $(next_version)
+# usage: e.g. make release RELEASE_VERSION=2.13.0
+release:
+	@if [ -z "$(RELEASE_VERSION)" ]; then echo "error: set RELEASE_VERSION (e.g. make release RELEASE_VERSION=2.13.0)" >&2; exit 1; fi
+	./scripts/release/prepare-docker.sh $(RELEASE_VERSION)
+
+# --- Other goals ---
 
 # This is a shortcut for creating a shell running inside the project virtual environment.
 sh: