Skip to content

Add esrally race --prepare-only#2071

Open
fressi-elastic wants to merge 6 commits into
elastic:masterfrom
fressi-elastic:feature/race-prepare-only
Open

Add esrally race --prepare-only#2071
fressi-elastic wants to merge 6 commits into
elastic:masterfrom
fressi-elastic:feature/race-prepare-only

Conversation

@fressi-elastic

@fressi-elastic fressi-elastic commented Mar 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds --prepare-only to esrally race so Rally can provision Elasticsearch (when the pipeline does), download and prepare track data under the Rally root, then exit successfully without running the challenge schedule or printing benchmark results.

Behavior

  • After preparation completes, the driver and engine are shut down the same way as after a normal race (so --preserve-install and pipeline semantics still apply).
  • In prepare-only mode, Rally does not persist a partial race record via race_store.store_race during preparation.

Add --prepare-only to stop after track preparation (and provisioning when
applicable) without running the challenge or writing benchmark results.
Skip persisting a partial race record in this mode. Shut down the driver
and engine like a normal race end so --preserve-install semantics apply.

Document the flag in race.rst and pipelines.rst; add unit tests for
BenchmarkCoordinator preparation paths.
@fressi-elastic fressi-elastic force-pushed the feature/race-prepare-only branch from 18740bf to 82c87b1 Compare March 24, 2026 13:57
@fressi-elastic

Copy link
Copy Markdown
Contributor Author

I tested it using regular Elasticsearch instance provisioning.

esrally race --distribution-version=9.2.0 --track=geopoint --challenge=append-fast-with-conflicts --prepare-only --test-mode

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[INFO] Race id is [74024f74-4dd7-4d3b-b97b-031c5927b1c2]
[INFO] Preparing for race ...
[INFO] Downloading Elasticsearch 9.2.0 (491.7MB total size)                         [100%]
[INFO] Decompressing track data from [/Users/fressi/.rally/benchmarks/data/geopoint/documents-1k.json.bz2] to [/Users/fressi/.rally/benchmarks/data/geopoint/documents-1k.json] ... [OK]
[INFO] Preparing file offset table for [/Users/fressi/.rally/benchmarks/data/geopoint/documents-1k.json] ... [OK]
[INFO] Prepared track [geopoint], challenge [append-fast-with-conflicts] and car ['defaults'] with version [9.2.0]; prepare-only mode - not running the benchmark or persisting a race record.


--------------------------------
[INFO] SUCCESS (took 56 seconds)
--------------------------------

I didn't tested with other pipelines, example:

esrally race --pipeline=benchmark-only --target-hosts=127.0.0.1:9200 --track=geonames --prepare-only --test-mode

@fressi-elastic

Copy link
Copy Markdown
Contributor Author

I tested it inside of a container with benchmark-only pipeline:

esrally race --pipeline=benchmark-only --target-hosts=es01:9200 --track=geonames --prepare-only --test-mode

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[INFO] Race id is [91ce5978-7ec8-4250-8538-7a1dcbb1e3ea]
[INFO] Downloading track data (20.5KB total size)                                 [100.0%]
[INFO] Decompressing track data from [/rally/.rally/benchmarks/data/geonames/documents-2-1k.json.bz2] to [/rally/.rally/benchmarks/data/geonames/documents-2-1k.json] ... [OK]
[INFO] Preparing file offset table for [/rally/.rally/benchmarks/data/geonames/documents-2-1k.json] ... [OK]
[INFO] Prepared track [geonames], challenge [append-no-conflicts] and car ['external'] with version [9.3.1]; prepare-only mode - not running the benchmark or persisting a race record.

Prepare-only mode now prepares track data locally only: skip Elasticsearch
clients and REST checks in the driver, seed distribution metadata from config
or minimum ES version, use in-memory reporting, and avoid mechanic lifecycle
when no engine is started. Race control sends Success directly when mechanic
is absent; console messages use force/flush for quiet mode.

Update race and pipeline docs to match (benchmark-only example uses
distribution-version; no cluster stop messaging).

Add integration tests and rally-prepare-only-it.ini; extend unit tests.
Makefile: introduce PYTEST/DEBUG/PYTEST_FLAGS and route some IT targets
through the test target.
- Default want_cluster_prepared to True and document subprocess roles.
- Assert track repo and dataset cache exist under RALLY_HOME after prepare-only.
- Assert prepare-only log skips engine and avoids mechanic provision paths.
- Slice full-race log tail to assert Rally provisions ES vs benchmark-only external cluster.
- Remove unused uuid import; fix comment spelling (artifacts).
- Import ES_METRICS_STORE at module scope and narrow http_port before
  waiting on cluster health (mypy arg-type, pylint import-outside-toplevel).
- Annotate TestCluster.http_port as int | None and install http_port as int.
- Drop useless pylint unused-argument suppression on configure_file_handler.

Made-with: Cursor
@fressi-elastic fressi-elastic added this to the 2.14.0 milestone Mar 30, 2026
@fressi-elastic fressi-elastic added enhancement Improves the status quo feedback needed An open question blocks progress highlight A substantial improvement that is worth mentioning separately in release notes labels Mar 30, 2026
@gareth-ellis gareth-ellis requested a review from a team March 31, 2026 11:09
@pquentin pquentin removed request for a team and gareth-ellis May 28, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improves the status quo feedback needed An open question blocks progress highlight A substantial improvement that is worth mentioning separately in release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant