Skip to content

[chore] Retire threshold_tests#643

Open
dvdplm wants to merge 7 commits into
mainfrom
dvdplm/chore/retire-features-threshold_tests
Open

[chore] Retire threshold_tests#643
dvdplm wants to merge 7 commits into
mainfrom
dvdplm/chore/retire-features-threshold_tests

Conversation

@dvdplm

@dvdplm dvdplm commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Before this PR the core-client had a bespoke threshold_tests feature to determin which integration tests to run. This PR retires that feature and, after measuring test run times, replaces it with slow_tests for tests that are effectively slow.

The number of tests that run for each PR has increased a little bit. At the same time the number of CI jobs decreases from 4 to 1. The slowest test is test_threshold_insecure by a very large margin, so 4 nextest processes is enough to parallelize that.

Part of https://github.com/zama-ai/kms-internal/issues/3017

Local test timings used to decide which tests are "slow":

1126.662	test_threshold_insecure
326.360	test_threshold_mpc_context_switch_6
224.047	test_threshold_default_preproc_keygen
186.076	test_threshold_concurrent_preproc_keygen
115.664	test_threshold_reshare
93.969	test_threshold_mpc_context_init
5.134	test_threshold_concurrent_crs
1.372	test_threshold_mpc_context_switch
0.874	test_threshold_custodian_backup
0.793	test_threshold_restore_from_backup
0.775	test_threshold_insecure_default_keygen
0.572	test_threshold_abort_key_gen
0.557	test_threshold_abort_crs_gen
0.007	config_conformance_client_local_threshold

…ts` where appropriate. Also collapse CI setup from

3 to 1 runner.
@dvdplm dvdplm requested review from a team as code owners June 15, 2026 07:03
@cla-bot cla-bot Bot added the cla-signed The CLA has been signed. label Jun 15, 2026
@dvdplm dvdplm self-assigned this Jun 15, 2026
@github-actions

Copy link
Copy Markdown

Consolidated Tests Results 2026-06-15 - 07:46:52

Test Results

passed 7 passed

Details

tests 7 tests
clock not captured
tool junit-to-ctrf
build build-and-test arrow-right test-reporter link #2706
pull-request [chore] Retire `threshold_tests` link #643

test-reporter: Run #2706

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
7 7 0 0 0 0 0 not captured

🎉 All tests passed!

Tests

View All Tests
Test Name Status Flaky Duration
k8s_test_crs_uniqueness 44.1s
k8s_test_insecure_keygen_encrypt_and_public_decrypt 1m 49s
k8s_test_insecure_keygen_encrypt_multiple_types 2m 5s
k8s_test_keygen_and_crs 1m 50s
k8s_test_keygen_uniqueness 4m 32s
k8s_test_centralized_insecure 58.4s
nightly_full_gen_tests_default_k8s_centralized_sequential_crs 1.8s

🍂 No flaky tests in this run.

Github Test Reporter by CTRF 💚

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes the bespoke threshold_tests feature flag from the kms-core-client crate and CI, replacing it with the workspace-wide slow_tests convention to gate only the heaviest threshold integration tests while simplifying CI sharding.

Changes:

  • Renamed/retired threshold_tests in kms-core-client and replaced it with slow_tests.
  • Updated core-client integration test gating (#[cfg(feature = "slow_tests")] / cfg_attr(..., ignore)) and refreshed related docs.
  • Simplified CI for core-client threshold tests from multiple partitions/jobs to a single job that runs with --features slow_tests.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
core-client/tests/README.md Updates local test commands and feature-flag documentation to reflect slow_tests.
core-client/tests/integration/integration_test.rs Moves heavy tests/helpers to slow_tests gating and removes threshold_tests conditionals.
core-client/Cargo.toml Replaces the threshold_tests feature with slow_tests and updates feature documentation.
.github/workflows/parallel-testing.yml Collapses core-client threshold CI from multiple partitions to a single slow_tests job.
.github/workflows/main.yml Switches the nightly core-client job to build/run with slow_tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread core-client/tests/README.md Outdated
Comment thread core-client/tests/README.md
Comment thread core-client/tests/README.md Outdated
Comment thread core-client/tests/README.md Outdated
Comment thread core-client/Cargo.toml Outdated
dvdplm and others added 4 commits June 15, 2026 17:30
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

Consolidated Tests Results 2026-06-15 - 16:08:38

Test Results

passed 7 passed

Details

tests 7 tests
clock not captured
tool junit-to-ctrf
build build-and-test arrow-right test-reporter link #2723
pull-request [chore] Retire `threshold_tests` link #643

test-reporter: Run #2723

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
7 7 0 0 0 0 0 not captured

🎉 All tests passed!

Tests

View All Tests
Test Name Status Flaky Duration
k8s_test_crs_uniqueness 39.1s
k8s_test_insecure_keygen_encrypt_and_public_decrypt 1m 42s
k8s_test_insecure_keygen_encrypt_multiple_types 1m 57s
k8s_test_keygen_and_crs 1m 40s
k8s_test_keygen_uniqueness 4m 6s
k8s_test_centralized_insecure 51.6s
nightly_full_gen_tests_default_k8s_centralized_sequential_crs 1.6s

🍂 No flaky tests in this run.

Github Test Reporter by CTRF 💚

@github-actions

Copy link
Copy Markdown

Consolidated Tests Results 2026-06-16 - 13:08:48

Test Results

passed 7 passed

Details

tests 7 tests
clock not captured
tool junit-to-ctrf
build build-and-test arrow-right test-reporter link #2732
pull-request [chore] Retire `threshold_tests` link #643

test-reporter: Run #2732

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Pending ⏳ Other ❓ Flaky 🍂 Duration ⏱️
7 7 0 0 0 0 0 not captured

🎉 All tests passed!

Tests

View All Tests
Test Name Status Flaky Duration
k8s_test_crs_uniqueness 42.8s
k8s_test_insecure_keygen_encrypt_and_public_decrypt 1m 47s
k8s_test_insecure_keygen_encrypt_multiple_types 2m
k8s_test_keygen_and_crs 1m 47s
k8s_test_keygen_uniqueness 4m 18s
k8s_test_centralized_insecure 1m 1s
nightly_full_gen_tests_default_k8s_centralized_sequential_crs 1.9s

🍂 No flaky tests in this run.

Github Test Reporter by CTRF 💚

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed The CLA has been signed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants