Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and behavior
Hosted Python CI currently runs only the DS4 benchmark tests. The shared C++ runner also lets an empty filter return skip (77), lets an empty binary succeed, and lets
--exactmatch a tag belonging to another test. CTest can therefore report success without executing the intended case.This change makes
pyproject.tomlthe model-free pytest entry point and runs the existing harness/concurrency/helper suites in CI. It makes empty execution/discovery fail and restricts--exactto full test names, while preserving ordinary substring/tag selection and explicit hardware skips.A shared pytest fixture compiles small C++ binaries without CUDA/HIP and exercises the real runner and CMake/CTest adapter. The tests cover assertion failures, exceptions, missing selections, tag/name collisions, explicit skips, and empty discovery.
harness/tests/README.mddocuments scope and tool requirements.Validation
pytest -qcommand on Linux.ruff check .andgit diff --checkpassed.Based on main and independent of #636 and #687. Their qualification policies and report formats are not changed here.