Skip to content

ci: upload omni-github test-result artifacts from CI jobs#6234

Open
mataylor-nvidia wants to merge 18 commits into
isaac-sim:developfrom
mataylor-nvidia:mataylor/omni-github-test-results
Open

ci: upload omni-github test-result artifacts from CI jobs#6234
mataylor-nvidia wants to merge 18 commits into
isaac-sim:developfrom
mataylor-nvidia:mataylor/omni-github-test-results

Conversation

@mataylor-nvidia

@mataylor-nvidia mataylor-nvidia commented Jun 22, 2026

Copy link
Copy Markdown

Description

  • Add upload-omni-github-test-results composite action to convert JUnit XML into the omni-github test-result artifact contract.

  • Wire the upload step into install-ci, package test, and Docker test composite actions.

  • Use IsaacLab upstream repository ID 567038244 in artifact identities for omni-github registration.

  • Installation Tests workflow uploads pytest-results-*--v1-567038244-* artifacts

  • omni-github registration PR can reference real sample artifact names from this run

link to slack discussion:

https://nvidia.slack.com/archives/C0B16FFDDBP/p1782216333267729

link to artifact schema:

https://github.com/NVIDIA-Omniverse/omni-github/blob/main/docs/test-results/clients/artifact-upload/artifact-schema.md

reference to junit format: https://github.com/testmoapp/junitxml#structure

example artifact from most recent pipeline run: TODO

Fixes # (OMPE-98494)

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@greptile-apps

greptile-apps Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR introduces an upload-omni-github-test-results composite action that converts JUnit XML reports into the omni-github test-result artifact format and uploads them after each CI job, enabling cross-run test analytics. It is wired into the install-CI, Docker-test, and package-test composite actions, and three build-workflow jobs get explicit omni-github-test-type labels.

  • New composite action (upload-omni-github-test-results): derives runner platform automatically from RUNNER_OS/RUNNER_ARCH, calls a Python converter, validates the output against a bundled JSON Schema, and uploads the artifact with an omni-github-compatible name (<prefix>--v1-<repo_id>-<run_id>-<attempt>-<check_run_id>).
  • JUnit converter (junit_to_omni_github_results.py): handles passed, failed, skipped, and crash/timeout-flagged test rows; includes unit tests covering all cases.
  • Caller changes: install-ci-run, run-tests, and run-package-tests composite actions each add the upload step, forwarding artifact URLs as log_paths for traceability.

Confidence Score: 5/5

Safe to merge — the change is additive CI infrastructure with no impact on production code paths.

All changes are confined to GitHub Actions composite actions and a standalone Python converter. The converter logic is well-tested, platform derivation was moved from an unreliable default to automatic shell detection, and callers wire the new step correctly. The only notes are a missing explicit pip install jsonschema before the schema-validation call and a cosmetic extra space in one YAML id key — neither affects correctness on the target GitHub-hosted runners.

No files require special attention; the new composite action and its Python converter are self-contained.

Important Files Changed

Filename Overview
.github/actions/upload-omni-github-test-results/action.yml New composite action that derives platform from runner env vars, converts JUnit XML via Python, validates against a bundled JSON schema, then uploads the artifact. The schema validation step relies on jsonschema being pre-installed rather than explicitly installing it.
.github/actions/upload-omni-github-test-results/junit_to_omni_github_results.py JUnit-to-omni-github converter; correctly handles passed, failed, skipped, and crash/timeout-flagged test cases. Platform derivation moved from this script to the shell wrapper.
.github/actions/upload-omni-github-test-results/test_junit_to_omni_github_results.py Unit tests covering failure, skip, crash/timeout detection, and log-path population; all assertions look correct.
.github/actions/upload-omni-github-test-results/result-json.schema.json Bundled JSON Schema for local validation; minItems: 1 on the tests array means empty test runs will fail schema validation (pre-existing concern noted in review thread).
.github/actions/install-ci-run/action.yml Adds id to the existing upload step so its URL can be forwarded to the new composite action; wiring looks correct.
.github/actions/run-tests/action.yml Wires the new upload action after each Docker test run; introduces a minor id : key with an extra space (style nit), and correctly threads both artifact URLs through.
.github/actions/run-package-tests/action.yml Adds omni-github-test-type input and passes it through to the inner run-tests composite action; straightforward delegation change.
.github/workflows/build.yaml Sets explicit omni-github-test-type values (training-e2e, rendering-correctness, rendering-correctness-kitless) for the three differentiated test jobs.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant CI as CI Job (install-ci / run-tests)
    participant UA as actions/upload-artifact
    participant UO as upload-omni-github-test-results
    participant PY as junit_to_omni_github_results.py
    participant JS as jsonschema CLI
    participant GH as GitHub Artifacts

    CI->>UA: Upload JUnit XML report (id: upload-junit-report)
    UA-->>CI: artifact-url output
    CI->>UO: junit-file, junit-log-url, artifact-prefix, test-type
    UO->>UO: Derive app_platform from RUNNER_OS/RUNNER_ARCH
    UO->>PY: --junit-file --output-dir --test-tool-id ...
    PY->>PY: Parse JUnit XML → convert testcases
    PY-->>UO: writes _testoutput/test_results.json + manifest
    UO->>JS: validate test_results.json against schema
    JS-->>UO: exit 0 (valid) or non-zero (invalid)
    UO->>GH: upload-artifact (name includes repository_id + run_id + check_run_id)
    GH-->>UO: artifact uploaded
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant CI as CI Job (install-ci / run-tests)
    participant UA as actions/upload-artifact
    participant UO as upload-omni-github-test-results
    participant PY as junit_to_omni_github_results.py
    participant JS as jsonschema CLI
    participant GH as GitHub Artifacts

    CI->>UA: Upload JUnit XML report (id: upload-junit-report)
    UA-->>CI: artifact-url output
    CI->>UO: junit-file, junit-log-url, artifact-prefix, test-type
    UO->>UO: Derive app_platform from RUNNER_OS/RUNNER_ARCH
    UO->>PY: --junit-file --output-dir --test-tool-id ...
    PY->>PY: Parse JUnit XML → convert testcases
    PY-->>UO: writes _testoutput/test_results.json + manifest
    UO->>JS: validate test_results.json against schema
    JS-->>UO: exit 0 (valid) or non-zero (invalid)
    UO->>GH: upload-artifact (name includes repository_id + run_id + check_run_id)
    GH-->>UO: artifact uploaded
Loading

Reviews (7): Last reviewed commit: "fix test" | Re-trigger Greptile

Comment thread .github/actions/upload-omni-github-test-results/junit_to_omni_github_results.py Outdated
Comment thread .github/actions/upload-omni-github-test-results/action.yml Outdated
@mataylor-nvidia

Copy link
Copy Markdown
Author

@greptileai

…or-nvidia/IsaacLab into mataylor/omni-github-test-results
@mataylor-nvidia mataylor-nvidia changed the title ci: upload omni-github test-result artifacts from CI jobs Draft: ci: upload omni-github test-result artifacts from CI jobs Jun 22, 2026
@mataylor-nvidia mataylor-nvidia changed the title Draft: ci: upload omni-github test-result artifacts from CI jobs ci: upload omni-github test-result artifacts from CI jobs Jun 22, 2026
@mataylor-nvidia mataylor-nvidia marked this pull request as draft June 22, 2026 20:03
Comment thread .github/actions/install-ci-run/action.yml Outdated
@mataylor-nvidia mataylor-nvidia self-assigned this Jun 24, 2026
@mataylor-nvidia mataylor-nvidia force-pushed the mataylor/omni-github-test-results branch from ba6ba0e to 245ebd1 Compare June 24, 2026 15:19
@mataylor-nvidia mataylor-nvidia marked this pull request as ready for review June 24, 2026 15:22
Comment thread .github/actions/upload-omni-github-test-results/junit_to_omni_github_results.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants