Skip to content

Commit f210e30

Browse files
rh-amarinrh-amarin
committed
HYPERFLEET-633 - docs: add HyperFleet release contract and integration testing strategy
1 parent c96748d commit f210e30

1 file changed

Lines changed: 168 additions & 0 deletions

File tree

‎hyperfleet/docs/release-contract.md‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
---
2+
Status: Draft
3+
Owner: HyperFleet Team
4+
Last Updated: 2026-06-02
5+
---
6+
7+
# HyperFleet Release Contract
8+
9+
## What & Why
10+
11+
### What
12+
13+
This document defines the formal release contract between the HyperFleet team and its partner teams (GCP Offering Team and ROSA Regional Platform Team). It covers:
14+
15+
- **Release handoff contract**: what artifacts are produced, how partners are notified, and what SLAs apply
16+
- **Integration testing strategy**: how each team gates HyperFleet changes and how tests are coordinated across teams
17+
- **Test ownership map**: which team owns which layer of testing to eliminate redundancy and close coverage gaps
18+
19+
### Why
20+
21+
- Without a defined contract, release handoffs are ad-hoc and require manual coordination between teams, slowing delivery and increasing error risk
22+
- Unclear test ownership creates either coverage gaps (bugs reaching production) or overlapping test suites (longer pipelines without benefit)
23+
- Partner teams deploying HyperFleet via Argo CD and Terraform need predictable, machine-consumable artifacts (OCI Helm charts) to automate rollout
24+
- A shared testing strategy is required before building confidence in continuous delivery pipelines end-to-end
25+
26+
### Out of scope
27+
28+
- No automated deployment of HyperFleet releases to partner integration environments
29+
30+
---
31+
32+
## Partner Teams
33+
34+
| Team | Platform | Deployment Method |
35+
|------|----------|-------------------|
36+
| GCP Offering Team | GCP | Argo CD + Terraform + Tekton Pipelines |
37+
| ROSA Regional Platform | AWS ROSA | Argo CD + Terraform + AWS CodePipelines |
38+
39+
---
40+
41+
## Release Handoff Contract
42+
43+
### Release Artifacts
44+
45+
For each HyperFleet release, the following artifacts are produced and made available to partner teams:
46+
47+
| Artifact | Location | Format | Notes |
48+
|----------|----------|--------|-------|
49+
| Container images | `quay.io/redhat-services-prod/hyperfleet-tenant/hyperfleet/hyperfleet-{component}:{version}` | OCI image | Built automatically by Konflux on release tag push (PaC tag-triggered pipeline); includes signed provenance (SLSA L3) and SBOM |
50+
| Helm charts | `quay.io/redhat-services-prod/hyperfleet-tenant/hyperfleet-{component}-chart:{version}` (see [Helm Chart Distribution](#helm-chart-distribution)) | OCI artifact | Published via Konflux `build-helm-chart-oci-ta` task; required for ROSA/Argo CD consumption |
51+
| API contract lib | [api-spec](https://github.com/openshift-hyperfleet/hyperfleet-api-spec) | NPM library | Base TypeSpec types for API contract |
52+
| API contract template | [api-spec-template](https://github.com/openshift-hyperfleet/hyperfleet-api-spec-template) | GitHub repository | Used as example |
53+
| Release notes | `hyperfleet-release` repo, `releases/release-X.Y/` | Markdown | Compatibility matrix, breaking changes, upgrade guide |
54+
| Compatibility matrix | `hyperfleet-release` repo | Markdown table | Maps validated component version combinations |
55+
| Git tags | Per-component repos + `hyperfleet-release` | `vX.Y.Z` / `release-X.Y` | See [Release Process](release/hyperfleet-release-process.md) |
56+
57+
58+
When a GA release is published, it will have detail of which ROSA/GCP versions have passed the integration tests to use as compatibility matrix. This allows to potentially introduce a breaking change in one release, that may be only deployable by another pillar.
59+
60+
### Helm Chart Distribution
61+
62+
HyperFleet team publishes charts as OCI artifacts to `quay.io/redhat-services-prod/hyperfleet-tenant/` via Konflux (using the `build-helm-chart-oci-ta` task), eliminating the temporary workaround and the Git coupling. Blocked on the Konflux team providing a managed release pipeline for external registries (RELEASE-2363).
63+
64+
65+
### Notification SLA
66+
67+
When a HyperFleet GA release is published:
68+
69+
| Event | Channel | Timeline | Recipients |
70+
|-------|---------|----------|------------|
71+
| Release candidate available | `#hyperfleet-releases` Slack | RC cut day | GCP team, ROSA team |
72+
| GA release published | `#hyperfleet-releases` Slack | GA day | GCP team, ROSA team |
73+
| Breaking change in next release | `#hyperfleet-releases` Slack | ≥ 1 sprint before GA | GCP team, ROSA team |
74+
| Hotfix / patch release | `#hyperfleet-releases` Slack | Within 2 hours of GA tag | GCP team, ROSA team |
75+
76+
77+
At this point in time (Jun 26) breaking changes are not blockers to HyperFleet releases as ROSA/GCP teams do not have to keep long running clusters and migrate data.
78+
79+
### Rollback / Recovery
80+
81+
HyperFleet uses a **roll-forward** strategy for MVP: issues are fixed via patch releases rather than rollback. See [Release Process — Release Recovery Strategy](release/hyperfleet-release-process.md#55-release-recovery-strategy).
82+
83+
HyperFleet commits to (exact times TBD):
84+
85+
- Producing a patch release within **48 hours** for Blocker/Critical regressions
86+
- Producing a patch release within **1 week** for Major regressions
87+
- Maintaining N-1 backward compatibility so partner teams can remain pinned to the previous validated release while a fix is in flight
88+
89+
---
90+
91+
## Integration Testing Strategy
92+
93+
### Decision: Nightly Runs with OCI Chart Injection
94+
95+
**Agreed approach** (as of March 31, 2026 meeting):
96+
97+
- Start with **nightly runs** against HyperFleet `main` branch, not presubmit jobs
98+
- Test against the **latest known-good stable version** of the ROSA regional platform (production Maestro version), replacing only the HyperFleet component under test
99+
- The ROSA team will **temporarily enable OCI chart pushing** so the HyperFleet team can inject PR-built charts into the ROSA deployment pipeline
100+
- Evaluate **non-blocking presubmit** integration with the HyperFleet release repository as a follow-up
101+
102+
**Rationale**: Running full ROSA environment provisioning (~40 minutes + E2E duration) as a presubmit would significantly impact development velocity without proportional benefit at the current team scale. Nightly runs provide meaningful feedback without blocking day-to-day development.
103+
104+
**Note on ROSA's existing pre-merge capability**: The ROSA repo already has a working cross-component E2E pre-merge mechanism (triggered via Prow comment on PRs). The decision to start with nightly runs is about HyperFleet's readiness to onboard to that mechanism — not a limitation of the ROSA infrastructure. Per-PR testing remains the target once the OCI chart injection step is stable.
105+
106+
107+
### Team Test Ownership
108+
109+
| Layer | Owner | Scope | Runs on |
110+
|-------|-------|-------|---------|
111+
| Unit tests | HyperFleet | Each component in isolation | Every PR (presubmit) |
112+
| Integration tests | HyperFleet | Cross-component API contracts | Every PR (presubmit) |
113+
| HyperFleet E2E | HyperFleet | HyperFleet stack end-to-end | Nightly (main branch) |
114+
| ROSA integration | ROSA Team | Full ROSA region + HyperFleet override | Nightly (HyperFleet main) |
115+
| GCP integration | GCP Team | GCP deployment + HyperFleet | Nightly (HyperFleet main) via Tekton `gcp-region-e2e-pipeline` |
116+
| Release gate | HyperFleet | All of the above must pass | Before GA tag |
117+
118+
**Important**: HyperFleet releases ship with a compatibility matrix for the ROSA/GCP versions tested. If newer ROSA/GCP integration tests fail, that doesn't gate the HyperFleet release as long as it passes the successful ROSA/GCP compatible versions.
119+
120+
### Testing Gaps Identified
121+
122+
| Gap | Owning Team | Mitigation |
123+
|-----|-------------|------------|
124+
| HyperFleet not yet onboarded to ROSA's pre-merge E2E mechanism | HyperFleet | Onboard to `openshift/release` Prow config + create `quay.io/rrp-dev-ci/` image repos (see onboarding steps above) |
125+
| Helm chart override (OCI) not yet wired into ROSA CI | ROSA + HyperFleet | Temporary OCI setup by ROSA team (Q2 2026, immediate action); replaced by Konflux `build-helm-chart-oci-ta` pipeline — blocked on Konflux managed release pipeline for external registries (RELEASE-2363); see [Helm OCI Distribution Design](release/helm-oci-distribution-design.md) |
126+
| GCP integration tests not yet publishing chart overrides into HyperFleet CI | GCP + HyperFleet | Blocked on GCP-334 (CLM/Argo CD integration in progress); nightly Tekton pipeline already exists but consumes pinned tags, not PR-built charts |
127+
| Multi-component PR testing (API + Adapter in same PR) | HyperFleet | Nightly tests use `main` for all other components; single-component override per nightly run is the starting point |
128+
| Presubmit integration gate for HyperFleet release repo | HyperFleet | Future action: non-blocking presubmit on `hyperfleet-release` repo |
129+
130+
131+
---
132+
133+
## Alternatives Considered
134+
135+
### 1. Non-blocking Presubmit on HyperFleet Release Repository
136+
137+
Run the full ROSA integration pipeline as an optional, non-blocking presubmit job triggered on the `hyperfleet-release` repo.
138+
139+
**Rejected for now**: A ~40-minute+ non-blocking job provides weak signal — developers may ignore it, especially if failures are infrequent. Starting with nightly runs builds confidence in the pipeline before promoting it to presubmit. This remains a **future action**.
140+
141+
### 2. Consumer-Driven Contract Testing (Pact-style)
142+
143+
Define formal API contracts using a consumer-driven contract testing tool (e.g., Pact). ROSA and GCP publish their expectations; HyperFleet CI verifies them on every PR.
144+
145+
**Rejected for MVP**: The integration surface between HyperFleet and partner teams is primarily at the Helm chart / deployment configuration level, not a REST API contract boundary. Consumer-driven contract testing tools are better suited to service-to-service REST contracts. Helm value schema validation is a lighter-weight alternative to investigate post-MVP.
146+
147+
### 3. Automated Rollout to Integration Environments on GA
148+
149+
Trigger automatic deployment of each HyperFleet GA release to ROSA and GCP integration environments via webhooks.
150+
151+
**Rejected for MVP**: ROSA's pipeline takes ~40 minutes per run and requires environment-specific configuration overrides. Automating this safely requires tooling (OCI charts via Konflux, pipeline webhooks) not yet in place. Deferred to post-Q2 2026.
152+
153+
---
154+
155+
## Related Documents
156+
157+
- [HyperFleet Release Process](release/hyperfleet-release-process.md) — release cadence, branching, artifacts
158+
- [Konflux Release Pipeline Design](release/konflux-release-pipeline-design.md) — build and release pipeline architecture, CEL tag matching, version injection
159+
- [Helm OCI Distribution Design](release/helm-oci-distribution-design.md) — how Helm charts are published as OCI artifacts via Konflux
160+
- [ADR 0014 — Konflux for Container Image Build and Release](../adrs/0014-konflux-build-and-release.md)
161+
- [ADR 0016 — Helm OCI Distribution via Konflux](../adrs/0016-helm-oci-distribution.md)
162+
- [Versioning Trade-offs](versioning-trade-offs.md) — SDK versioning, rollback considerations
163+
- [E2E Testing Framework Spike Report](e2e-testing/e2e-testing-framework-spike-report.md)
164+
- [E2E Run Strategy Spike Report](e2e-testing/e2e-run-strategy-spike-report.md)
165+
- [ROSA — Adding a Component for Pre-merge E2E Testing](https://github.com/openshift-online/rosa-regional-platform/blob/main/docs/adding-component-pre-merge.md) — onboarding guide for the `/test rosa-regionality-compatibility-e2e` Prow trigger
166+
- [ROSA — Testing Strategy Design](https://github.com/openshift-online/rosa-regional-platform/blob/main/docs/design/testing-strategy.md) — three CI workflows (pre-merge, nightly integration, nightly ephemeral)
167+
- GCP-334 — CLM Components Deployment (linked Jira epic)
168+
- HYPERFLEET-633 — Define release contract and integration testing strategy

0 commit comments

Comments
 (0)