From 38e64f4c4e28b992943e790ba8d103e5d0768eb3 Mon Sep 17 00:00:00 2001 From: Tyler <109685178+tlongwell-block@users.noreply.github.com> Date: Thu, 24 Sep 2026 11:36:10 -0400 Subject: [PATCH 1/2] fix(ci): bootstrap the reusable MinIO image in GHCR (#7869) Upstream MinIO image pulls are blocking CI (#7867). Add a separate publisher for `ghcr.io/block/buzz-minio:latest`, so the reusable image can be bootstrapped before CI adopts it in #7870. - Build the checksummed official MinIO/mc release binaries from #7868 on a digest-pinned Alpine base. - Build only when image/workflow/Compose inputs change or on manual dispatch. PRs build and smoke-test; only `block/buzz` main can publish. - Run the real Compose healthcheck and initializer, object upload/read/delete, and anonymous-access denial before publishing `latest` plus a unique `sha--run--` tag. - Manual dispatch disables Docker's layer cache to refresh Alpine packages. Normal image builds retain caching; each maintenance rebuild keeps its own tag. ### Rollout This PR contains only the publisher and opt-in Compose override. **It does not switch ordinary CI consumers**, so its checks do not require the unpublished image. This is a separate PR from adoption, as required by Buzz's squash-only merge rules. 1. Merge this PR; its main-branch workflow builds, tests, and publishes the image. 2. Make the `buzz-minio` GHCR package public and verify an anonymous pull of `ghcr.io/block/buzz-minio:latest`. New GHCR packages default to private. 3. Rebase draft #7870 onto current main, retarget it to main, run its integration checks, and merge it to switch all MinIO-backed CI jobs to the published image. ### Validation - Updated hosted [MinIO image build and smoke test](https://github.com/block/buzz/actions/runs/36018177877) passed at `700ab6d5346b37a64c9defde0016dc65c98aa790`; publication correctly skipped on the PR. - Hosted [ordinary CI](https://github.com/block/buzz/actions/runs/36018178123) passed at the same head. Its actual Git comparison detected only the five publisher files, all application path filters were false, and the MinIO-dependent integration checks were correctly skipped. Synced current main before this run to eliminate an earlier stale-base comparison that included unrelated ACP changes. - Actual five changed paths were passed through the repo's pinned `dorny/paths-filter`: all application domains were false. All 12 existing path-selection cases and the required-context isolation contract also passed. - Publisher actionlint, smoke-script shellcheck, and commit/push hooks passed. - Fresh local rebuild of `700ab6d5346b37a64c9defde0016dc65c98aa790` passed with `docker buildx build --no-cache --platform linux/amd64 --load`. Logs confirm new Alpine index fetches and package installation rather than a cached install layer. The committed smoke script passed under local AMD64 emulation using the current merged Compose services with isolated resource names and no published ports: healthcheck, initializer/private bucket, object upload/read/delete, and anonymous HTTP 403. Temporary containers, network, and volume were removed. Both pinned binary checksums match the official release SHA-256 files. - Full `just ci` was previously attempted: workspace formatting/Clippy, desktop checks, and Tauri formatting passed; the run was stopped during Tauri Clippy. No full local application-suite pass is claimed; hosted application suites correctly skip this publisher-only change. --------- Signed-off-by: Tyler Longwell (cherry picked from commit 99c2acf90cfbb1cb2d3a8bd900c0ec1642e20540) --- .github/ci/minio/Dockerfile | 16 +++++++ .github/ci/minio/README.md | 51 ++++++++++++++++++++ .github/ci/minio/smoke-test.sh | 27 +++++++++++ .github/workflows/minio-image.yml | 80 +++++++++++++++++++++++++++++++ docker-compose.ci.yml | 12 +++++ 5 files changed, 186 insertions(+) create mode 100644 .github/ci/minio/Dockerfile create mode 100644 .github/ci/minio/README.md create mode 100644 .github/ci/minio/smoke-test.sh create mode 100644 .github/workflows/minio-image.yml create mode 100644 docker-compose.ci.yml diff --git a/.github/ci/minio/Dockerfile b/.github/ci/minio/Dockerfile new file mode 100644 index 00000000000..79fe0e8e350 --- /dev/null +++ b/.github/ci/minio/Dockerfile @@ -0,0 +1,16 @@ +# CI-only AMD64 image. Versions match the development Compose services. +# Upstream registries deny pulls; official release binaries remain available. +FROM alpine:3.22.6@sha256:5291449c3df73caf6ed85e649dec1b9e818b39a5d8c871e97afc13e9cd5e8fa8 + +LABEL org.opencontainers.image.source="https://github.com/block/buzz" \ + org.opencontainers.image.title="Buzz CI MinIO" \ + org.opencontainers.image.licenses="AGPL-3.0-only" + +RUN apk add --no-cache ca-certificates curl + +ADD --checksum=sha256:7c5bd8512c6e966455b1d198209358b2d191c77a83ab377c4073281065fb855f --chmod=755 \ + https://github.com/minio/minio/releases/download/RELEASE.2025-09-07T16-13-09Z/minio.linux-amd64.RELEASE.2025-09-07T16-13-09Z /usr/local/bin/minio +ADD --checksum=sha256:01f866e9c5f9b87c2b09116fa5d7c06695b106242d829a8bb32990c00312e891 --chmod=755 \ + https://github.com/minio/mc/releases/download/RELEASE.2025-08-13T08-35-41Z/mc.linux-amd64.RELEASE.2025-08-13T08-35-41Z /usr/local/bin/mc + +ENTRYPOINT ["minio"] diff --git a/.github/ci/minio/README.md b/.github/ci/minio/README.md new file mode 100644 index 00000000000..51bf7c70a25 --- /dev/null +++ b/.github/ci/minio/README.md @@ -0,0 +1,51 @@ +# Buzz CI MinIO + +`ghcr.io/block/buzz-minio:latest` contains MinIO and `mc` for the disposable +Linux AMD64 CI runners. `docker-compose.ci.yml` selects it for both services; +development and deployment defaults stay in `docker-compose.yml`. + +The **MinIO image** workflow builds only when its inputs change, or on a manual +dispatch. Pull requests build and smoke-test without publishing. On `main`, a +successful smoke test publishes the same image as +`sha--run--` and `latest`. The run-specific tag +preserves each build, including package refreshes from the same source commit. +Once consumers adopt the override, ordinary CI only pulls it; there is no build +fallback or dependency on the publisher. `latest` deliberately floats, and +Compose always pulls it. Docker's pull output records the resolved digest. + +The Dockerfile uses the same upstream releases as the development services, +with checksummed official GitHub release binaries and a digest-pinned Alpine +base. MinIO and `mc` are AGPL-3.0; their corresponding source is available at +the [MinIO release](https://github.com/minio/minio/tree/RELEASE.2025-09-07T16-13-09Z) +and [mc release](https://github.com/minio/mc/tree/RELEASE.2025-08-13T08-35-41Z). + +## First publication + +The image must exist and be publicly pullable before the CI switch can pass. +Use two separate PRs: Buzz only permits squash merges, so two commits in one +PR cannot stage this rollout. + +1. Merge the publisher-only PR containing `.github/ci/minio/`, + `.github/workflows/minio-image.yml`, and the opt-in `docker-compose.ci.yml`. + Ordinary CI does not select the override yet, so it does not need this image + to validate the publisher PR. The merge triggers the first publication. +2. After publication succeeds, an org/package admin must make **buzz-minio** + public in its GitHub package settings (new GHCR packages default to private, + even for public repositories). Verify an anonymous pull of + `ghcr.io/block/buzz-minio:latest`. +3. Rebase the separate consumer PR onto `main`, run its integration checks, and + merge it. That PR selects the override in relay and mesh lifecycle CI. + +Subsequent publications preserve package visibility. + +## Updating or rebuilding + +Update the release URLs/checksums or base digest in the Dockerfile and open a +PR. Merging triggers publication. To rebuild the existing recipe (for example, +to pick up Alpine package updates), dispatch **MinIO image** from `main`. +Manual dispatch disables Docker's layer cache so the package installation +runs again. Other refs can build and test but cannot publish `latest`. + +To reproduce a CI run with a recorded version, set `MINIO_CI_IMAGE` to +the published run-specific tag or digest while using +`COMPOSE_FILE=docker-compose.yml:docker-compose.ci.yml`. diff --git a/.github/ci/minio/smoke-test.sh b/.github/ci/minio/smoke-test.sh new file mode 100644 index 00000000000..c04f1c0cf1f --- /dev/null +++ b/.github/ci/minio/smoke-test.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# Run on a disposable Docker host with the CI Compose files and a built image. +set -euo pipefail +: "${MINIO_CI_IMAGE:?Set MINIO_CI_IMAGE to the locally built image}" +: "${COMPOSE_FILE:?Select docker-compose.yml and docker-compose.ci.yml}" + +# Exercise the real healthcheck and initializer without pulling over the image +# under test. No application services or tests need a MinIO build step. +docker compose up -d --wait --wait-timeout 90 --pull never minio +docker compose run --rm --no-deps --pull never minio-init +docker compose exec -T minio sh -eu -c ' + minio --version + mc --version + mc alias set local http://localhost:9000 buzz_dev buzz_dev_secret + printf "buzz-minio-smoke\n" > /tmp/expected + mc cp /tmp/expected local/buzz-media/smoke-test + mc cat local/buzz-media/smoke-test > /tmp/actual + cmp /tmp/expected /tmp/actual + status=$(curl --silent --show-error --output /dev/null --write-out "%{http_code}" \ + http://localhost:9000/buzz-media/smoke-test) + test "$status" = 403 + mc rm local/buzz-media/smoke-test + if mc stat local/buzz-media/smoke-test; then + echo "Deleted object is still present" >&2 + exit 1 + fi +' diff --git a/.github/workflows/minio-image.yml b/.github/workflows/minio-image.yml new file mode 100644 index 00000000000..5a9e970dc64 --- /dev/null +++ b/.github/workflows/minio-image.yml @@ -0,0 +1,80 @@ +name: MinIO image + +# A separate publisher, never a dependency of ordinary CI. Image changes get +# a build/smoke test on PRs; only reviewed main commits can move :latest. +on: + push: + branches: [main] + paths: + - '.github/ci/minio/**' + - '.github/workflows/minio-image.yml' + - 'docker-compose.yml' + - 'docker-compose.ci.yml' + pull_request: + paths: + - '.github/ci/minio/**' + - '.github/workflows/minio-image.yml' + - 'docker-compose.yml' + - 'docker-compose.ci.yml' + workflow_dispatch: + +concurrency: + group: minio-image-${{ github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +permissions: + contents: read + +env: + IMAGE_NAME: ghcr.io/block/buzz-minio + COMPOSE_FILE: docker-compose.yml:docker-compose.ci.yml + MINIO_CI_IMAGE: ghcr.io/block/buzz-minio:sha-${{ github.sha }}-run-${{ github.run_id }}-${{ github.run_attempt }} + +jobs: + image: + name: Build and smoke-test MinIO + runs-on: ubuntu-24.04 + timeout-minutes: 15 + permissions: + contents: read + packages: write + steps: + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 + with: + persist-credentials: false + - uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0 + - name: Build image + uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0 + with: + context: .github/ci/minio + platforms: linux/amd64 + load: true + tags: ${{ env.MINIO_CI_IMAGE }} + labels: org.opencontainers.image.revision=${{ github.sha }} + # A maintenance dispatch must refresh apk packages even with a warm + # layer cache. Normal image changes can still reuse cached layers. + no-cache: ${{ github.event_name == 'workflow_dispatch' }} + cache-from: type=gha,scope=minio + cache-to: type=gha,scope=minio,mode=max + - name: Smoke-test CI services + run: bash .github/ci/minio/smoke-test.sh + - name: Clean up smoke test + if: always() + run: docker compose down --volumes + - name: Log in to GHCR + if: github.repository == 'block/buzz' && github.ref == 'refs/heads/main' && github.event_name != 'pull_request' + uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + - name: Publish tested image + if: github.repository == 'block/buzz' && github.ref == 'refs/heads/main' && github.event_name != 'pull_request' + run: | + docker push "$MINIO_CI_IMAGE" + docker tag "$MINIO_CI_IMAGE" "$IMAGE_NAME:latest" + docker push "$IMAGE_NAME:latest" + { + echo "Published $MINIO_CI_IMAGE and $IMAGE_NAME:latest" + echo 'First publication: make the buzz-minio package public before enabling CI consumers.' + } >> "$GITHUB_STEP_SUMMARY" diff --git a/docker-compose.ci.yml b/docker-compose.ci.yml new file mode 100644 index 00000000000..9fe6e7a9a8e --- /dev/null +++ b/docker-compose.ci.yml @@ -0,0 +1,12 @@ +# CI pulls the published image; only minio-image.yml builds it. +# MINIO_CI_IMAGE lets the publisher smoke-test an unpublished image or an +# operator reproduce a run with its recorded run-specific tag/digest. +services: + minio: + image: ${MINIO_CI_IMAGE:-ghcr.io/block/buzz-minio:latest} + platform: linux/amd64 + pull_policy: always + minio-init: + image: ${MINIO_CI_IMAGE:-ghcr.io/block/buzz-minio:latest} + platform: linux/amd64 + pull_policy: always From 2b1c2db158b565a3bf26039aec63c4a3582045f5 Mon Sep 17 00:00:00 2001 From: Tyler <109685178+tlongwell-block@users.noreply.github.com> Date: Thu, 24 Sep 2026 12:28:17 -0400 Subject: [PATCH 2/2] fix(ci): consume the published MinIO image (#7870) Switch every CI job that starts MinIO to the public `ghcr.io/block/buzz-minio:latest` image published by #7869. Integration jobs pull the image; image builds stay in the separate, path-filtered publisher workflow. - Apply the CI Compose override to both desktop integration shards, backend integration, relay E2E, and mesh lifecycle, including script-mediated Compose calls. Both the server and bucket initializer use the Buzz image. - Override both MinIO images in Helm's quickstart CI values, covering the server, bucket initializer, and relay's wait-for-bucket init container in the gated kind installation. - Select integration and mesh checks when either Compose file changes, with regression coverage using the real pinned `dorny/paths-filter` action. publication](https://github.com/block/buzz/actions/runs/36021389137/job/107706874954) succeeded. An anonymous pull using an empty Docker configuration returned the publisher's digest: `sha256:b8470bbeafbf57b20c86cf63804682b714bdcfdbb517f3770247e321e623f48f`. This PR is rebased onto main and ready for consumer integration checks. - Ran the repository's MinIO smoke script against the actual published AMD64 image: healthy startup, bucket initialization, object upload/read/delete, and rejection of anonymous reads all passed. Used the rendered CI Compose services with isolated local ports and resource names. - Verified `mc` runs and writes its configuration as Helm's non-root UID/GID 65532 with dropped capabilities and no privilege escalation. - Helm 3.16.4 lint and quickstart rendering passed; all three rendered MinIO container references use the Buzz image. - All 14 CI selection cases and the required-context isolation check passed. Mesh workflow passes actionlint; shared workflows pass with shellcheck disabled for pre-existing findings. - Fresh repository-wide review traced workflow and script consumers and found no remaining CI image omissions. Full application integration checks and a real kind installation have not been run locally. --------- Signed-off-by: Tyler Longwell (cherry picked from commit 797012ff01a6d499959b45ed2e56f7927c6a4d6b) --- .github/workflows/_ci-relay.yml | 6 ++++++ .github/workflows/ci.yml | 2 ++ .github/workflows/mesh-lifecycle.yml | 6 ++++++ deploy/charts/buzz/ci/quickstart-values.yaml | 2 ++ 4 files changed, 16 insertions(+) diff --git a/.github/workflows/_ci-relay.yml b/.github/workflows/_ci-relay.yml index 5fa7eb53c25..dadbcd5881f 100644 --- a/.github/workflows/_ci-relay.yml +++ b/.github/workflows/_ci-relay.yml @@ -207,6 +207,8 @@ jobs: --archive-file target/ci/postgres-tests.tar.zst desktop-e2e-integration-shard: + env: + COMPOSE_FILE: docker-compose.yml:docker-compose.ci.yml name: Desktop E2E Integration (${{ matrix.shard }}/2) runs-on: ubuntu-latest timeout-minutes: 20 @@ -400,6 +402,8 @@ jobs: echo "Desktop E2E Integration shards passed" backend-integration: + env: + COMPOSE_FILE: docker-compose.yml:docker-compose.ci.yml name: Backend Integration (relay e2e) runs-on: ubuntu-latest timeout-minutes: 20 @@ -541,6 +545,8 @@ jobs: if-no-files-found: ignore relay-e2e: + env: + COMPOSE_FILE: docker-compose.yml:docker-compose.ci.yml name: Relay E2E runs-on: ubuntu-latest timeout-minutes: 20 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a663ef957c5..eb870baed26 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -55,6 +55,8 @@ jobs: - 'deny.toml' - '.github/workflows/ci.yml' - '.github/workflows/_ci-*.yml' + - 'docker-compose.yml' + - 'docker-compose.ci.yml' - 'scripts/run-tests.sh' - 'scripts/model-capabilities.json' - 'scripts/normative-corpus.json' diff --git a/.github/workflows/mesh-lifecycle.yml b/.github/workflows/mesh-lifecycle.yml index c699439980b..1b311574b64 100644 --- a/.github/workflows/mesh-lifecycle.yml +++ b/.github/workflows/mesh-lifecycle.yml @@ -22,6 +22,8 @@ on: - 'scripts/ci-mesh-lifecycle-smoke.sh' - 'scripts/start-relay-for-tests.sh' - '.github/workflows/mesh-lifecycle.yml' + - 'docker-compose.yml' + - 'docker-compose.ci.yml' pull_request: paths: - 'crates/buzz-relay/examples/mesh_*.rs' @@ -34,6 +36,8 @@ on: - 'scripts/ci-mesh-lifecycle-smoke.sh' - 'scripts/start-relay-for-tests.sh' - '.github/workflows/mesh-lifecycle.yml' + - 'docker-compose.yml' + - 'docker-compose.ci.yml' workflow_dispatch: concurrency: @@ -45,6 +49,8 @@ env: jobs: lifecycle-smoke: + env: + COMPOSE_FILE: docker-compose.yml:docker-compose.ci.yml name: Relay-Driven Mesh Lifecycle Smoke runs-on: ubuntu-24.04 timeout-minutes: 45 diff --git a/deploy/charts/buzz/ci/quickstart-values.yaml b/deploy/charts/buzz/ci/quickstart-values.yaml index 4dcf6bcd21b..1a7f4d58389 100644 --- a/deploy/charts/buzz/ci/quickstart-values.yaml +++ b/deploy/charts/buzz/ci/quickstart-values.yaml @@ -9,6 +9,8 @@ redis: enabled: true minio: enabled: true + image: ghcr.io/block/buzz-minio:latest + mcImage: ghcr.io/block/buzz-minio:latest relayUrl: wss://buzz.test.local ownerPubkey: "0000000000000000000000000000000000000000000000000000000000000001" relay: