[Shashank Mittal (shashank-iitbhu)]

Description

This PR changes four things in CI:

1. Tagged releases publish a second asset, caterpillar-linux-arm64, for EKS nodes on AWS Graviton.
2. The Docker Hub push, which has never worked, is rebuilt to publish a multi-arch (linux/amd64 + linux/arm64) patternoss/caterpillar image, following the same pattern as patterninc/heimdall's docker-image.yml.
3. Pull requests now run go vet and go test.
4. Both workflows get current action versions and least-privilege permissions.

The existing caterpillar asset does not change. It keeps the same name and is built by the same command on x86-64 glibc (ubuntu-24.04, which is what ubuntu-latest resolves to today). Images and tools that download releases/download/<tag>/caterpillar keep working.

Merge order and prerequisites

• build: require Go 1.27 (go.mod, Dockerfile builder, README) #133 must merge before this PR. The two PRs are no longer independent. The new Docker jobs build build/Dockerfile, and on main that file still uses golang:1.24.7-alpine, which can't build the module (go.mod needs a newer Go). build: require Go 1.27 (go.mod, Dockerfile builder, README) #133 moves it to golang:1.27-alpine. If this PR merges first, the Docker jobs fail on every push to main until build: require Go 1.27 (go.mod, Dockerfile builder, README) #133 lands. The binary jobs and the GitHub release are not affected either way.
• A repository admin must add two repository secrets before the Docker jobs can pass: DOCKERHUB_USERNAME and DOCKERHUB_TOKEN (a Docker Hub access token that can push to patternoss/caterpillar). This repo has neither today, which is why the old login step always failed with "Username and password required". heimdall has them as repository secrets. Until they are added, both Docker jobs fail at their first step with an error naming the missing secret(s).

.github/workflows/release.yaml

Triggers. The workflow runs on pushes to main, v* tags and the latest tag (the same triggers as heimdall's image workflow).

Push	Binary jobs	release (GitHub release)	Docker image tag
main	build and verify	no	dev
v1.2.3 tag	build, verify, upload artifact	yes	1.2.3
latest tag	skipped	no	latest

Job graph. There are two independent chains with no needs between them, so a Docker Hub failure can't block the release assets, and the other way round.

build-linux-amd64 ─┐
                   ├─> release        (v* tags only)
build-linux-arm64 ─┘

docker-build (amd64, arm64) ──> docker-merge

• Native linux/arm64 binary. A new build-linux-arm64 job runs on the GitHub-hosted ubuntu-24.04-arm runner, which is free for public repos. It uses the same CGO_ENABLED=1 GOOS=linux go build command as the x86-64 build and runs natively, so cgo needs no cross-compiling. confluent-kafka-go v2.15.0 ships a prebuilt librdkafka_glibc_linux_arm64.a and links it statically.
• Binary jobs. build-linux-amd64 and build-linux-arm64 each build their binary, check the file output (ELF 64-bit ... x86-64 or ELF 64-bit ... ARM aarch64) and run test/pipelines/hello_name.yaml with it. On v* tags they also upload the binary as a workflow artifact. They don't run for the latest tag.
• release runs on v* tags only, after both binary jobs succeed. It downloads both binaries, checks their architectures again and uploads both in one softprops/action-gh-release call with fail_on_unmatched_files: true. Two jobs never write the release at the same time, and a release can't be published with an asset missing. The conditions changed from refs/tags/ to refs/tags/v, so pushing the latest tag never creates a GitHub release.
• docker-build is a matrix job. It builds build/Dockerfile (default target, no changes to the Dockerfile) for linux/amd64 on ubuntu-24.04 (the same pinned runner as the binary job) and for linux/arm64 on ubuntu-24.04-arm. Each platform builds on its own native runner, so no QEMU is needed. Each pushes its image to Docker Hub by digest (push-by-digest=true, untagged) and uploads the digest as a digest-<arch> artifact. It uses the GitHub Actions build cache, scoped per platform, and sets fail-fast: false, as heimdall does.
• docker-merge runs after both docker-build legs succeed:
  • It works out the tag from the ref: <version> with the v stripped for v* tags, dev for main, and latest for the latest tag. It fails on any other ref, or if the result isn't a valid Docker tag.
  • It requires exactly two digests, then runs docker buildx imagetools create to push the manifest list.
• Image verification after the push (docker-merge):
  • docker buildx imagetools inspect must list both linux/amd64 and linux/arm64. Attestation entries (unknown/unknown) are ignored.
  • For each platform, it runs docker create --platform <p>, then docker cp to copy /caterpillar out of the container, and requires file to report x86-64 or ARM aarch64 respectively. create and cp don't execute the image, so the arm64 check needs no QEMU on the x86-64 runner.
• Missing secrets. Both Docker jobs start with a step that receives DOCKERHUB_USERNAME/DOCKERHUB_TOKEN through env, checks each with -z, and fails with an ::error:: naming whichever is missing. This replaces the old, obscure login error, and the jobs never skip silently.
• The image has no ENTRYPOINT, as before. The binary is /caterpillar, so callers name it in the command: docker run patternoss/caterpillar:dev /caterpillar -conf ....
• Permissions. The workflow defaults to contents: read, which covers every job except release. Only release gets contents: write, and it doesn't check out or run any repository code. The Docker jobs push with the Docker Hub token, not GITHUB_TOKEN.
• Untrusted values in shell. The ref, tag and digest reach run: scripts through env, not ${{ }} interpolation.
• Concurrency. There's one run per ref. A newer push to main cancels the superseded main run, but a tag run is never cancelled mid-publish.
• x86-64 runner pinned to ubuntu-24.04 instead of ubuntu-latest. This doesn't change the binary today. It means the asset's glibc baseline only moves when we decide to move it, not when GitHub moves ubuntu-latest. GitHub's runner annotation on this PR's CI run says ubuntu-latest starts moving to Ubuntu 26 on October 19, 2026.

Tag scheme change. The old (never successful) step tagged latest on every tag push and main on pushes to main. This PR uses heimdall's scheme instead: latest only moves when the latest git tag is pushed, and main pushes are tagged dev.

.github/workflows/ci.yaml (pull requests)

• Go 1.24.7 → 1.27, the current supported release.
  • Both workflows use go-version: '1.27', which installs the latest 1.27.x.
  • go-version-file: go.mod would install the exact patch named in go.mod instead.
• New go vet -copylocks=false ./... step.
  • Plain go vet ./... fails on main today with three existing copylocks findings: archive.go:81, compress.go:46 and file.go:223. All three copy a task struct that embeds task.Base.
  • Only that check is turned off, and a comment in the workflow says why. Every other vet analyzer now gates PRs. The copylocks findings should be fixed in a separate PR, and the check turned back on then.
• New go test -race ./... step.
  • The tests (internal/pkg/pipeline, .../ack, .../task) need no external services, so nothing is excluded.
  • They pass with -race, which matters for the concurrency code in ack and task.
• Permissions and concurrency. The workflow gets permissions: contents: read, and a new push to a PR cancels the run for its previous commit.
• Job name unchanged. The job is still called build, so any check names people rely on stay the same.

README.md

A new "Docker Image" section describes patternoss/caterpillar: its two platforms, the three tags (<version>, dev, latest) and when each is pushed. It also notes there is no ENTRYPOINT and gives an example docker run. The section doesn't touch the Go-version line that #133 edits, so the two PRs merge without conflicts (checked with git merge-tree).

Both workflows

• Actions updated. The old majors run on Node 20, and every run already logs a deprecation warning for it. All the majors below run on Node 24.

  Action	Before	After
  actions/checkout	v4	v7
  actions/setup-go	v5	v7
  actions/upload-artifact	v4	v7
  actions/download-artifact	v4	v8
  softprops/action-gh-release	v2	v3.0.3
  docker/setup-buildx-action	v3	v4
  docker/login-action	v3	v4
  docker/build-push-action	v5	v7
  docker/metadata-action	v5	removed (the tag is set by a shell step, as in heimdall)

  • None of the breaking changes in these majors affect how this repo uses them. Binary artifacts are downloaded by name, and digests by pattern.
  • The Docker actions are one major ahead of heimdall's (v3/v3/v6). The breaking changes are the move to Node 24 and ESM, plus removed deprecated inputs and env vars that this workflow doesn't use.
  • The GitHub-maintained and Docker actions stay on major tags. softprops/action-gh-release is pinned to its v3.0.3 commit (efb3536…), because it is a third-party action that runs with a token that can write releases.

• persist-credentials: false on checkout. No job pushes to git, so the token doesn't need to stay in .git/config.

Validation done locally

• actionlint: v1.7.12 reports 0 errors on both workflows, run through the rhysd/actionlint Docker image with shellcheck 0.11.0 enabled.
• linux/arm64 image: build/Dockerfile builds for linux/arm64, with build: require Go 1.27 (go.mod, Dockerfile builder, README) #133's builder change applied in a scratch copy. I built it two ways: from this branch with only that one-line change, and from this branch merged with build: require Go 1.27 (go.mod, Dockerfile builder, README) #133. In both images, /caterpillar -conf test/pipelines/hello_name.yaml runs and prints its output. The image has no entrypoint, and file reports ELF 64-bit LSB executable, ARM aarch64 for /caterpillar.
• Merge and verify scripts: I ran the exact run: scripts from docker-merge against a local registry. The inputs were two stand-in images pushed by digest with buildx (push-by-digest=true, default provenance attestations).
  • imagetools create produced a list with linux/amd64, linux/arm64 and two attestation entries.
  • The verify step passed and reported x86-64 and ARM aarch64 for the two /caterpillar files.
  • With only the arm64 digest, verify fails with has no linux/amd64 image. With one digest file, create fails with Expected 2 platform digests.
  • The tag step returns dev, 2.3.4 and latest for main, v2.3.4 and latest. It rejects refs/tags/v, other branches, and a tag name containing $(...).
• CI steps in golang:1.27-bookworm (go1.27.3, linux/arm64): go build, go vet -copylocks=false ./... and go test -race ./... all pass.
  • I ran them on this branch, on this branch merged with current main and build: require Go 1.27 (go.mod, Dockerfile builder, README) #133, and with -race -count=10 to look for flaky tests.
  • Plain go vet ./... reports the same three copylocks findings on main, so they aren't caused by this PR.
• arm64 binary: the same native build in golang:1.27-bookworm produces ELF 64-bit LSB executable, ARM aarch64 ... for GNU/Linux. ldd shows only libc and libm, because librdkafka is linked statically. The binary runs hello_name.yaml. When this job was first added, the same build also completed a Kafka write/read round trip against a local broker. The build command hasn't changed since, and I didn't repeat that test for this push.
• Not verified locally:
  • The x86-64 binary and the x86-64 image. This Mac can't build x86-64 natively. The binary build command hasn't changed, and the image uses the same Dockerfile as arm64.
  • The workflows on GitHub itself. release.yaml doesn't run on pull requests, so the Docker jobs first run on the push to main after this merges. That run needs build: require Go 1.27 (go.mod, Dockerfile builder, README) #133 merged and the two secrets added.

How to verify after merging

# After the next push to main (needs #133 and the DOCKERHUB_* secrets):
docker buildx imagetools inspect patternoss/caterpillar:dev   # lists linux/amd64 and linux/arm64

# After the next v* tag:
gh release view <tag> --repo patterninc/caterpillar --json assets --jq '.assets[].name'
# caterpillar
# caterpillar-linux-arm64
docker buildx imagetools inspect patternoss/caterpillar:<version without v>

Types of changes

• Docs change / refactoring / dependency upgrade
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation and I have updated the documentation accordingly.
• I have added tests to cover my changes.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.