Skip to content
Build & Test Suite

ci: bump amazonlinux/linux kernel tags #195

Sign in to view logs

ci: bump amazonlinux/linux kernel tags

ci: bump amazonlinux/linux kernel tags #195

Workflow file for this run

.github/workflows/suite.yml at 0f69e12
name: Build & Test Suite
on:
push:
branches: [ "main", "feat/**" ]
pull_request:
branches: [ "main" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.26.3'
- name: Install clang + squashfs-tools
# squashfs-tools provides `unsquashfs`, used by the squashfs
# writer's external-validator tests and the rootfs pipeline's
# external round-trip tests. Those tests skip cleanly when
# the binary is absent, so omitting this step would silently
# downgrade them to no-ops on every CI run — defeating their
# purpose of catching on-wire format regressions before the
# firecracker-runtime job tries to boot against a broken
# rootfs.
run: sudo apt-get update && sudo apt-get install -y clang squashfs-tools
- name: Build
run: go build -v ./...
- name: Test
run: go test -v ./...
- name: Build-check e2e-tagged tests
# `//go:build e2e` files are invisible to the default `go build`
# / `go test` above, so a refactor that breaks their API (e.g.
# 561d6cc removing config.OAuthConfig) lands green and only
# surfaces when someone runs the suite locally with -tags=e2e.
# `-run=^$` skips execution — the e2e suite needs a host clang
# and live listeners we don't want in this fast job. We only
# care that it compiles.
run: go test -tags=e2e -count=1 -run=^$ ./internal/e2e/...
- name: Cross-build release binaries
# Produces dist/<os>-<arch>/{hpcc,hpcc-agent,hpcc-pause}[.exe]
# for linux/windows/darwin × amd64/arm64. CGO is off in the
# Makefile so this works from the plain ubuntu runner with no
# extra cross-toolchain setup.
run: make dist
- name: Upload release binaries
uses: actions/upload-artifact@v4
with:
name: hpcc-binaries
path: dist/
if-no-files-found: error
image-store:
name: Image store integration
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.26.3'
- name: Start containerd
run: |
# GitHub-hosted Ubuntu runners install the `containerd.io` package
# as part of the Docker setup. The systemd unit isn't always
# active by default — make sure it is, then loosen the socket
# permissions so the unprivileged runner user can dial it without
# sudo (the socket is normally root:root 0600).
sudo systemctl start containerd
sudo systemctl --no-pager status containerd
sudo chmod 666 /run/containerd/containerd.sock
ls -l /run/containerd/containerd.sock
- name: Probe containerd
run: |
# Sanity-check the socket is actually serving the gRPC API before
# the test job runs — otherwise the failure mode is a confusing
# connection-refused inside Go.
sudo ctr version
- name: Run integration tests
env:
# Default path matches what the test code already expects, but
# set it explicitly so this workflow remains valid if the test
# default ever changes.
CONTAINERD_ADDRESS: /run/containerd/containerd.sock
# Integration tests are behind the `integration` build tag so
# the regular CI's `go test ./...` doesn't even compile them.
run: go test -tags=integration -count=1 -v ./internal/worker/image/
windows-build:
name: Windows build & unit tests
runs-on: windows-2022
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.26.3'
- name: Build
# The Windows side of the worker (runtime/hcsshim.go, the
# cdimage store, the pause binary) must compile and link
# against the Windows containerd/hcsshim deps without falling
# back to build tags. Catching that regression on every push
# is what this job exists for — the Linux build matrix can't
# see it.
#
# No test step on purpose: the package-level unit tests run
# on Linux in the `build` job, and several test files in this
# repo predate Windows support (Linux-only path strings,
# `/bin/echo`, syscall.Stat_t, etc.). End-to-end Windows
# coverage is the `windows-runtime` job below, which
# exercises the runhcs path against a real containerd.
run: go build ./...
windows-runtime:
name: Hcsshim runtime e2e
runs-on: windows-2022
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.26.3'
- name: Install containerd + runhcs shim
shell: pwsh
# The official Windows containerd release tarball ships
# containerd.exe, ctr.exe, and containerd-shim-runhcs-v1.exe
# under bin/; that's everything we need. The shim must sit
# alongside containerd on PATH or be discoverable at the
# shim path containerd embeds — extracting into
# C:\Program Files\containerd\bin satisfies both.
run: |
$ErrorActionPreference = 'Stop'
$version = '1.7.27'
$url = "https://github.com/containerd/containerd/releases/download/v$version/containerd-$version-windows-amd64.tar.gz"
$tar = "$env:RUNNER_TEMP\containerd.tar.gz"
$install = 'C:\Program Files\containerd'
New-Item -Path $install -ItemType Directory -Force | Out-Null
Invoke-WebRequest -Uri $url -OutFile $tar -UseBasicParsing
tar.exe -xzf $tar -C $install
"$install\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
& "$install\bin\containerd.exe" --version
& "$install\bin\containerd-shim-runhcs-v1.exe" --version
- name: Register and start containerd service
shell: pwsh
# --register-service writes a Windows service that runs
# containerd.exe under LocalSystem. The default config
# (snapshotter `windows`, runtime `io.containerd.runhcs.v1`,
# pipe at \\.\pipe\containerd-containerd) is exactly what
# the integration test expects, so we don't override it.
run: |
$ErrorActionPreference = 'Stop'
$install = 'C:\Program Files\containerd'
& "$install\bin\containerd.exe" --register-service
Start-Service containerd
# Wait for the named pipe to appear so the test doesn't
# race the service. WaitForSingleObject-on-pipe is the
# canonical pattern; PowerShell doesn't expose it, so we
# poll Test-Path which works against \\.\pipe paths.
$deadline = (Get-Date).AddSeconds(60)
while ((Get-Date) -lt $deadline) {
if (Test-Path \\.\pipe\containerd-containerd) { break }
Start-Sleep -Milliseconds 500
}
if (-not (Test-Path \\.\pipe\containerd-containerd)) {
Write-Error 'containerd pipe never appeared'
Get-Service containerd | Format-List *
exit 1
}
& "$install\bin\ctr.exe" version
- name: Build hpcc-pause.exe
shell: pwsh
# The pause/ go module is independent of the main module
# on purpose (kept tiny so the injected binary doesn't drag
# in the worker's deps). Build it standalone and surface the
# path to the integration test via HPCC_HCSSHIM_PAUSE.
run: |
$ErrorActionPreference = 'Stop'
$out = "$env:RUNNER_TEMP\hpcc-pause.exe"
Push-Location pause
go build -o $out .
Pop-Location
"HPCC_HCSSHIM_PAUSE=$out" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
- name: Run hcsshim runtime integration test
# The integration tag flips the build-tagged Windows-only
# file into the test binary. Process isolation is selected
# by the test itself (GitHub-hosted windows-2022 runners
# don't expose nested virtualization, so Hyper-V isolation
# would fail to boot the utility VM).
#
# The timeout is the only ceiling on this test — the test
# body intentionally doesn't impose per-step timeouts so it
# can be retargeted at a larger toolchain image (an MSVC
# build image with Visual Studio installed is ~30 GB) by
# bumping this flag alone. The current nanoserver-only run
# finishes in a couple of minutes; 60m is the headroom for
# future heavier images and for cold-cache pulls on slow
# registry CDNs.
env:
HPCC_HCSSHIM_ADDRESS: \\.\pipe\containerd-containerd
timeout-minutes: 75
run: go test -tags=integration -count=1 -v -timeout=60m -run TestHcsshim_EndToEnd_Integration ./internal/worker/runtime/...
windows-runtime-hyperv:
name: Hcsshim runtime e2e (Hyper-V isolation)
# Self-hosted runner labelled `nested` for nested-virt-capable
# Hyper-V hosts. GitHub-hosted windows-2022 runners can't boot a
# utility VM, so the process-isolation job above is all we can run
# there; this job is what gives us coverage of the real audit-
# recognised boundary (Hyper-V utility VM + hpcc-agent over
# HvSocket). Skipped automatically when no matching runner is
# available.
runs-on: [self-hosted, nested]
steps:
- uses: actions/checkout@v4
- name: Set up Go
# No-op on a self-hosted runner that already has the right
# version. setup-go is idempotent and adds to PATH for this
# job either way.
uses: actions/setup-go@v4
with:
go-version: '1.26.3'
- name: Build hpcc-pause.exe and hpcc-agent.exe
shell: powershell
# The Hyper-V path needs BOTH binaries staged: pause.exe goes
# into cdimage's prepared-image alias (worker config still
# carries it for the process-isolation fallback) and the
# agent is the actual PID 1 of the utility VM. Both go to
# $RUNNER_TEMP so artefacts don't accumulate in the working
# copy across runs.
run: |
$ErrorActionPreference = 'Stop'
$pause = "$env:RUNNER_TEMP\hpcc-pause.exe"
$agent = "$env:RUNNER_TEMP\hpcc-agent.exe"
Push-Location pause
go build -o $pause .
Pop-Location
Push-Location agent
go build -o $agent .
Pop-Location
"HPCC_HCSSHIM_PAUSE=$pause" | Out-File -FilePath $env:GITHUB_ENV -Encoding ascii -Append
"HPCC_HCSSHIM_AGENT=$agent" | Out-File -FilePath $env:GITHUB_ENV -Encoding ascii -Append
- name: Run Hyper-V isolation integration test
# HPCC_HCSSHIM_RUN_HYPERV unblocks TestHcsshim_HyperV_EndToEnd_Integration;
# without it the test skips so the same test binary is
# safe to run on non-nested hosts. PauseHostPath /
# AgentHostPath come from the previous step's GITHUB_ENV.
#
# 75-minute job timeout / 60-minute test timeout matches the
# process-isolation job so a future bigger image (MSVC, ~30
# GB) doesn't need separate ceiling-tuning per isolation mode.
env:
HPCC_HCSSHIM_ADDRESS: \\.\pipe\containerd-containerd
HPCC_HCSSHIM_RUN_HYPERV: '1'
timeout-minutes: 75
run: go test -tags=integration -count=1 -v -timeout=60m -run TestHcsshim_HyperV_EndToEnd_Integration ./internal/worker/runtime/...
firecracker-runtime:
name: Firecracker runtime e2e
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.26.3'
- name: Enable KVM group perms
# Default Ubuntu udev rules give /dev/kvm to the `kvm` group only;
# firecracker runs as the dropped jailer uid (nobody by default in
# this test) which isn't in that group. Mode 0666 + static_node is
# the standard runner workaround for this.
run: |
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
sudo udevadm control --reload-rules
sudo udevadm trigger --name-match=kvm
ls -l /dev/kvm
- name: Install firecracker + jailer
# Pinned to a known-working release. The release archive layout
# is `release-<version>-<arch>/{firecracker,jailer}-<version>-<arch>`.
run: |
set -euo pipefail
VERSION=v1.15.1
ARCH=$(uname -m)
curl -fSL "https://github.com/firecracker-microvm/firecracker/releases/download/${VERSION}/firecracker-${VERSION}-${ARCH}.tgz" \
-o /tmp/fc.tgz
mkdir -p /tmp/fc
tar -xzf /tmp/fc.tgz -C /tmp/fc
sudo install -m 0755 "/tmp/fc/release-${VERSION}-${ARCH}/firecracker-${VERSION}-${ARCH}" /usr/local/bin/firecracker
sudo install -m 0755 "/tmp/fc/release-${VERSION}-${ARCH}/jailer-${VERSION}-${ARCH}" /usr/local/bin/jailer
firecracker --version
jailer --version
- name: Download test kernel
# Firecracker publishes CI-validated kernels under spec.ccfc.min.
# We only need a vmlinux that boots — the rootfs is built in-test
# from a public busybox image so we don't need a published rootfs.
run: |
set -euo pipefail
ARCH=$(uname -m)
mkdir -p /tmp/fcassets
curl -fSL "https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/v1.15/${ARCH}/vmlinux-6.1.155" \
-o /tmp/fcassets/vmlinux
ls -l /tmp/fcassets/vmlinux
- name: Run firecracker runtime e2e
# `sudo -E` because jailer needs root for cgroup setup and
# chroot. We pass through PATH so `go` resolves, and the
# GOPATH/GOMODCACHE so go test reuses the runner's module cache.
env:
HPCC_FIRECRACKER_BIN: /usr/local/bin/firecracker
HPCC_JAILER_BIN: /usr/local/bin/jailer
HPCC_TEST_KERNEL: /tmp/fcassets/vmlinux
run: |
sudo -E env "PATH=$PATH" \
go test -tags=integration -count=1 -v -timeout=10m \
-run TestFirecracker_.*_Integration \
./internal/worker/runtime/...