Skip to content

Commit 533fe6b

Browse files
authored
ci: publish reusable base image cache (#375)
This PR makes the Docker `base` stage a first-class CI artifact and uses it to speed up both CI image builds and devcontainer startup. The new base-image workflow publishes versioned `base` images to GHCR, exports BuildKit registry caches for those base variants, and maintains the default `ghcr.io/bnlnpps/simphony:base` alias used by the devcontainer. - Add `build-push-base.yaml` to build and publish the Dockerfile `base` target for supported CUDA/Ubuntu/OptiX/Geant4/CMake combinations. - Export base-stage BuildKit caches to `ghcr.io/bnlnpps/simphony-buildcache:<base-variant>` with `mode=max`. - Refresh base images on: - manual workflow dispatch - weekly schedule - pushes to `main` that touch base-relevant inputs - Add `pull: true` for base builds so scheduled/manual refreshes pick up updated upstream CUDA parent images. - Add a `base` alias for the default environment: - CUDA 13.0.2 - Ubuntu 24.04 - OptiX 9.0.0 - Geant4 11.4.1 - CMake 4.2.1 - Update PR, push, and release image workflows to import both: - `BASE_CACHE_REF` for shared base-stage layers - `CACHE_REF` for target-specific `develop`/`release` layers - Switch the default devcontainer to pull `ghcr.io/bnlnpps/simphony:base` directly for faster startup. - Document the new published base image variants in the README. The Dockerfile `base` stage is the expensive, slow-moving part of the image build. It installs system dependencies and builds large dependencies such as CLHEP, Geant4, OptiX headers, CMake, and Python dependencies. Publishing this stage separately gives CI and developers a stable reusable foundation while keeping `develop` and `release` image builds focused on source changes. This also makes the devcontainer contract explicit: `ghcr.io/bnlnpps/simphony:base` is now produced by CI rather than being an implied or stale tag. The root devcontainer is optimized for the fast default path by pulling the published base image.
1 parent 1ad5561 commit 533fe6b

6 files changed

Lines changed: 136 additions & 16 deletions

File tree

.devcontainer/devcontainer.json

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,6 @@
11
{
22
"name": "simphony",
3-
"build": {
4-
"dockerfile": "../Dockerfile",
5-
"context": "..",
6-
"target": "base",
7-
// Uncomment to rebuild the devcontainer with custom build args, e.g. a different CUDA version.
8-
//"args": {
9-
// "CUDA_VERSION": "12.5.1"
10-
//},
11-
"cacheFrom": [
12-
"ghcr.io/bnlnpps/simphony:base"
13-
]
14-
},
3+
"image": "ghcr.io/bnlnpps/simphony:base",
154
"features": {
165
"./features/llvm": {
176
"version": "23"

.github/workflows/build-pull-request.yaml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,9 +97,11 @@ jobs:
9797
run: |
9898
IMAGE_NAME=ghcr.io/$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')-buildcache
9999
REF_SANITIZED=PR-${{ github.event.pull_request.number }}
100+
BASE_VARIANT=cuda${{ matrix.cuda_version }}-base-${{ matrix.os }}-optix${{ matrix.optix_version }}-geant4${{ matrix.geant4_version }}-cmake${{ matrix.cmake_version }}
100101
BUILD_VARIANT=cuda${{ matrix.cuda_version }}-${{ matrix.target }}-${{ matrix.os }}-optix${{ matrix.optix_version }}-geant4${{ matrix.geant4_version }}-cmake${{ matrix.cmake_version }}
101102
echo IMAGE_NAME=${IMAGE_NAME} >> $GITHUB_ENV
102103
echo IMAGE_TAG=${REF_SANITIZED}-${BUILD_VARIANT} >> $GITHUB_ENV
104+
echo BASE_CACHE_REF=${IMAGE_NAME}:${BASE_VARIANT} >> $GITHUB_ENV
103105
echo CACHE_REF=${IMAGE_NAME}:${BUILD_VARIANT} >> $GITHUB_ENV
104106
105107
- name: Checkout code
@@ -128,7 +130,9 @@ jobs:
128130
OPTIX_VERSION=${{ matrix.optix_version }}
129131
GEANT4_VERSION=${{ matrix.geant4_version }}
130132
CMAKE_VERSION=${{ matrix.cmake_version }}
131-
cache-from: type=registry,ref=${{ env.CACHE_REF }}
133+
cache-from: |
134+
type=registry,ref=${{ env.BASE_CACHE_REF }}
135+
type=registry,ref=${{ env.CACHE_REF }}
132136
push: ${{ matrix.target == 'develop' && github.event.pull_request.head.repo.full_name == github.repository }}
133137

134138
test-develop-image:
Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
name: Build Push Base Image
2+
3+
on:
4+
workflow_dispatch:
5+
schedule:
6+
- cron: '0 4 * * 1'
7+
push:
8+
branches:
9+
- main
10+
paths:
11+
- Dockerfile
12+
- pyproject.toml
13+
- uv.lock
14+
- optiphy/**
15+
- .github/workflows/build-push-base.yaml
16+
17+
concurrency:
18+
group: ${{ github.workflow }}-${{ github.ref_name || github.event_name }}
19+
cancel-in-progress: true
20+
21+
permissions:
22+
contents: read
23+
packages: write
24+
25+
jobs:
26+
27+
build-push-base:
28+
if: ${{ github.ref_name == 'main' }}
29+
runs-on: ubuntu-latest
30+
31+
strategy:
32+
fail-fast: false
33+
matrix:
34+
include:
35+
- os: ubuntu24.04
36+
cuda_version: 13.2.0
37+
optix_version: 9.1.0
38+
geant4_version: 11.4.1
39+
cmake_version: 4.3.1
40+
default_alias: false
41+
- os: ubuntu24.04
42+
cuda_version: 13.0.2
43+
optix_version: 9.0.0
44+
geant4_version: 11.4.1
45+
cmake_version: 4.2.1
46+
default_alias: true
47+
- os: ubuntu24.04
48+
cuda_version: 12.5.1
49+
optix_version: 9.0.0
50+
geant4_version: 11.4.1
51+
cmake_version: 3.28.3
52+
default_alias: false
53+
- os: ubuntu22.04
54+
cuda_version: 12.1.1
55+
optix_version: 8.0.0
56+
geant4_version: 11.3.2
57+
cmake_version: 3.22.1
58+
default_alias: false
59+
60+
steps:
61+
- name: Define environment variables
62+
run: |
63+
IMAGE_NAME=ghcr.io/$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')
64+
CACHE_IMAGE_NAME=${IMAGE_NAME}-buildcache
65+
BUILD_VARIANT=cuda${{ matrix.cuda_version }}-base-${{ matrix.os }}-optix${{ matrix.optix_version }}-geant4${{ matrix.geant4_version }}-cmake${{ matrix.cmake_version }}
66+
echo IMAGE_NAME=${IMAGE_NAME} >> $GITHUB_ENV
67+
echo IMAGE_TAG=${BUILD_VARIANT} >> $GITHUB_ENV
68+
echo CACHE_REF=${CACHE_IMAGE_NAME}:${BUILD_VARIANT} >> $GITHUB_ENV
69+
70+
- name: Checkout code
71+
uses: actions/checkout@v4
72+
73+
- name: Set up Docker Buildx
74+
uses: docker/setup-buildx-action@v3
75+
76+
- name: Log in to GHCR
77+
uses: docker/login-action@v3
78+
with:
79+
registry: ghcr.io
80+
username: ${{ github.actor }}
81+
password: ${{ secrets.GITHUB_TOKEN }}
82+
83+
- name: Build and push base image
84+
uses: docker/build-push-action@v6
85+
with:
86+
context: .
87+
pull: true
88+
push: true
89+
tags: |
90+
${{ env.IMAGE_NAME }}:${{ env.IMAGE_TAG }}
91+
target: base
92+
build-args: |
93+
OS=${{ matrix.os }}
94+
CUDA_VERSION=${{ matrix.cuda_version }}
95+
OPTIX_VERSION=${{ matrix.optix_version }}
96+
GEANT4_VERSION=${{ matrix.geant4_version }}
97+
CMAKE_VERSION=${{ matrix.cmake_version }}
98+
cache-from: type=registry,ref=${{ env.CACHE_REF }}
99+
cache-to: type=registry,ref=${{ env.CACHE_REF }},mode=max
100+
no-cache-filters: ${{ github.event_name != 'push' && 'base' || '' }}
101+
102+
- name: Add base alias tag for default CUDA
103+
if: ${{ github.ref_name == 'main' && matrix.default_alias }}
104+
run: |
105+
docker buildx imagetools create -t ${{ env.IMAGE_NAME }}:base ${{ env.IMAGE_NAME }}:${{ env.IMAGE_TAG }}
106+
107+
cleanup:
108+
runs-on: ubuntu-latest
109+
needs: build-push-base
110+
steps:
111+
- uses: dataaxiom/ghcr-cleanup-action@v1
112+
with:
113+
packages: ${{ github.event.repository.name }},${{ github.event.repository.name }}-buildcache

.github/workflows/build-push.yaml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,9 +61,11 @@ jobs:
6161
run: |
6262
IMAGE_NAME=ghcr.io/$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')
6363
CACHE_IMAGE_NAME=${IMAGE_NAME}-buildcache
64+
BASE_VARIANT=cuda${{ matrix.cuda_version }}-base-${{ matrix.os }}-optix${{ matrix.optix_version }}-geant4${{ matrix.geant4_version }}-cmake${{ matrix.cmake_version }}
6465
BUILD_VARIANT=cuda${{ matrix.cuda_version }}-${{ matrix.target }}-${{ matrix.os }}-optix${{ matrix.optix_version }}-geant4${{ matrix.geant4_version }}-cmake${{ matrix.cmake_version }}
6566
echo IMAGE_NAME=${IMAGE_NAME} >> $GITHUB_ENV
6667
echo IMAGE_TAG=${BUILD_VARIANT} >> $GITHUB_ENV
68+
echo BASE_CACHE_REF=${CACHE_IMAGE_NAME}:${BASE_VARIANT} >> $GITHUB_ENV
6769
echo CACHE_REF=${CACHE_IMAGE_NAME}:${BUILD_VARIANT} >> $GITHUB_ENV
6870
6971
- name: Checkout code
@@ -93,7 +95,9 @@ jobs:
9395
OPTIX_VERSION=${{ matrix.optix_version }}
9496
GEANT4_VERSION=${{ matrix.geant4_version }}
9597
CMAKE_VERSION=${{ matrix.cmake_version }}
96-
cache-from: type=registry,ref=${{ env.CACHE_REF }}
98+
cache-from: |
99+
type=registry,ref=${{ env.BASE_CACHE_REF }}
100+
type=registry,ref=${{ env.CACHE_REF }}
97101
cache-to: type=registry,ref=${{ env.CACHE_REF }},mode=max
98102

99103
- name: Add devel alias tag for default CUDA

.github/workflows/release.yaml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,12 @@ jobs:
3030
IMAGE_NAME=ghcr.io/$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]')
3131
CACHE_IMAGE_NAME=${IMAGE_NAME}-buildcache
3232
REF_SANITIZED=$(echo "${{ github.ref_name }}" | sed 's/[^a-zA-Z0-9._-]/-/g')
33+
BASE_VARIANT=cuda${{ matrix.cuda_version }}-base-${{ matrix.os }}-optix${{ matrix.optix_version }}-geant4${{ matrix.geant4_version }}-cmake${{ matrix.cmake_version }}
3334
BUILD_VARIANT=cuda${{ matrix.cuda_version }}-${{ matrix.target }}-${{ matrix.os }}-optix${{ matrix.optix_version }}-geant4${{ matrix.geant4_version }}-cmake${{ matrix.cmake_version }}
3435
echo IMAGE_NAME=${IMAGE_NAME} >> $GITHUB_ENV
3536
echo IMAGE_TAG=${REF_SANITIZED}-${BUILD_VARIANT} >> $GITHUB_ENV
3637
echo IMAGE_TAG_SHORT=${REF_SANITIZED} >> $GITHUB_ENV
38+
echo BASE_CACHE_REF=${CACHE_IMAGE_NAME}:${BASE_VARIANT} >> $GITHUB_ENV
3739
echo CACHE_REF=${CACHE_IMAGE_NAME}:${BUILD_VARIANT} >> $GITHUB_ENV
3840
3941
- name: Checkout code
@@ -65,7 +67,9 @@ jobs:
6567
OPTIX_VERSION=${{ matrix.optix_version }}
6668
GEANT4_VERSION=${{ matrix.geant4_version }}
6769
CMAKE_VERSION=${{ matrix.cmake_version }}
68-
cache-from: type=registry,ref=${{ env.CACHE_REF }}
70+
cache-from: |
71+
type=registry,ref=${{ env.BASE_CACHE_REF }}
72+
type=registry,ref=${{ env.CACHE_REF }}
6973
cache-to: type=registry,ref=${{ env.CACHE_REF }},mode=max
7074

7175
cleanup:

README.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,11 +51,17 @@ and development-environment details.
5151

5252
## Supported Images
5353

54-
The `build-push.yaml` workflow publishes the versioned container tags below to GHCR on pushes to `main`, while `release.yaml` publishes the `latest` alias on tagged releases. Tag entries link to the
54+
The `build-push-base.yaml` workflow publishes reusable `base` images for development environments and Docker build
55+
cache warmup. The `build-push.yaml` workflow publishes the versioned container tags below to GHCR on pushes to `main`,
56+
while `release.yaml` publishes the `latest` alias on tagged releases. Tag entries link to the
5557
[Simphony package page](https://github.com/BNLNPPS/simphony/pkgs/container/simphony).
5658

5759
| Target | OS | CUDA | OptiX | Geant4 | Alias | Tag |
5860
|---|---|---:|---:|---:|---|---|
61+
| `base` | `ubuntu24.04` | `13.2.0` | `9.1.0` | `11.4.1` | | [cuda13.2.0-base-ubuntu24.04-optix9.1.0-geant411.4.1-cmake4.3.1](https://github.com/BNLNPPS/simphony/pkgs/container/simphony) |
62+
| `base` | `ubuntu24.04` | `13.0.2` | `9.0.0` | `11.4.1` | `base` | [cuda13.0.2-base-ubuntu24.04-optix9.0.0-geant411.4.1-cmake4.2.1](https://github.com/BNLNPPS/simphony/pkgs/container/simphony) |
63+
| `base` | `ubuntu24.04` | `12.5.1` | `9.0.0` | `11.4.1` | | [cuda12.5.1-base-ubuntu24.04-optix9.0.0-geant411.4.1-cmake3.28.3](https://github.com/BNLNPPS/simphony/pkgs/container/simphony) |
64+
| `base` | `ubuntu22.04` | `12.1.1` | `8.0.0` | `11.3.2` | | [cuda12.1.1-base-ubuntu22.04-optix8.0.0-geant411.3.2-cmake3.22.1](https://github.com/BNLNPPS/simphony/pkgs/container/simphony) |
5965
| `release` | `ubuntu24.04` | `13.2.0` | `9.1.0` | `11.4.1` | | [cuda13.2.0-release-ubuntu24.04-optix9.1.0-geant411.4.1-cmake4.3.1](https://github.com/BNLNPPS/simphony/pkgs/container/simphony) |
6066
| `release` | `ubuntu24.04` | `13.0.2` | `9.0.0` | `11.4.1` | `latest` | [cuda13.0.2-release-ubuntu24.04-optix9.0.0-geant411.4.1-cmake4.2.1](https://github.com/BNLNPPS/simphony/pkgs/container/simphony) |
6167
| `release` | `ubuntu22.04` | `12.1.1` | `8.0.0` | `11.3.2` | | [cuda12.1.1-release-ubuntu22.04-optix8.0.0-geant411.3.2-cmake3.22.1](https://github.com/BNLNPPS/simphony/pkgs/container/simphony) |

0 commit comments

Comments
 (0)