Skip to content

Commit a6692e5

Browse files
timzsutimzsu
committed
patch image publish workflow
Signed-off-by: Zhengyuan Su <su.zhengyuan@u.nus.edu>
1 parent cca591d commit a6692e5

2 files changed

Lines changed: 106 additions & 12 deletions

File tree

‎.github/workflows/image-publish.yml‎

Lines changed: 66 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,67 @@ env:
2626
IMAGE_NAME: mlsys-io/lumilake_server
2727

2828
jobs:
29+
build-pr:
30+
name: Build image
31+
if: github.event_name == 'pull_request'
32+
runs-on: ubuntu-latest
33+
permissions:
34+
contents: read
35+
steps:
36+
- name: Checkout
37+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
38+
with:
39+
persist-credentials: false
40+
41+
- name: Set up Buildx
42+
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
43+
44+
- name: Compute image tags
45+
id: tags
46+
env:
47+
SHA: ${{ github.sha }}
48+
run: |
49+
set -eu
50+
short_sha="${SHA::7}"
51+
echo "tags=${REGISTRY}/${IMAGE_NAME}:pr-${short_sha}" >> "$GITHUB_OUTPUT"
52+
53+
- name: Set up uv
54+
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
55+
with:
56+
version: "0.11.8"
57+
python-version: "3.12"
58+
# No uv cache restore in a publishing workflow (cache-poisoning).
59+
enable-cache: false
60+
61+
- name: Sync server requirements from uv.lock
62+
run: uv run --frozen scripts/dev/sync_requirements.py --write
63+
64+
- name: Verify lockfile sync was a no-op
65+
run: |
66+
if ! git diff --quiet src/lumilake_server/requirements.txt; then
67+
echo "src/lumilake_server/requirements.txt drifted from uv.lock." >&2
68+
echo "Run: uv run scripts/dev/sync_requirements.py --write" >&2
69+
git --no-pager diff src/lumilake_server/requirements.txt
70+
exit 1
71+
fi
72+
73+
- name: Build and push
74+
id: build
75+
uses: docker/build-push-action@1dc73863535b631f98b2378be8619f83b136f4a0 # v6.17.0
76+
with:
77+
context: .
78+
file: ./Dockerfile
79+
push: false
80+
tags: ${{ steps.tags.outputs.tags }}
81+
provenance: false
82+
build-args: |
83+
BUILD_VERSION=${{ github.ref_name }}
84+
BUILD_REF=${{ github.sha }}
85+
BUILD_CREATED=${{ github.event.repository.updated_at }}
86+
2987
build-and-push:
30-
name: Build multi-arch image
88+
name: Build and push multi-arch image
89+
if: github.event_name == 'push'
3190
runs-on: ubuntu-latest
3291
permissions:
3392
contents: read
@@ -39,13 +98,12 @@ jobs:
3998
persist-credentials: false
4099

41100
- name: Set up QEMU
42-
uses: docker/setup-qemu-action@5306bad0baa6b616b9934712d4b9bc20069df9bf # v3.6.0
101+
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
43102

44103
- name: Set up Buildx
45104
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
46105

47106
- name: Log in to GHCR
48-
if: github.event_name != 'pull_request'
49107
uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0
50108
with:
51109
registry: ${{ env.REGISTRY }}
@@ -58,18 +116,15 @@ jobs:
58116
REF_NAME: ${{ github.ref_name }}
59117
REF_TYPE: ${{ github.ref_type }}
60118
SHA: ${{ github.sha }}
61-
EVENT_NAME: ${{ github.event_name }}
62119
run: |
63120
set -eu
64121
short_sha="${SHA::7}"
65122
tags=("${REGISTRY}/${IMAGE_NAME}:${short_sha}")
66123
# :latest is owned by release-images.yml (gated on env approval).
67-
if [[ "${EVENT_NAME}" == "push" ]]; then
68-
if [[ "${REF_TYPE}" == "tag" && "${REF_NAME}" == v* ]]; then
69-
tags+=("${REGISTRY}/${IMAGE_NAME}:${REF_NAME}")
70-
elif [[ "${REF_NAME}" == "main" ]]; then
71-
tags+=("${REGISTRY}/${IMAGE_NAME}:dev")
72-
fi
124+
if [[ "${REF_TYPE}" == "tag" && "${REF_NAME}" == v* ]]; then
125+
tags+=("${REGISTRY}/${IMAGE_NAME}:${REF_NAME}")
126+
elif [[ "${REF_NAME}" == "main" ]]; then
127+
tags+=("${REGISTRY}/${IMAGE_NAME}:dev")
73128
fi
74129
( IFS=,; echo "tags=${tags[*]}" ) >> "$GITHUB_OUTPUT"
75130
@@ -100,7 +155,7 @@ jobs:
100155
context: .
101156
file: ./Dockerfile
102157
platforms: linux/amd64,linux/arm64
103-
push: ${{ github.event_name != 'pull_request' }}
158+
push: true
104159
tags: ${{ steps.tags.outputs.tags }}
105160
provenance: false
106161
build-args: |
@@ -109,7 +164,6 @@ jobs:
109164
BUILD_CREATED=${{ github.event.repository.updated_at }}
110165
111166
- name: Smoke-test pushed image
112-
if: github.event_name != 'pull_request'
113167
env:
114168
SHA: ${{ github.sha }}
115169
run: |

‎docs/FIRST_RELEASE_ROADMAP.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# First Release Roadmap
2+
3+
This document tracks the remaining work before publishing Lumilake's first OSS release. The migration cleanup has already removed the private repository history, dead legacy surfaces, and non-OSS module references. The remaining work is now release hardening rather than source migration.
4+
5+
## 1. Documentation and Package Polish
6+
7+
Clean up public-facing docs so they describe the current OSS project only.
8+
9+
- Update `CONTRIBUTING.md` to remove stale references to `requirements.txt`, `sync_requirements.py`, the `requirements-sync` pre-commit hook, and the `requirements-sync` CI workflow.
10+
- Update workflow documentation to match the current CI set: lint/typecheck, tests, security, package build, env examples, PR title, and DCO sign-off.
11+
- Remove the permanent forbidden legacy env-key list from `scripts/dev/check_env_examples.py`; the env example check should validate the current contract without carrying a historical denylist.
12+
- Add package metadata in `pyproject.toml`: license expression, authors/maintainers, project URLs, keywords, and classifiers.
13+
- Add concise public reference docs for the surfaces users need after install: environment variables, CLI commands, API overview, architecture, and plugin/hooks model.
14+
- Ensure README stays as the short quick start and points to the deeper docs instead of duplicating them.
15+
- Update the issue templates to match the structure of Flowmesh
16+
17+
## 2. E2E Release Check
18+
19+
Run the project through a real user flow on the self-hosted environment and fix any issues found.
20+
21+
- Start from a clean checkout and install with the documented commands.
22+
- Run `lumilake deploy init` and inspect the generated `.env`.
23+
- Fill the required local test values and run `lumilake deploy up`.
24+
- Verify server health and API docs are reachable.
25+
- Submit at least one representative workflow through the CLI or SDK.
26+
- Watch the job to completion and verify result retrieval.
27+
- Confirm runtime artifacts and intermediate outputs go to `S3_ARCHIVE_PREFIX` or lumid.data, not container-local storage.
28+
- Tear the deployment down with the documented command and confirm no unexpected local state is left behind.
29+
30+
The e2e check should be recorded as a release checklist item, not committed as a secret-bearing test. If a reusable script is added, it must accept all credentials and endpoints from the environment and avoid embedding local paths or private data.
31+
32+
## 3. Release Machinery
33+
34+
Add the workflows and docs needed to publish repeatably.
35+
36+
- Add a release documentation page covering version bump, lock refresh, tag policy, TestPyPI validation, PyPI publishing, and rollback/yank guidance.
37+
- Add release metadata validation, including tag-to-version consistency.
38+
- Add a GitHub release workflow using pinned actions and trusted publishing where possible.
39+
- Decide whether the first release publishes only the `lumilake` package or also any split helper packages.
40+
- Run the package build workflow before tagging and verify the built wheel/sdist import cleanly in a fresh environment.

0 commit comments

Comments
 (0)