Skip to content

fix(ci): stamp unique Ev2 buildver.txt per build to prevent chart freeze#1704

Merged
zanejohnson-azure merged 1 commit into
ci_prodfrom
zane/fix-ev2-buildver-dynamic
Jun 2, 2026
Merged

fix(ci): stamp unique Ev2 buildver.txt per build to prevent chart freeze#1704
zanejohnson-azure merged 1 commit into
ci_prodfrom
zane/fix-ev2-buildver-dynamic

Conversation

@zanejohnson-azure

Copy link
Copy Markdown
Contributor

Problem

The Arc-extension Helm chart published to MCR (oci://mcr.microsoft.com/azuremonitor/containerinsights/prod1/stable/azuremonitor-containers) froze at 3.3.0 even after charts/azuremonitor-containerinsights/Chart.yaml was bumped to 3.4.0 (#1699).

Root cause: the Ev2 Managed-SDP rollouts read their artifacts version from a static, checked-in buildver.txt (referenced as versionFile in each RolloutSpec.json). Because that value never changed (1.0.0.1 for the chart-push group), Ev2 deduplicated artifact registration — logging "already registered … skipping artifacts registration" — and kept rolling out the stale bundle that was first registered back in mid-May. Stage_1 (pushChartToAcr.shhelm package with no --version) publishes whatever Chart.yaml version is inside that frozen bundle, so the published chart stayed at 3.3.0.

Fix

Mirror how Azure/prometheus-collector (ama-metrics) handles this: dynamically write $(Build.BuildNumber) into every service group's buildver.txt during the setup task of the build pipeline (def 444), before the deployment artifacts are copied to staging. A unique version every build forces Ev2 to register fresh artifacts, so the current Chart.yaml version actually ships.

Service groups stamped:

  • mergebranch-multiarch-agent-deployment
  • mergebranch-multiarch-agent-deployment-Managed-SDP
  • arc-k8s-extension
  • arc-k8s-extension-Managed-SDP ← the frozen chart-push group
  • arc-k8s-extension-release-v2
  • arc-k8s-extension-release-v2-Managed-SDP

buildver.txt lives outside artifacts.tar.gz and is copied separately by the CopyFiles@2 step, so stamping it in the setup task (after the tar steps) is correct.

Notes / rollout

  • The durable fix only takes effect on the next def-444 build. To actually ship 3.4.0: merge → trigger a ci_prod build of def 444 → re-run the prod-release pipeline Stage_1 (ci-arc-k8s-extension-all-regions-prod-release(MCR)) → approve → verify helm pull … --version 3.4.0 from MCR (allow MCR mirror lag). Do not run Stage_2+ (they are trigger: manual).
  • Build.BuildNumber defaults to YYYYMMDD.r (e.g. 20260602.3), which is numerically greater than the old 1.0.0.x labels, so there is no version-regression risk for Ev2.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

The Arc-extension Helm chart published to MCR froze at 3.3.0 even after
Chart.yaml was bumped to 3.4.0. Ev2 Managed-SDP rollouts read their
artifacts version from a static checked-in buildver.txt (RolloutSpec.json
versionFile). Because the value never changed, Ev2 deduped registration
("already registered ... skipping artifacts registration") and kept rolling
out the stale bundle; Stage_1 pushChartToAcr.sh packages straight from the
frozen Chart.yaml, so the published chart version stayed 3.3.0.

Mirror Azure/prometheus-collector: write $(Build.BuildNumber) into each
service group's buildver.txt in the setup task (before the deployment
CopyFiles), forcing fresh Ev2 artifact registration on every build.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@zanejohnson-azure zanejohnson-azure requested a review from a team as a code owner June 2, 2026 22:01
@zanejohnson-azure

Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@zanejohnson-azure zanejohnson-azure enabled auto-merge (squash) June 2, 2026 23:20
@zanejohnson-azure zanejohnson-azure merged commit badf4fa into ci_prod Jun 2, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants