proxmoxvm: guest bootstrap strategy — Ansible/SSH (A) vs cloud-init templates (B)

[patrick-hermann-sva]

Context

The native machinery/proxmoxvm Configuration (#79, planned in #78) provisions a Proxmox VM by cloning a prepared template via the bpg EnvironmentVM MR. Open question: how does the guest get bootstrapped (SSH key / network / hostname, then base-OS config)?

Decision so far: go with Path A first for the PoC. This issue documents both options so we can decide the long-term direction later.

The two layers of "cloud-init" (why this isn't obvious)

1. Provider/Proxmox side — bpg's initialization {} block attaches a cloud-init drive to the clone regardless of the template. This always "works" at the API level.
2. Guest side — for that drive to actually inject anything, the guest OS in the template must have cloud-init installed, enabled, and reset (cloud-init clean before templatizing; NoCloud/ConfigDrive datasource allowed).

Our current templates (e.g. sthings-u26, VMID 110) are Packer + Ansible images built around a baked-in sthings user + SSH + DHCP + qemu-guest-agent — not cloud-init images (no ciuser/ipconfig0/sshkeys; root disk on virtio0). The existing Terraform module provisions over vm_ssh_user/password, i.e. SSH, not cloud-init. So a cloud-init drive attached to these templates is most likely inert in the guest.

Option A — Ansible/SSH over existing templates ✅ (chosen first)

• Flow: clone → boot with template's baked-in sthings user + DHCP → qemu-guest-agent reports IP onto status.share.ip → optional AnsibleRun (already wired in proxmoxvm, reusing cicd/ansible-run) does base-OS config over SSH.
• Pros: zero template work; mirrors the proven Terraform flow exactly; ready now.
• Cons: relies on baked-in credentials in the image; not a credential-less/declarative bootstrap.

Code implication for Path A

The Composition currently always emits an initialization block, which would force a (likely-inert) cloud-init drive onto these templates. For Path A this should become conditional — emit initialization only when cloud-init fields are actually set on the XR (cloudInit.sshKeys / username / explicit ipv4Address / datastoreId), so a plain sthings-u26 clone stays pure-Ansible and cloud-init is strictly opt-in.

• Make initialization conditional in apis/composition.yaml (opt-in on cloud-init fields).
• Realistic examples/xr.yaml drops the cloudInit block (pure Path A); keep a cloud-init example under xr-max.yaml.
• Live PoC against LabUL: clone VMID 110 → IP on status.share.ip → AnsibleRun over SSH → teardown.

Option B — cloud-init-native templates (deferred)

• Rebuild the Packer images so the guest is cloud-init-ready: install + enable cloud-init, allow the NoCloud/ConfigDrive datasource, cloud-init clean before converting to template, and drop the baked-in interactive creds.
• Then EnvironmentVM.initialization injects SSH key + IP + hostname at first boot — credential-less, fully declarative.
• Pros: cleaner cloud-native story; no baked-in secrets; less reliance on Ansible for day-0.
• Cons: a Packer pass per template (sthings-u26, sthings-u24, ubuntu24, rocky9, …); new behaviour to validate; coordination with the packer/ builds.

Open questions

• Does sthings-u26 actually ship the cloud-init package (even if unconfigured)? Ubuntu cloud images include it by default; a Packer build may have removed/disabled it. Worth confirming in the image before assuming Path B scope.
• If we adopt B, where do the Packer changes live (stuttgart-things/stuttgart-things packer/builds/...) and do we keep A as a fallback for non-cloud-init OSes (e.g. the Windows templates)?

Decision

• Now: Path A (Ansible/SSH), no template changes.
• Later (this issue): whether to invest in Path B cloud-init templates for a credential-less declarative path.

Refs #78, #79