Custom Windows Node Images for vSphere Kubernetes Service: Build, Configure, and Deploy with vSphere Kubernetes Service Image Builder
A complete technical guide to building production-ready Windows node images for vSphere Kubernetes Service on VCF 9 — including custom application installation with Ansible.
Author: Julius Nicolescu Date: March 2026
Running Windows workloads on Kubernetes in a VMware Cloud Foundation (VCF) environment requires custom-built Windows node images that are pre-configured with the Kubernetes stack — containerd, kubelet, CNI plugins, and all required Windows prerequisites. VMware provides the vks-image-builder tool to automate this process using HashiCorp Packer and Ansible.
This guide walks through:
- Building a Windows Server 2022 Kubernetes node image from scratch using vks-image-builder
- Publishing the image to a vSphere Content Library
- Registering the image with a VKS Supervisor for use in Windows node pools
- Customizing the golden image by embedding additional software — demonstrated with Notepad++ (a simple MSI install) and Fluentd / Fluent Package v6.0 (a complex service installation with configuration management)
- How the Build Pipeline Works
- Prerequisites
- Build Host Preparation
- Clone the vks-image-builder Repository
- Upload Windows ISOs to vSphere
- Configure vSphere Packer Variables
- Prepare the Windows Unattended Answer File
- Set the Windows Administrator Password
- Build the Windows 2022 Node Image
- Monitor and Troubleshoot the Build
- Publish the Image to a vSphere Content Library
- Register the Content Library with the Supervisor
- Verify the Image is Available
- Understanding the Windows Customization Pattern
- Customization: Install Notepad++
- Customization: Install Fluentd (Fluent Package v6.0)
- Verify Customizations on a Running Node
Understanding the architecture before running any commands helps prevent the most common build failures.
Linux Build Host
$HOME/vks-image-builder/ ← cloned from GitHub
│
├── ansible-windows/ ← Ansible role for Windows VMs
│ ├── files/custom/ ← binaries / config files
│ ├── tasks/main.yml ← Ansible task entry point
│ └── tasks/<custom>.yml ← your customization tasks
│
└── packer-variables/ ← vSphere + OS-specific vars
├── vsphere.j2 ← vCenter connection settings
└── windows/vsphere-windows.j2 ← ISO paths on datastore
$HOME/image-builder/image/ ← OVA output directory
$HOME/image-builder/windows_autounattend.xml ← unattended setup
Docker
├── artifacts container ← serves K8s release binaries
└── image builder container ← runs Packer + Ansible
│ WinRM (5986)
│ HTTP (8082 - answer file + K8s bins)
▼
vSphere Cluster
Packer VM ──── boots from Windows ISO
──── autounattend.xml drives unattended setup
──── Ansible runs over WinRM: containerd, kubelet,
CNI, custom packages
──── Packer exports OVA → $HOME/image-builder/image/
Build phases:
| Phase | Duration | What Happens |
|---|---|---|
| Boot & Windows Setup | ~20–30 min | VM boots from ISO; autounattend.xml drives fully automated Windows installation and initial reboots |
| Packer WinRM Connect | ~2–5 min | Packer waits for WinRM to become available, then connects |
| Ansible Provisioning | ~30–45 min | Ansible configures containerd, kubelet, CNI, Windows prerequisites, and any custom tasks from tasks/main.yml |
| Sysprep & Export | ~10–15 min | Windows is sysprepped, the VM is shut down, and Packer exports the OVA |
Important: The total build time is typically 60–90 minutes. If the Packer VM is migrated by DRS during the build, Packer loses its WinRM connection and the build hangs. Create a DRS override rule to pin the Packer VM to a single host before starting.
| Tool | Purpose |
|---|---|
| Docker (linux/amd64) | Runs the image builder and artifact server containers |
govc |
VMware vSphere CLI — uploads ISOs and manages content libraries |
git |
Clones the vks-image-builder repository |
curl |
Downloads the Windows answer file template |
make |
Drives the build pipeline |
sed |
Patches configuration files inline |
| File | Where to Get It |
|---|---|
Windows Server 2022 ISO (en-us_windows_server_2022_v21H2_x64_dvd_20348.iso) |
Microsoft Evaluation Center |
VMware Tools ISO (VMware-Tools-windows-12.5.4-24964629.iso) |
Broadcom Support KB368758 |
- vCenter Server with at least one cluster, datastore, and VM network reachable from the build host
- A dedicated VM folder in vCenter to contain build VMs (e.g.,
vcf9-content) - DRS configured with a VM-Host affinity rule to prevent vMotion of the Packer build VM
| Traffic | Direction | Ports |
|---|---|---|
| Packer HTTP server (answer file + K8s binaries) | Build host → vSphere VM | TCP 8082 (configurable, use range 8080–8090) |
| WinRM (Ansible provisioning) | Build host → vSphere VM | TCP 5986 |
| vCenter API | Build host → vCenter | TCP 443 |
Packer starts an HTTP server on the build host that the booting Windows VM fetches the answer file from. Open the port range before running any builds:
firewall-cmd --permanent --add-port=8080-8090/tcp
firewall-cmd --reload
firewall-cmd --list-allRemove any artefacts from prior builds to guarantee a clean environment:
# Remove previous build workspace directories
rm -rf $HOME/image-builder $HOME/vks-image-builder
# Stop and remove all Docker containers
docker stop $(docker ps -q)
docker rm $(docker ps -aq)
# Remove all Docker images to ensure the builder image is rebuilt fresh
docker rmi -f $(docker images -aq)
# Verify clean state
docker ps -a
docker imagesgit clone https://github.com/vmware/vks-image-builder.git $HOME/vks-image-builder
cd $HOME/vks-image-builder
# Confirm the supported OS target list — windows-2022-efi must appear
make list-supported-osExpected output:
photon-5
ubuntu-2204-efi
ubuntu-2404-efi
windows-2022-efi
The Packer build reads the Windows Server 2022 installation ISO and VMware Tools ISO directly from a vSphere datastore. Both must be uploaded before the build starts.
export GOVC_URL='https://vcsa01.vcf.nicolescu.org'
export GOVC_USERNAME='administrator@vsphere.local'
export GOVC_PASSWORD='<your-vcenter-password>'
export GOVC_INSECURE=1
# Upload Windows Server 2022 Standard ISO
govc datastore.upload \
--ds="lab-cl01-ds-vsan01" \
--dc="lab-dc01" \
/data/ISO/en-us_windows_server_2022_v21H2_x64_dvd_20348.iso \
ISO/en-us_windows_server_2022_v21H2_x64_dvd_20348.iso
# Upload VMware Tools ISO
govc datastore.upload \
--ds="lab-cl01-ds-vsan01" \
--dc="lab-dc01" \
/data/ISO/VMware-Tools-windows-12.5.4-24964629.iso \
ISO/VMware-Tools-windows-12.5.4-24964629.iso
# Create the VM folder that will hold the Packer build VM
govc folder.create /lab-dc01/vm/vcf9-contentTwo JSON variable files control how Packer connects to vSphere and where it places the build VM. Both files use the .j2 extension (Jinja2 template format consumed by the image builder).
This file is shared across all OS targets (Ubuntu, Photon, Windows):
cat << 'EOF' > $HOME/vks-image-builder/packer-variables/vsphere.j2
{
"vcenter_server": "vcsa01.vcf.nicolescu.org",
"username": "administrator@vsphere.local",
"password": "<your-vcenter-password>",
"datacenter": "lab-dc01",
"datastore": "lab-cl01-ds-vsan01",
"folder": "vcf9-content",
"cluster": "lab-cl01",
"network": "lab-cl01-vds01-pg-vm-mgmt",
"insecure_connection": "true",
"linked_clone": "true",
"create_snapshot": "true",
"destroy": "true"
}
EOFKey parameters:
| Parameter | Purpose |
|---|---|
folder |
vSphere VM folder where the Packer build VM is created |
destroy |
When true, Packer deletes the build VM from vSphere after exporting the OVA — the OVA file on disk is kept |
create_snapshot |
Creates a snapshot before the build completes — used internally by the export process |
insecure_connection |
Skips TLS certificate verification when connecting to vCenter |
This file tells Packer where to find the Windows installation and VMware Tools ISOs on the datastore. The path syntax uses vSphere's datastore bracket notation:
cat << 'EOF' > $HOME/vks-image-builder/packer-variables/windows/vsphere-windows.j2
{
"os_iso_path": "[lab-cl01-ds-vsan01] ISO/en-us_windows_server_2022_v21H2_x64_dvd_20348.iso",
"vmtools_iso_path": "[lab-cl01-ds-vsan01] ISO/VMware-Tools-windows-12.5.4-24964629.iso"
}
EOFWindows requires an XML answer file (autounattend.xml) that drives fully automated, non-interactive installation. The upstream Kubernetes image-builder project provides a reference file. Download it and apply two customizations: set the Administrator password and switch the OS edition to the full Desktop Experience.
mkdir -p $HOME/image-builder/image
# Download the upstream reference answer file
curl https://raw.githubusercontent.com/kubernetes-sigs/image-builder/refs/heads/main/images/capi/packer/ova/windows/windows-2022-efi/autounattend.xml \
-o $HOME/image-builder/windows_autounattend-default.xml
# Keep the original as a reference backup
cp $HOME/image-builder/windows_autounattend-default.xml \
$HOME/image-builder/windows_autounattend.xml
# Patch 1: Set the Administrator password for unattended setup
sed -i 's|<Value></Value>|<Value><your-admin-password></Value>|g' \
$HOME/image-builder/windows_autounattend.xml
# Patch 2: Switch from SERVERSTANDARDCORE to the full Desktop Experience edition
sed -i 's|<Value>Windows Server 2022 SERVERSTANDARDCORE</Value>|<Value>Windows Server 2022 SERVERSTANDARD</Value>|g' \
$HOME/image-builder/windows_autounattend.xmlWindows Server 2022 edition selection:
| XML Value | Description | Recommended For |
|---|---|---|
SERVERSTANDARD |
Standard with Desktop Experience (full GUI) | VKS Windows nodes — best compatibility |
SERVERSTANDARDCORE |
Standard Server Core (no GUI, smaller footprint) | Minimal installs only |
SERVERDATACENTER |
Datacenter with Desktop Experience | Production enterprise environments |
SERVERDATACENTERCORE |
Datacenter Server Core | Minimal production installs |
The password used in
autounattend.xmland theWINDOWS_ADMIN_PASSWORDPacker variable (set in the next step) must match. Packer uses this password to connect to the VM over WinRM after Windows installation completes.
In addition to the answer file, the Packer variable file for Windows must also know the Administrator password so it can authenticate over WinRM during the Ansible provisioning phase. Patch the defaults file:
sed -i '/"enable_auto_kubelet_service_restart": "false"/c\
"enable_auto_kubelet_service_restart": "false",\
"windows_admin_password": "<your-admin-password>"' \
$HOME/vks-image-builder/packer-variables/windows/default-args-windows.j2This sed command inserts "windows_admin_password" immediately after the enable_auto_kubelet_service_restart key, keeping the JSON valid.
The TKR_SUFFIX is a versioning label appended to the OVA filename and embedded in the image metadata. It must correspond to a Kubernetes release version that exists on your VKS Supervisor. Query the Supervisor before building:
kubectl get kubernetesrelease | grep v1.34.1Output:
v1.34.1---vmware.1-vkr.5 v1.34.1+vmware.1-vkr.5 True True 5d
# ^^^^^ this is the TKR_SUFFIX value to use
export IMAGE_ARTIFACTS_PATH=$HOME/image-builder/image
export ARTIFACTS_CONTAINER_PORT=8081
export PACKER_HTTP_PORT=8082
export KUBERNETES_VERSION=v1.34.1+vmware.1
export TKR_SUFFIX=vkr.5
export DEBUGGING=true
export WINDOWS_ADMIN_PASSWORD='<your-admin-password>'Variable reference:
| Variable | Purpose |
|---|---|
IMAGE_ARTIFACTS_PATH |
Host directory where OVA files are written after the build |
ARTIFACTS_CONTAINER_PORT |
Port on the build host where the Kubernetes release artifact server listens |
PACKER_HTTP_PORT |
Port on the build host where Packer serves the Windows answer file |
KUBERNETES_VERSION |
The Kubernetes version to install in the node image |
TKR_SUFFIX |
The TKR (Tanzu Kubernetes Release) suffix label appended to the OVA name |
DEBUGGING |
When true, enables verbose Packer and Ansible output in the build log |
WINDOWS_ADMIN_PASSWORD |
Administrator password — must match the value in autounattend.xml |
cd $HOME/vks-image-builder
# Clean any previous containers and output files
# Equivalent to: clean-containers + clean-image-artifacts
make clean
# Pull and start the Kubernetes release artifact server
# This container serves the kubelet, kubeadm, CNI, and containerd binaries
# that Ansible installs inside the Windows VM during the build
make run-artifacts-container
# Build the Docker image for the Packer/Ansible toolchain
make build-image-builder-container
# Confirm the target is listed
make list-supported-os
# Build the Windows Server 2022 node image
make build-node-image \
OS_TARGET=windows-2022-efi \
TKR_SUFFIX=vkr.5 \
HOST_IP=192.168.200.7 \
ARTIFACTS_CONTAINER_PORT=8081 \
PACKER_HTTP_PORT=8082 \
IMAGE_ARTIFACTS_PATH=$HOME/image-builder/image \
AUTO_UNATTEND_ANSWER_FILE_PATH=$HOME/image-builder/windows_autounattend.xml \
WINDOWS_ADMIN_PASSWORD='<your-admin-password>'HOST_IP is the IP address of the build host as seen from the vSphere VM network — not the vCenter IP. The Packer VM will fetch the answer file and Kubernetes binaries from this address during the build.
The build runs in a detached Docker container. Stream its logs:
docker logs -f v1.34.1---vmware.1-windows-2022-efi-image-builderThe container name format is <kubernetes-version>-<os-target>-image-builder.
If the build hangs or you need to inspect the Packer configuration, open a shell into the running container:
docker exec -it v1.34.1---vmware.1-windows-2022-efi-image-builder /bin/bashThe image builder merges all variable files into a single JSON before passing them to Packer. Inspect the final merged values:
# Copy the image builder workspace to the local filesystem
docker cp v1.34.1---vmware.1-windows-2022-efi-image-builder:/image-builder /tmp/image-builder
# Review the final merged Packer variables
cat /tmp/image-builder/images/capi/packer-variables.json| Symptom | Root Cause | Resolution |
|---|---|---|
| Build hangs after Packer reports "Waiting for WinRM" | Packer VM was vMotioned to a different ESXi host mid-build | Add a DRS VM-Host affinity rule to pin the Packer VM before starting |
| WinRM connection timeout | Windows firewall blocking WinRM, or autounattend.xml password mismatch | Verify the password matches in both autounattend.xml and WINDOWS_ADMIN_PASSWORD; check port 5986 |
| HTTP 404 on answer file fetch | Packer HTTP server unreachable from the vSphere VM network | Ensure PACKER_HTTP_PORT (8082) is open on the build host firewall and HOST_IP is correct |
| Artifact container fails to pull | Kubernetes release not available in the registry | Verify KUBERNETES_VERSION matches an available release and the build host has internet access |
| Ansible task failure | Task file syntax error or missing file in files/custom/ |
Check the Ansible error in docker logs, fix the task YAML, and rerun after make clean |
After a successful build, the OVA lands in $IMAGE_ARTIFACTS_PATH under a subdirectory named after the OS target. Both the Linux control plane images and the Windows node image must reside in the same Content Library, because even a Windows-only workload cluster requires Linux VMs for the Kubernetes control plane.
export GOVC_URL='https://vcsa01.vcf.nicolescu.org'
export GOVC_USERNAME='administrator@vsphere.local'
export GOVC_PASSWORD='<your-vcenter-password>'
export GOVC_INSECURE=1
# Create the content library (skip if it already exists)
govc library.create -ds lab-cl01-ds-vsan01 vcf9-content-library
# Enable self-signed certificate acceptance for the library
govc library.update -k=true vcf9-content-library
# Upload all three OVAs
vcf_library_item=(
"$HOME/image-builder/image/ova-photon-5/photon-5-amd64-v1.34.1---vmware.1-vkr.5.ova"
"$HOME/image-builder/image/ova-ubuntu-2404/ubuntu-2404-amd64-v1.34.1---vmware.1-vkr.5.ova"
"$HOME/image-builder/image/ova-windows-2022/windows-2022-amd64-v1.34.1---vmware.1-vkr.5.ova"
)
for item in "${vcf_library_item[@]}"; do
echo "Uploading [ ${item} ]"
govc library.import vcf9-content-library "${item}"
doneThe content library is registered at the Supervisor level and applies globally — it is not scoped to individual vSphere Namespaces.
Via the vSphere UI:
- Navigate to Supervisor Management → Supervisors → select your supervisor
- Select Configure → General
- Under Kubernetes Service, add
vcf9-content-library
Upgrade the VKS Service version if your target Kubernetes version requires a newer service version:
- Navigate to Supervisor Management → Services → Kubernetes Service
- Select Actions → Manage Service
- Update the version — for example, from
3.4.1-embedded+v1.33to3.5.0+v1.34
Connect via the VCF CLI and confirm the release is available:
export SUPERVISOR_IP=192.168.230.7
# Refresh or create the supervisor context
vcf context refresh supervisor --insecure-skip-tls-verify
# Switch to the supervisor namespace
vcf context use supervisor
# List all available Kubernetes releases and confirm the newly built one is present
vcf kubernetes-release getThe newly built release should appear with COMPATIBLE: True and ACTIVE: True:
NAME VERSION COMPATIBLE ACTIVE
v1.34.1---vmware.1-vkr.5 v1.34.1+vmware.1-vkr.5 True True ← new image
After the Content Library is synchronized, the Supervisor creates VirtualMachineImage (VMI) objects for each imported OVA. Verify the Windows image is present and correctly typed:
# List all OS images for the target Kubernetes version
kubectl get osimage | grep v1.34.1+vmware.1Expected output — a Windows 2022 entry must appear:
NAME VERSION OS VERSION ARCH TYPE AGE
vmi-12cf4a77696ba5e9b v1.34.1+vmware.1 ubuntu 24.04 amd64 cvmi 6m27s
vmi-6ddf7f28cfef477f8 v1.34.1+vmware.1 windows 2022 amd64 cvmi 5m6s
vmi-706ea95af1adaba0b v1.34.1+vmware.1 photon 5 amd64 cvmi 7m21s
# Confirm the vkr.5-suffixed images are present as ClusterVirtualMachineImages
kubectl get cvmi | grep vkr.5Expected output:
NAME DISPLAY-NAME TYPE K8S-VERSION GUESTOS
vmi-12cf4a77696ba5e9b ubuntu-2404-amd64-v1.34.1---vmware.1-vkr.5 OVF v1.34.1+vmware.1-vkr.4 ubuntu64Guest
vmi-6ddf7f28cfef477f8 windows-2022-amd64-v1.34.1---vmware.1-vkr.5 OVF v1.34.1+vmware.1-vkr.4 windows2019srvNext_64Guest
vmi-706ea95af1adaba0b photon-5-amd64-v1.34.1---vmware.1-vkr.5 OVF v1.34.1+vmware.1-vkr.4 vmwarePhoton64Guest
The Windows image is now available for use in VKS Windows node pools.
Before adding custom applications, it helps to understand how customization hooks into the build pipeline.
The ansible-windows directory inside vks-image-builder is an Ansible role that runs against the Windows VM over WinRM during the Packer provisioning phase. The role has a conventional structure:
$HOME/vks-image-builder/ansible-windows/
│
├── defaults/
│ └── main.yml ← Ansible role default variables
│
├── files/
│ └── custom/ ← Files to be transferred into the Windows VM
│ ├── fluentd.conf ← Fluentd configuration
│ ├── fluent-package-6.0.1-x64.msi
│ └── npp.8.9.1.Installer.x64.msi
│
├── tasks/
│ ├── main.yml ← Entry point — import_tasks entries go here
│ ├── common.yml ← Built-in: Windows common prerequisites
│ ├── containerd.yml ← Built-in: containerd runtime
│ ├── registry.yml ← Built-in: container registry configuration
│ ├── notepadpp.yml ← Custom: Notepad++ installation
│ └── fluentd.yml ← Custom: Fluentd installation and service config
│
└── templates/
├── containerd/config.toml
└── registry/config.yml
Every customization follows the same three-step pattern:
Step 1: Place files
Copy installer MSIs and config files into:
ansible-windows/files/custom/<filename>
Step 2: Create a task file
Write an Ansible task YAML in:
ansible-windows/tasks/<name>.yml
Step 3: Register the task
Append to ansible-windows/tasks/main.yml:
- import_tasks: <name>.yml
| Module | Purpose |
|---|---|
ansible.windows.win_file |
Create directories, delete files |
ansible.windows.win_copy |
Transfer files from build host into the Windows VM |
ansible.windows.win_shell |
Run PowerShell or cmd.exe commands |
ansible.windows.win_stat |
Check if a path exists (used for verification) |
ansible.windows.win_service |
Manage Windows services (start mode, state) |
ansible.builtin.fail |
Abort the build with a custom error message |
Ansible's win_shell module supports a creates argument — a path that, if it already exists, causes the shell task to be skipped entirely. This makes installer steps idempotent:
- name: Install Notepad++ silently
ansible.windows.win_shell: |
msiexec.exe /i C:\Temp\npp.8.9.1.Installer.x64.msi /qn /norestart
args:
creates: 'C:\Program Files\Notepad++\notepad++.exe'If notepad++.exe already exists (e.g., on a re-run), the msiexec command is skipped.
Notepad++ is a simple example of embedding a GUI application into a Windows Kubernetes node image using a silent MSI installation. While a text editor may not be a production use case, the pattern is identical for any MSI-packaged software.
Place the installer in the files/custom/ directory of the Ansible role on the build host. The Ansible win_copy module will transfer this file into the Windows VM during the build.
mkdir -p $HOME/vks-image-builder/ansible-windows/files/custom
curl -L -o $HOME/vks-image-builder/ansible-windows/files/custom/npp.8.9.1.Installer.x64.msi \
https://github.com/notepad-plus-plus/notepad-plus-plus/releases/download/v8.9.1/npp.8.9.1.Installer.x64.msi
ls -lh $HOME/vks-image-builder/ansible-windows/files/custom/npp.8.9.1.Installer.x64.msiOffline environments: If the build host does not have internet access, download the MSI separately and copy it to the path above using
scpor another file transfer method.
Create the task file that will be executed by Ansible inside the Windows VM during the build:
cat << 'EOF' > $HOME/vks-image-builder/ansible-windows/tasks/notepadpp.yml
# ---------------------------------------------------------------------------
# Customization: Install Notepad++ (Unattended MSI Install)
# ---------------------------------------------------------------------------
# Purpose:
# - Copies the Notepad++ MSI installer into the Windows image during build
# - Performs a fully silent installation using msiexec
# - Verifies installation and cleans up the installer afterward
# ---------------------------------------------------------------------------
- name: Ensure C:\Temp directory exists for custom installers
ansible.windows.win_file:
path: C:\Temp
state: directory
- name: Copy Notepad++ MSI into the Windows image
ansible.windows.win_copy:
src: files/custom/npp.8.9.1.Installer.x64.msi
dest: C:\Temp\npp.8.9.1.Installer.x64.msi
- name: Install Notepad++ silently using MSI unattended mode
ansible.windows.win_shell: |
msiexec.exe /i C:\Temp\npp.8.9.1.Installer.x64.msi /qn /norestart
args:
creates: 'C:\Program Files\Notepad++\notepad++.exe'
register: npp_install
- name: Verify Notepad++ installation succeeded
ansible.windows.win_stat:
path: 'C:\Program Files\Notepad++\notepad++.exe'
register: npp_check
- name: Fail build if Notepad++ is not installed correctly
ansible.builtin.fail:
msg: "Notepad++ installation failed or binary not found."
when: not npp_check.stat.exists
- name: Remove Notepad++ MSI installer after installation
ansible.windows.win_file:
path: C:\Temp\npp.8.9.1.Installer.x64.msi
state: absent
# ---------------------------------------------------------------------------
# End: Notepad++ customization
# ---------------------------------------------------------------------------
EOFTask walkthrough:
| Step | Module | Purpose |
|---|---|---|
Create C:\Temp |
win_file |
Ensures a staging directory exists for installers |
| Copy MSI | win_copy |
Transfers the installer from the build host into the Windows VM via WinRM |
Run msiexec |
win_shell |
/qn = silent, /norestart = suppress reboot; creates: makes it idempotent |
| Verify binary | win_stat |
Checks notepad++.exe exists — a missing binary means the install silently failed |
| Fail fast | fail |
Aborts the entire Packer build with a clear error message if verification fails |
| Cleanup | win_file |
Removes the MSI from the image — keeps the OVA size minimal |
cat << 'EOF' >> $HOME/vks-image-builder/ansible-windows/tasks/main.yml
- import_tasks: notepadpp.yml
EOFRe-run the full build from Section 9. Once the image is deployed to a node, verify the installation:
Test-Path "C:\Program Files\Notepad++\notepad++.exe"
# Expected: TrueFluentd is a production-grade example of a more complex customization — it involves installing a Windows service, shipping a default configuration file into the image, validating that configuration at build time, and ensuring the service starts automatically on every node boot.
Fluent Package v6.0 is the successor to td-agent (Treasure Agent). On Windows, it installs via a standard MSI and registers a Windows service named fluentdwinsvc.
Default install paths after MSI installation:
| Path | Purpose |
|---|---|
C:\opt\fluent |
Base installation directory |
C:\opt\fluent\bin\fluentd.bat |
CLI wrapper — use this to run Fluentd commands |
C:\opt\fluent\bin\fluent-gem.bat |
Gem installer for Fluentd plugins |
C:\opt\fluent\etc\fluent\fluentd.conf |
Active configuration file |
C:\opt\fluent\log\ |
Fluentd's own log output |
C:\opt\fluent\pos\ |
Position files — track how far Fluentd has read each log file |
C:\opt\fluent\buffer\ |
Buffered log data awaiting flush |
fluentdwinsvc |
Windows service name |
mkdir -p $HOME/vks-image-builder/ansible-windows/files/custom
curl -L -o $HOME/vks-image-builder/ansible-windows/files/custom/fluent-package-6.0.1-x64.msi \
https://fluentd.cdn.cncf.io/lts/6/windows/fluent-package-6.0.1-x64.msi
ls -lh $HOME/vks-image-builder/ansible-windows/files/custom/fluent-package-6.0.1-x64.msiThe Fluentd configuration defines what logs to collect, how to enrich them, and where to send them. Create it on the build host — Ansible will transfer it into the image during the build.
mkdir -p $HOME/vks-image-builder/ansible-windows/files/custom
cat << 'EOF' > $HOME/vks-image-builder/ansible-windows/files/custom/fluentd.conf
# =============================================================================
# SECTION 0: GLOBAL SETTINGS
# =============================================================================
# log_level controls Fluentd's own operational verbosity.
# Use 'info' for production; 'debug' when troubleshooting.
<system>
log_level info
</system>
# =============================================================================
# SECTION 1: INPUTS
# =============================================================================
# INPUT 1A: Tail Kubernetes container logs on this Windows node.
# The containerd runtime writes logs to C:\var\log\containers\<pod>_<ns>_<container>-<id>.log
# New log files matching the wildcard are picked up automatically as pods are scheduled.
# pos_file tracks read offsets — never delete it on a running system, or Fluentd
# will re-read all existing log files from the beginning and produce duplicates.
<source>
@type tail
@id in_tail_k8s_containers
path C:\var\log\containers\*.log
pos_file C:\opt\fluent\pos\containers.pos
refresh_interval 5
read_from_head true
tag kube.windows
<parse>
@type none
</parse>
</source>
# INPUT 1B: Forward receiver — accept logs pushed from other Fluent Bit / Fluentd agents.
# Listens on TCP/24224 (the standard Fluentd forward protocol port).
# Comment out this block entirely if you do not need log forwarding.
<source>
@type forward
@id in_forward
bind 0.0.0.0
port 24224
</source>
# =============================================================================
# SECTION 2: FILTERS
# =============================================================================
# FILTER 2A: Enrich kube.windows records with cluster name and node hostname.
# k8s_cluster reads the K8S_CLUSTER_NAME environment variable set on the node.
# k8s_node resolves to the Windows hostname at runtime.
# To set the cluster name before starting fluentdwinsvc:
# [System.Environment]::SetEnvironmentVariable('K8S_CLUSTER_NAME','my-cluster','Machine')
<filter kube.windows>
@type record_transformer
@id add_cluster_and_node
<record>
k8s_cluster ${K8S_CLUSTER_NAME}
k8s_node ${hostname}
</record>
</filter>
# =============================================================================
# SECTION 3: OUTPUTS
# =============================================================================
# Match directives are evaluated top to bottom. A record is consumed by the
# FIRST <match> whose tag pattern matches. Use @type copy to send to multiple
# destinations simultaneously.
# OUTPUT 3A: Print all logs to stdout — safe default for build validation
# and initial deployment. '**' matches every record regardless of tag.
<match **>
@type stdout
@id out_stdout
</match>
# OUTPUT 3B: Write kube.windows logs to a local file (currently inactive).
# This block is unreachable because 'match **' above consumes all records first.
# To activate file output alongside stdout, replace both <match> blocks with:
#
# <match **>
# @type copy
# <store>
# @type stdout
# </store>
# <store>
# @type file
# path C:\opt\fluent\log\kube-windows.log
# append true
# <buffer>
# @type memory
# flush_interval 5s
# </buffer>
# </store>
# </match>
<match kube.windows>
@type file
@id out_file_local
path C:\opt\fluent\log\kube-windows.log
append true
<buffer>
@type memory
flush_interval 5s
</buffer>
</match>
EOFConfiguration walkthrough:
| Section | Directive | What It Does |
|---|---|---|
<system> |
log_level info |
Controls Fluentd's own diagnostic log verbosity |
| Input 1A | @type tail |
Watches C:\var\log\containers\*.log — the path where containerd writes Kubernetes container logs on Windows nodes |
| Input 1A | pos_file |
Tracks read position per file so Fluentd resumes correctly after a service restart |
| Input 1A | read_from_head true |
On first boot, reads all existing log content; set to false if only new logs are needed |
| Input 1A | @type none (parse) |
No parsing — logs are forwarded as raw strings for maximum compatibility with Windows container runtimes |
| Input 1B | @type forward |
Listens on port 24224 to receive logs forwarded from other Fluent Bit or Fluentd agents |
| Filter 2A | record_transformer |
Adds k8s_cluster (from K8S_CLUSTER_NAME env var) and k8s_node (Windows hostname) to every log record |
| Output 3A | @type stdout |
Catch-all ** pattern writes all logs to stdout — safe default for build validation and initial deployment |
| Output 3B | @type file |
Currently unreachable — match ** above consumes all records first. To write to both stdout and a local file simultaneously, replace both <match> blocks with a single <match **> using @type copy (pattern shown in the config comments) |
Customizing for production: To forward logs to an external aggregator (Elasticsearch, Splunk, a remote Fluentd), add an additional
<match>block with the appropriate output plugin. The stdout and local file outputs are safe defaults for image validation.
The Fluentd task file is more comprehensive than the Notepad++ example. It handles directory preparation, MSI installation, config file deployment, config validation via dry-run, Windows service configuration, and cleanup — with fail-fast checks at each critical step.
cat << 'EOF' > $HOME/vks-image-builder/ansible-windows/tasks/fluentd.yml
# Create required working directories before the MSI installer runs.
- name: Ensure C:\Temp exists (installer staging directory)
ansible.windows.win_file:
path: C:\Temp
state: directory
- name: Ensure Fluent v6 working directories exist
ansible.windows.win_file:
path: "{{ item }}"
state: directory
loop:
- C:\opt\fluent
- C:\opt\fluent\buffer
- C:\opt\fluent\pos
- C:\opt\fluent\log
- C:\opt\fluent\etc\fluent
# Transfer the MSI from the build host into the Windows VM.
- name: Copy Fluent Package v6 MSI installer into the image
ansible.windows.win_copy:
src: files/custom/fluent-package-6.0.1-x64.msi
dest: C:\Temp\fluent-package-6.0.1-x64.msi
# Silent, unattended install. The 'creates' guard makes this idempotent —
# msiexec is skipped on re-runs if fluentd.bat already exists.
- name: Install Fluent Package v6 silently (unattended)
ansible.windows.win_shell: |
msiexec.exe /i C:\Temp\fluent-package-6.0.1-x64.msi /qn /norestart
args:
creates: 'C:\opt\fluent\bin\fluentd.bat'
register: fluent_pkg_install
# Verify fluentd.bat — the primary entry point for all Fluentd operations on Windows.
- name: Verify Fluent v6 CLI wrapper exists
ansible.windows.win_stat:
path: 'C:\opt\fluent\bin\fluentd.bat'
register: fluentd_wrapper
- name: Fail build if Fluent v6 did not install correctly
ansible.builtin.fail:
msg: |
Fluent Package v6 installation failed — fluentd.bat not found.
msiexec return code: {{ fluent_pkg_install.rc }}
stdout: {{ fluent_pkg_install.stdout | default('(empty)') }}
stderr: {{ fluent_pkg_install.stderr | default('(empty)') }}
when: not fluentd_wrapper.stat.exists
# Deploy the custom Fluentd configuration into the image.
- name: Backup existing fluentd.conf (if present)
ansible.windows.win_copy:
src: C:\opt\fluent\etc\fluent\fluentd.conf
dest: C:\opt\fluent\etc\fluent\fluentd.conf.bak
remote_src: yes
ignore_errors: true
- name: Copy Fluent v6 default fluentd.conf into the image
ansible.windows.win_copy:
src: files/custom/fluentd.conf
dest: C:\opt\fluent\etc\fluent\fluentd.conf
# Validate the config at build time — fails immediately on syntax errors
# or references to plugins that are not installed.
- name: Validate Fluentd configuration (v6 dry-run)
ansible.windows.win_shell: |
cmd.exe /c ""C:\opt\fluent\bin\fluentd.bat" -c "C:\opt\fluent\etc\fluent\fluentd.conf" --dry-run"
register: fluentd_validate
failed_when: fluentd_validate.rc != 0
# Verify the Windows service was registered by the MSI.
- name: Ensure Fluentd Windows service exists (fluentdwinsvc)
ansible.windows.win_service:
name: fluentdwinsvc
register: fluent_service
- name: Fail if Fluentd v6 service is missing
ansible.builtin.fail:
msg: "Fluentd Windows service (fluentdwinsvc) not found — MSI registration may have failed."
when: fluent_service.exists is not defined or not fluent_service.exists
# Configure the service for automatic startup and wait until running.
- name: Set Fluentd service to start automatically
ansible.windows.win_service:
name: fluentdwinsvc
start_mode: auto
- name: Restart Fluentd v6 service to apply config
ansible.windows.win_service:
name: fluentdwinsvc
state: restarted
- name: Wait until Fluentd v6 service is running
ansible.windows.win_service:
name: fluentdwinsvc
register: fluent_service_state
retries: 6
delay: 5
until: fluent_service_state.state == "running"
- name: Fail build if Fluentd service is not running
ansible.builtin.fail:
msg: "Fluentd v6 service did not reach running state within 30 seconds."
when: fluent_service_state.state != "running"
# Remove the MSI installer to keep the golden image clean.
- name: Remove Fluent Package MSI installer (cleanup)
ansible.windows.win_file:
path: C:\Temp\fluent-package-6.0.1-x64.msi
state: absent
EOFTask walkthrough by step:
| Step | Tasks | Purpose |
|---|---|---|
| 1 — Directories | win_file loop |
Creates C:\opt\fluent\{buffer,pos,log,etc\fluent} before the MSI runs, ensuring they exist even if the MSI creates them differently |
| 2 — Copy MSI | win_copy |
Transfers the MSI from the build host into C:\Temp\ inside the Windows VM |
| 3 — Install | win_shell + win_stat + fail |
Runs a silent install; verifies fluentd.bat exists; fails the build with the msiexec exit code if not |
| 4 — Config | win_copy (×2) |
Backs up any existing config then deploys the custom fluentd.conf |
| 5 — Validate | win_shell --dry-run |
Runs fluentd.bat --dry-run which parses the config file without starting the service; fails the build if the config has syntax errors |
| 6 — Service | win_service (×4) |
Verifies the service was registered by the MSI; sets startup to Automatic; restarts; polls until running with a 30-second total timeout |
| 7 — Cleanup | win_file |
Deletes the MSI from C:\Temp\ — not from the installed location |
Why validate at build time? The
--dry-runstep catches configuration errors before they are baked into the golden image. Without it, a config syntax error would only surface after deploying a cluster node — potentially hours later.
cat << 'EOF' >> $HOME/vks-image-builder/ansible-windows/tasks/main.yml
- import_tasks: fluentd.yml
EOFAfter adding both customizations, the ansible-windows directory should look like this:
$HOME/vks-image-builder/ansible-windows/
│
├── defaults/
│ └── main.yml
│
├── files/
│ ├── custom/
│ │ ├── fluentd.conf ← Fluentd configuration file
│ │ ├── fluent-package-6.0.1-x64.msi ← Fluent Package v6.0 installer
│ │ └── npp.8.9.1.Installer.x64.msi ← Notepad++ installer
│ └── scripts/
│ └── log-redirect.ps1
│
├── tasks/
│ ├── common.yml ← Built-in
│ ├── containerd.yml ← Built-in
│ ├── fluentd.yml ← Added: Fluentd customization
│ ├── main.yml ← Updated: new import_tasks entries
│ ├── notepadpp.yml ← Added: Notepad++ customization
│ ├── registry.yml ← Built-in
│ └── ssh-server.yml ← Optional: OpenSSH Server
│
└── templates/
├── containerd/config.toml
└── registry/config.yml
The tail of tasks/main.yml should contain:
# ... built-in tasks above ...
- import_tasks: notepadpp.yml
- import_tasks: fluentd.ymlRe-run the full build from Section 9. During the Ansible provisioning phase, the build logs will show each Fluentd task executing in sequence. The dry-run validation step is especially useful to watch — it will print the parsed configuration and confirm it is valid before the service is started.
After provisioning a VKS cluster with a Windows node pool that uses the new image, connect to a Windows node and run the following PowerShell checks to confirm each customization was applied successfully.
# Verify the binary exists
Test-Path "C:\Program Files\Notepad++\notepad++.exe"
# Expected: True# Verify the CLI wrapper exists
Test-Path "C:\opt\fluent\bin\fluentd.bat"
# Expected: True
# Check the service status — should be Running and StartType Automatic
Get-Service fluentdwinsvc | Select-Object Name, Status, StartType
# Expected:
# Name Status StartType
# ---- ------ ---------
# fluentdwinsvc Running Automatic
# Confirm the configuration file was deployed
Test-Path "C:\opt\fluent\etc\fluent\fluentd.conf"
# Expected: True
# Inspect the active configuration
Get-Content "C:\opt\fluent\etc\fluent\fluentd.conf"
# View the working directories
Get-Item "C:\opt\fluent\*" | Select-Object Name, Mode# Verify sshd binary
Test-Path "C:\Windows\System32\OpenSSH\sshd.exe"
# Expected: True
# Check sshd service
Get-Service sshd | Select-Object Name, Status, StartType