Skip to content

8391: Set or Propagate Labels#83

Open
m-ildefons wants to merge 3 commits into
harvester:masterfrom
m-ildefons:gh-8391
Open

8391: Set or Propagate Labels#83
m-ildefons wants to merge 3 commits into
harvester:masterfrom
m-ildefons:gh-8391

Conversation

@m-ildefons

Copy link
Copy Markdown
Member
  • Set labels automatically, based on cluster name, machine set, and pool name
  • Propagate user-defined labels to VM and VMI CRDs

related-to: harvester/harvester#8391

Problem:

As described in #8391, there is a need to be able to define custom labels for VMs created with the Harvester node driver.
One use case for such labels is that a user, who creates a downstream cluster in Rancher may want to label some of the nodes of that downstream cluster such that they can later be selected by a load balancer as backends.

Solution:

Add a new flag to the Harvester node driver, which expects a string containing the labels and their values.
The format of the string should be that each label and value is separated by = and each label-value-pair is separated by ,.
Thus labels may be specified as follows:

foo=bar,baz=bla,xxx=yyy

will result in the labels:

metadata:
  labels:
    foo: bar
    baz: bla
    xxx: yyy

These labels are applied both to the VM CRD and the VMI CRD.

Related Issue(s):

related-to: harvester/harvester#8391
SURE-9641

Test plan:

TBD

Additional documentation or context

- Set labels automatically, based on cluster name, machine set, and pool
  name
- Propagate user-defined labels to VM and VMI CRDs

related-to: harvester/harvester#8391

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Add module replacement for glog back to ensure fix for CVE-2024-45339 is
included.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons m-ildefons marked this pull request as ready for review October 8, 2025 12:02

@Vicente-Cheng Vicente-Cheng left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, LGTM. I'm just curious about the scenario where the labels did not propagate.

Additionally, I suggest separating the refactor commit from the feature enhancement commit.
Most changes are related to the refactor.

That would help the reviewer to get an understanding efficiently.
Thanks!

Comment thread Dockerfile.dapper

WORKDIR ${DAPPER_SOURCE}

# Make this a separate step, so it doesn't have to be done every time a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this?
We can set up everything before executing the make test.

Comment thread go.mod
module github.com/harvester/docker-machine-driver-harvester

go 1.24.7
go 1.25.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/1.25.0/1.25.2

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using

go 1.25

toolchain go1.25.2

Comment thread harvester/create.go
vm.Spec.Template.Spec.Domain.CPU.DedicatedCPUPlacement = d.CPUPinning
vm.Spec.Template.Spec.Domain.CPU.IsolateEmulatorThread = d.IsolateEmulatorThread

vm.Kind = kubevirtv1.VirtualMachineGroupVersionKind.Kind

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why do we move these two lines?

Comment thread harvester/create.go
builder.Labels(labels.Set{
machineSetNameLabelKey: machineSetName,
})
builder.VirtualMachineInstanceTemplateLabels(labels.Set{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious about this. I check the current guest cluster VM.
Looks like the VM label (metadata/labels) will propagate to the VMI/virt-launcher pod.

Just curious what case we need to add the same label to the spec/template/metadata/labels?
Which means the VM label did not propagate.

@w13915984028

Copy link
Copy Markdown
Member

The VM and VMI object are a bit complex:

There are labels, annotations on VM object specifically; and also some dedicated on VMI objects; those under vm.spec.template.metadata are propagated to vmi.

The reserved-memory (to eliminate OOM) is on vm.metadata.annotation, is needed for Rancher UI and terraform.

Back to general, we might require four kinds of general customization to meet different requirements:

vm.metadata.label, vm.metadata.annotation, vm.spec.template.metadata.label, vm.spec.template.metadata.annotation, the latter two are for vmi.

vm object:

- apiVersion: kubevirt.io/v1
  kind: VirtualMachine
  metadata:
    annotations:
      harvesterhci.io/mac-address: '{"nic-1":"9a:32:1e:e1:83:da"}' 
      harvesterhci.io/reservedMemory: 256Mi   // specific on vm
      harvesterhci.io/vmRunStrategy: RerunOnFailure
      harvesterhci.io/volumeClaimTemplates: '[{"metadata":{"name":"vm1-disk-0-lebdg","annotations":{"harvesterhci.io/imageId":"default/image-666qx"}},"spec":{"accessModes":["ReadWriteMany"],"resources":{"requests":{"storage":"10Gi"}},"volumeMode":"Block","storageClassName":"longhorn-image-666qx"}}]'
      kubevirt.io/latest-observed-api-version: v1
      kubevirt.io/storage-observed-api-version: v1
      network.harvesterhci.io/ips: '[]'
    creationTimestamp: "2025-11-17T14:19:01Z"
    finalizers:
    - kubevirt.io/virtualMachineControllerFinalize
    - wrangler.cattle.io/VMController.CleanupPVCAndSnapshot
    generation: 14
    labels:
      harvesterhci.io/creator: harvester
      harvesterhci.io/os: linux
      test.io/test: test
    name: vm1

...
  spec:
    runStrategy: RerunOnFailure
    template:
      metadata:
        annotations:
          harvesterhci.io/dynamic-ssh-key-names: '{}'
          harvesterhci.io/dynamic-ssh-key-users: '["rancher","wj"]'
          harvesterhci.io/sshNames: '[]'
          test.io/test: test-on-anno
        creationTimestamp: null
        labels:
          harvesterhci.io/vmName: vm1  // dedicated on vmi
          test.io/test: vmlabel
      spec:

vmi object:

- apiVersion: kubevirt.io/v1
  kind: VirtualMachineInstance
  metadata:
    annotations:
      harvesterhci.io/dynamic-ssh-key-names: '{}'
      harvesterhci.io/dynamic-ssh-key-users: '["rancher","wj"]'
      harvesterhci.io/sshNames: '[]'
      kubevirt.io/latest-observed-api-version: v1
      kubevirt.io/storage-observed-api-version: v1
      kubevirt.io/vm-generation: "14"
      test.io/test: test-on-anno
    creationTimestamp: "2025-11-18T09:17:49Z"
    finalizers:
    - kubevirt.io/virtualMachineControllerFinalize
    - foregroundDeleteVirtualMachine
    - wrangler.cattle.io/VMController.BackfillObservedNetworkMacAddress
    - wrangler.cattle.io/harvester-lb-vmi-controller
    generation: 11
    labels:
      harvesterhci.io/vmName: vm1
      kubevirt.io/nodeName: harv41
      test.io/test: vmlabel
    name: vm1
    namespace: default

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants