Skip to content

Add integration tests for inplace machine upgrade - #1144

Open
aaronfern wants to merge 1 commit into
gardener:masterfrom
aaronfern:inplace_it
Open

aaronfern wants to merge 1 commit into
gardener:masterfrom
aaronfern:inplace_it

Conversation

@aaronfern

Copy link
Copy Markdown
Member

What this PR does / why we need it:
This PR adds integration tests for machine inplace upgrade

The following scenarios have been added

  • Happy path - move machine to new mcs after successful upgrade
  • Machine transition to InPlaceUpdateFailed phase when node update fails and update-result=failed annotation is set
  • Machine transitions to InPlaceUpdateFailed when update result is not set within timeout
  • Manual inPlace upgrade

Which issue(s) this PR fixes:
Fixes partially #1123

Special notes for your reviewer:

Release note:

Added integration tests for inPlace machine upgrade

@aaronfern
aaronfern requested a review from a team as a code owner August 31, 2026 11:41
@gardener-prow

gardener-prow Bot commented Aug 31, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kon-angelo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gardener-prow gardener-prow Bot added do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 31, 2026
@gardener-prow gardener-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 4, 2026
return mcd
}

// NewMachineDeploymentWithName returns a MachineDeployment with a custom name and a unique label derived from that name,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The canonical MCM way of identifying MCD for a machine with the name label can be used here rather than creating a different key.

)

var (
testLabels = map[string]string{"test-label": "test-label"}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the use-case of adding this?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use case would be to be able to list machines created be tests easily without having to filter out machines that were already present or created by some other cluster operation

}

// InPlaceMcdLabels returns the unique label map used for a given inplace test MCD name
func InPlaceMcdLabels(mcdName string) map[string]string {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the only purpose of this is to identify machines belonging to a specific MCD, consider using the name label key.

Comment thread pkg/test/integration/common/helpers/machine_resources.go
Signed-off-by: aaronfern <aaron.francis.fernandes@sap.com>
@gardener-prow gardener-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 16, 2026

@takoverflow takoverflow left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the tests!

Just a general question, is there a reason why a lot of Get calls' errors are being ignored?

I've only explicitly added comments for a few but was wondering since it happened across tests.

originalOwnerRefMcsName := ""
for _, b := range runningMachines[0].ObjectMeta.OwnerReferences {
if b.Kind == "MachineSet" {
originalOwnerRefMcsName = b.Name

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can break out of the loop once we've found the MCS ownerRef.


ginkgo.By("Trigger inPlace rollout by updating MachineClass")
retryErr := retry.RetryOnConflict(retry.DefaultRetry, func() error {
md, _ := c.ControlCluster.McmClient.MachineV1alpha1().MachineDeployments(controlClusterNamespace).Get(ctx, helpers.InPlaceMcdNameHappyPath, metav1.GetOptions{})

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this Get error be checked for?


ginkgo.By("Add node.machine.sapcloud.io/update-result:successful label to node")
retryErr = retry.RetryOnConflict(retry.DefaultRetry, func() error {
node, _ := c.TargetCluster.Clientset.CoreV1().Nodes().Get(ctx, runningMachines[0].ObjectMeta.Labels[v1alpha1.NodeLabelKey], metav1.GetOptions{})

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Get call's error is not being handled

c.pollingInterval).Should(gomega.BeTrue())

ginkgo.By("Verify that old mcs has been scaled to zero")
// Old mcs is not removed because mcd.Spec.RevisionHistoryLimit is set to it's default value nil.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Old mcs is not removed because mcd.Spec.RevisionHistoryLimit is set to it's default value nil.
// Old mcs is not removed because mcd.Spec.RevisionHistoryLimit is set to its default value nil.


// Testcase #04 | Orphaned Resources
// Testcase #04 | InPlace Updates
ginkgo.Describe("InPlace machine update", ginkgo.Ordered, func() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can these tests not be run in parallel? Asking since they're using independent machines belonging to different MCDs.

@gardener-prow gardener-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 23, 2026
@gardener-prow

gardener-prow Bot commented Sep 23, 2026

Copy link
Copy Markdown

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants