Skip to content

Adjust effective-creation-timeout smartly per MachineDeployment - #1156

Open
elankath wants to merge 7 commits into
gardener:masterfrom
elankath:adjust-creation-timeout
Open

elankath wants to merge 7 commits into
gardener:masterfrom
elankath:adjust-creation-timeout

Conversation

@elankath

Copy link
Copy Markdown
Member

What this PR does / why we need it:

Which issue(s) this PR fixes:
Fixes #1155

Special notes for your reviewer:

Release note:

Adjust MachineDeployment's effective-creation-timeout on Machine Join Failures

@elankath elankath self-assigned this Sep 16, 2026
@elankath
elankath requested a review from a team as a code owner September 16, 2026 22:04
@gardener-prow

gardener-prow Bot commented Sep 16, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from elankath. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gardener-prow gardener-prow Bot added do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 16, 2026
@gardener-prow gardener-prow Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 16, 2026
@elankath
elankath force-pushed the adjust-creation-timeout branch from 65fea5a to e511cc3 Compare September 17, 2026 07:36
@gardener-prow gardener-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 21, 2026

@aaronfern aaronfern left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feedback after first round of review

Comment thread pkg/controller/deployment.go
return mcdDeepCopy, nil
}

func (dc *controller) checkAndAdjustMachineReplaceCycleCountAndEffectiveCreationTimeout(ctx context.Context, mcd *v1alpha1.MachineDeployment, machineMap map[types.UID]*v1alpha1.MachineList) (adjusted bool, err error) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a docstring explaining this function? it's quite complex and a quick explanation will help in the long term

Signed-off-by: elankath <tarun.ramakrishna.elankath@sap.com>
…tiveCreationTimeout

Signed-off-by: elankath <tarun.ramakrishna.elankath@sap.com>
…laceCycleCountThreshold

Signed-off-by: elankath <tarun.ramakrishna.elankath@sap.com>
Signed-off-by: elankath <tarun.ramakrishna.elankath@sap.com>
Signed-off-by: elankath <tarun.ramakrishna.elankath@sap.com>
Signed-off-by: elankath <tarun.ramakrishna.elankath@sap.com>
@elankath
elankath force-pushed the adjust-creation-timeout branch from b6dd98a to 0ba7155 Compare September 24, 2026 08:14
@gardener-prow gardener-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 24, 2026
@elankath

Copy link
Copy Markdown
Member Author

Activity: Rebased PR after upstream changes

return dc.rollback(ctx, d, machineSets, machineMap)
}

if adjusted, err := dc.checkAndAdjustMachineReplaceCycleCountAndEffectiveCreationTimeout(ctx, d, machineMap); adjusted || err != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering if this could be put in defer so that this is the very last operation done on the MCD.

@thiyyakat thiyyakat left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR. I've given a few comments after a preliminary review of the changes. I still need to review the mother lode – checkAndAdjustMachineReplaceCycleCountAndEffectiveCreationTimeout 😅 . PTAL.

Comment thread pkg/util/provider/machineutils/utils.go Outdated
Comment on lines +162 to +163
// IsMachineFailed checks if machine has failed
func IsMachineFailed(p *v1alpha1.Machine) bool {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The functions in utils.go no longer have Machine in their names. Please undo this change.

Suggested change
// IsMachineFailed checks if machine has failed
func IsMachineFailed(p *v1alpha1.Machine) bool {
// IsFailed checks if machine has failed
func IsFailed(p *v1alpha1.Machine) bool {

Comment on lines 99 to +103
metaObject, err := meta.Accessor(object)
if err != nil {
return nil, err
return
}
effectiveMachineCreationTimeoutStr, ok := metaObject.GetAnnotations()[v1alpha1.AnnotationKeyMachineEffectiveCreationTimeout]
durationStr, ok := metaObject.GetAnnotations()[v1alpha1.AnnotationKeyMachineEffectiveCreationTimeout]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not have a generic helper function to fetch annotation values based on the annotation key? It can be used in the Get* functions that you have introduced here. The parsing can be done in the caller.


// getMachineMachineSets returns the MachineSets matching the given Machine.
func (c *controller) getMachineMachineSets(machine *v1alpha1.Machine) ([]*v1alpha1.MachineSet, error) {
func (dc *controller) getMachineMachineSets(machine *v1alpha1.Machine) ([]*v1alpha1.MachineSet, error) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is the machineset controller, is the receiver name change from c to dc accurate/required?

If this was not intentional, please make the change elsewhere in this file as well.

}

func (c *controller) Run(workers int, stopCh <-chan struct{}) {
func (dc *controller) Run(workers int, stopCh <-chan struct{}) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason behind this change? Since the Run method handles all types of queues, not just for machine deployment?

Maybe I'm not expanding dc correctly in my head :) Do you not mean deployment controller?

@elankath elankath Sep 24, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was dc in more places and c in other places. Got lint error on inconsistent receiver name. Just changed to dc. I guess I should just change to c.

}

// FilterNodeConditionOfType filters conditions and returns the NodeCondition belonging to the given conditionType or nil if not present.
func FilterNodeConditionOfType(conditions []v1.NodeCondition, conditionType v1.NodeConditionType) *v1.NodeCondition {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm not wrong, GetCondition in this file does the same thing. If you don't want to pass the node object, then please modify GetCondition to accept only a list of conditions instead, or get rid of GetCondition and use only the function introduced here. I don't think there's a need for both functions.

Co-authored-by: thiyyakat <meghana.thiyyakat@sap.com>
Signed-off-by: Tarun Ramakrishna Elankath <tarun.ramakrishna.elankath@sap.com>
}
}
if adjustedAnnotations := createAdjustedAnnotations(oldInfo, newInfo); len(adjustedAnnotations) > 0 {
newMcd := mcd.DeepCopy()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can skip the deepcopy here as we have done it on L505

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Monitor Machine Join Failures and Adjust MachineDeployment effective-creation-timeout

4 participants