Skip to content

WIP: Add dual-stack (IPv6) support to kubetest2-ec2 deployer#530

Closed
nrb wants to merge 6 commits into
kubernetes-sigs:mainfrom
nrb:dual-stack-support
Closed

WIP: Add dual-stack (IPv6) support to kubetest2-ec2 deployer#530
nrb wants to merge 6 commits into
kubernetes-sigs:mainfrom
nrb:dual-stack-support

Conversation

@nrb

@nrb nrb commented Mar 5, 2026

Copy link
Copy Markdown

Extends the kubetest2-ec2 deployer with a new --ip-family flag (ipv4/ipv6/dual). When set to dual or ipv6, the deployer enables IPv6 on the default VPC at runtime: associates an Amazon-provided /56 CIDR with the VPC, carves a deterministic /64 per subnet (FNV hash of the IPv4 CIDR), enables IPv6 auto-assignment, and adds a ::/0 route via the IGW. Instances receive an IPv6 address on the primary ENI. The kubelet node-ip is set to the comma-joined IPv4,IPv6 pair via run-kubeadm.sh. Kubeadm is configured with dual-stack service and pod CIDRs when --ip-family=dual. IPv6 subnet CIDRs are disassociated on Down().

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 5, 2026
@ronaldngounou

ronaldngounou commented Mar 7, 2026

Copy link
Copy Markdown
Member

/skip

@ronaldngounou

ronaldngounou commented Mar 7, 2026

Copy link
Copy Markdown
Member

/test pull-kubernetes-e2e-ec2-canary

@damdo

damdo commented Mar 10, 2026

Copy link
Copy Markdown
Member

Hey @ronaldngounou it looks like tests are green, are we happy to move forward with this?
Thanks!

@damdo

damdo commented Mar 10, 2026

Copy link
Copy Markdown
Member

/assign @dims @wongma7 @upodroid

@damdo

damdo commented Mar 10, 2026

Copy link
Copy Markdown
Member

/assign @justinsb

@damdo

damdo commented Mar 10, 2026

Copy link
Copy Markdown
Member

ATM this is blocking testing of an AWS CCM dual-stack feature kubernetes/cloud-provider-aws#1313

@upodroid

upodroid commented Mar 10, 2026

Copy link
Copy Markdown
Member

Can we add an ipv6 variant of the quick job?

@nrb

nrb commented Mar 10, 2026

Copy link
Copy Markdown
Author

@upodroid Yep, will do.

@nrb nrb mentioned this pull request Mar 10, 2026
Merged
@nrb

nrb commented Mar 10, 2026

Copy link
Copy Markdown
Author

Added kubernetes/test-infra#36625

@damdo

damdo commented Mar 10, 2026

Copy link
Copy Markdown
Member

/test pull-kubernetes-e2e-ec2-cloud-provider-dual-stack-quick

@nrb

nrb commented Mar 10, 2026

Copy link
Copy Markdown
Author

Have been debugging the failed job locally - looks like there's a race when getting the CIDR associated with the VPC/subnet. Running the kubetest2 ec2 command once will hit the error (E0310 19:22:13.774032 3496 up.go:132] error starting instance for image ami-084568db4383264d4 : ensuring IPv6 on subnet subnet-04a35f639b4cef340: derive /64 for subnet subnet-04a35f639b4cef340: parse VPC IPv6 CIDR : invalid CIDR address:), then running it again will succeed, and I can see the relevant resources in the AWS console.

Will also improve the logging when I'm in there.

@wongma7

wongma7 commented Mar 11, 2026

Copy link
Copy Markdown

/approve

@k8s-ci-robot

k8s-ci-robot commented Mar 11, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nrb, wongma7

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 11, 2026
@nrb nrb mentioned this pull request Mar 11, 2026
Open
@linux-foundation-easycla

linux-foundation-easycla Bot commented Mar 11, 2026 •
edited
Loading

Copy link
Copy Markdown

CLA Signed

The committers listed above are authorized under a signed CLA.

@k8s-ci-robot k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. and removed cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 11, 2026
@dims dims removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 11, 2026
@nrb

nrb commented Mar 11, 2026

Copy link
Copy Markdown
Author

Hm, I don't think claude can accept the CLA...

@nrb nrb changed the title Add dual-stack (IPv6) support to kubetest2-ec2 deployer WIP: Add dual-stack (IPv6) support to kubetest2-ec2 deployer Mar 11, 2026
@damdo

damdo commented Mar 12, 2026

Copy link
Copy Markdown
Member

/test pull-kubernetes-e2e-ec2-cloud-provider-dual-stack-quick

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 12, 2026
@nrb nrb mentioned this pull request Mar 12, 2026
Open
@dims dims added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. and removed approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Mar 12, 2026
@nrb
nrb force-pushed the dual-stack-support branch from 6762096 to e83260b Compare March 12, 2026 21:50
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 12, 2026
@nrb
nrb force-pushed the dual-stack-support branch 2 times, most recently from 3457d6e to a51a1d3 Compare March 13, 2026 17:28
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 13, 2026
@nrb
nrb force-pushed the dual-stack-support branch from 83c4dd1 to 5d8dd7a Compare March 13, 2026 20:21
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 13, 2026
@nrb

nrb commented Mar 13, 2026

Copy link
Copy Markdown
Author

/test pull-kubernetes-e2e-ec2-cloud-provider-dual-stack-quick

@nrb
nrb force-pushed the dual-stack-support branch 4 times, most recently from 86e4354 to 444a144 Compare March 16, 2026 20:17
@nrb

nrb commented Mar 17, 2026

Copy link
Copy Markdown
Author

/test pull-kubernetes-node-e2e-containerd-serial-ec2-canary

1 similar comment
@damdo

damdo commented Mar 18, 2026

Copy link
Copy Markdown
Member

/test pull-kubernetes-node-e2e-containerd-serial-ec2-canary

nrb added 4 commits March 18, 2026 15:54
@nrb
Add dual-stack (IPv6) support to kubetest2-ec2 deployer
db7bb70 
Extends the kubetest2-ec2 deployer with a new --ip-family flag (ipv4/ipv6/dual).
Update teardown to wait for disassociation actions.
Update log messages for clarity.

Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>
Assisted-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@nrb
drop: Use a dual-stack CCM daemonset for dual stack job
2fef27c 
Temporary commit to demonstrate that the proposed template works.

Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>
@nrb
Compress all userdata files
8716a2b 
The userdata file were already close to the 16kb limit, and adding IPv6
support made it worse.

Compress the entire string, while also storing the uncompressed form in
the artifacts directory.

Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>
Assisted-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@nrb
Use IPv6 CIDRs and addresses for cluster bootstrap
1bd82ca 
Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>
Assisted-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@nrb
nrb force-pushed the dual-stack-support branch from 444a144 to 1bd82ca Compare March 18, 2026 20:02
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Mar 18, 2026
nrb added 2 commits March 18, 2026 17:23
@nrb
Pass IPFAMILY down to the instance
3628304 
Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>
@nrb
Don't auto-assign IPv6 addresses on subnet
77e5786 
Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>
@k8s-ci-robot

k8s-ci-robot commented Mar 19, 2026

Copy link
Copy Markdown
Contributor

@nrb: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubernetes-node-e2e-containerd-serial-ec2-canary 77e5786 link false /test pull-kubernetes-node-e2e-containerd-serial-ec2-canary

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@dims dims closed this May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nckturner nckturner Awaiting requested review from nckturner
@wongma7 wongma7 Awaiting requested review from wongma7

Assignees

@dims dims

@justinsb justinsb

@wongma7 wongma7

@upodroid upodroid

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@nrb @ronaldngounou @damdo @upodroid @wongma7 @k8s-ci-robot @dims @justinsb