Skip to content

feat(data-platform): add metrics-server Helm deployment for EKS cluster#17006

Open
Gary Henderson (Gary-H9) wants to merge 5 commits into
mainfrom
feat/data-platform-add-metrics-server-helm
Open

feat(data-platform): add metrics-server Helm deployment for EKS cluster#17006
Gary Henderson (Gary-H9) wants to merge 5 commits into
mainfrom
feat/data-platform-add-metrics-server-helm

Conversation

@Gary-H9
Copy link
Copy Markdown
Contributor

@Gary-H9 Gary Henderson (Gary-H9) commented Jun 3, 2026

This PR relates to ministryofjustice/data-platform#317.

Summary

Why

MOJDP EKS clusters were missing metrics-server, which prevents HPA from receiving pod and node metrics via metrics.k8s.io.

This is addressed by introducing:

Notes

@Gary-H9 Gary Henderson (Gary-H9) requested review from a team as code owners June 3, 2026 15:01
@github-actions github-actions Bot added the environments-repository Used to exclude PRs from this repo in our Slack PR update label Jun 3, 2026
@Gary-H9 Gary Henderson (Gary-H9) had a problem deploying to data-platform-development-cluster June 3, 2026 15:04 — with GitHub Actions Error
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please assign metrics-server to the system node group and run it in high-availibility mode, at least 2 replicas

…lerations

Signed-off-by: Jacob Woffenden <jacob.woffenden@justice.gov.uk>
@jacobwoffenden Jacob Woffenden (jacobwoffenden) deployed to data-platform-test-cluster June 8, 2026 11:24 — with GitHub Actions Active
@jacobwoffenden Jacob Woffenden (jacobwoffenden) deployed to data-platform-development-cluster June 8, 2026 11:25 — with GitHub Actions Active
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 8, 2026


Checkov Scan Success

Show Output
*****************************

Checkov will check the following folders:
terraform/environments/data-platform/cluster/configuration/helm/metrics-server
terraform/environments/data-platform/cluster

*****************************

Running Checkov in terraform/environments/data-platform/cluster/configuration/helm/metrics-server

checkov_exitcode=0

*****************************

Running Checkov in terraform/environments/data-platform/cluster
terraform scan results:

Passed checks: 102, Failed checks: 0, Skipped checks: 0

kubernetes scan results:

Passed checks: 77, Failed checks: 13, Skipped checks: 0

Check: CKV_K8S_22: "Use read-only filesystem for containers where possible"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-21

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_37: "Minimize the admission of containers with capabilities assigned"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-34

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_40: "Containers should run as a high UID to avoid host conflict"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-37

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_8: "Liveness Probe Should be Configured"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-7

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_9: "Readiness Probe Should be Configured"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-8

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_38: "Ensure that Service Account Tokens are only mounted where necessary"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-35

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_31: "Ensure that the seccomp profile is set to docker/default or runtime/default"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-29

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_23: "Minimize the admission of root containers"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-22

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_20: "Containers should not run with allowPrivilegeEscalation"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-19

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_29: "Apply security context to your pods and containers"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/ensure-securitycontext-is-applied-to-pods-and-containers

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_30: "Apply security context to your containers"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-28

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV_K8S_28: "Minimize the admission of containers with the NET_RAW capability"
	FAILED for resource: Deployment.whoami.whoami
	File: /src/kubernetes/e2e.yaml:9-45
	Guide: https://docs.prismacloud.io/en/enterprise-edition/policy-reference/kubernetes-policies/kubernetes-policy-index/bc-k8s-27

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---

Check: CKV2_K8S_6: "Minimize the admission of pods which lack an associated NetworkPolicy"
	FAILED for resource: Pod.whoami.whoami.app-whoami
	File: /src/kubernetes/e2e.yaml:9-45

		9  | apiVersion: apps/v1
		10 | kind: Deployment
		11 | metadata:
		12 |   name: whoami
		13 |   namespace: whoami
		14 |   labels:
		15 |     app: whoami
		16 | spec:
		17 |   replicas: 3
		18 |   selector:
		19 |     matchLabels:
		20 |       app: whoami
		21 |   template:
		22 |     metadata:
		23 |       labels:
		24 |         app: whoami
		25 |     spec:
		26 |       nodeSelector:
		27 |         compute.data-platform.service.justice.gov.uk/node-pool: general-spot-arm64
		28 |       tolerations:
		29 |         - key: "compute.data-platform.service.justice.gov.uk/node-pool"
		30 |           operator: "Equal"
		31 |           value: "general-spot-arm64"
		32 |           effect: "NoSchedule"
		33 |       containers:
		34 |         - name: whoami
		35 |           image: docker.io/traefik/whoami:v1.11@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab
		36 |           ports:
		37 |             - containerPort: 80
		38 |           resources:
		39 |             limits:
		40 |               cpu: "500m"
		41 |               memory: "256Mi"
		42 |             requests:
		43 |               cpu: "250m"
		44 |               memory: "128Mi"
		45 | ---


checkov_exitcode=1

TFLint Scan Success

Show Output
*****************************

Using default config
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.9.1)
tflint checking:
terraform/environments/data-platform/cluster/configuration/helm/metrics-server
terraform/environments/data-platform/cluster

*****************************

Running tflint in terraform/environments/data-platform/cluster/configuration/helm/metrics-server
tflint_exitcode=0

*****************************

Running tflint in terraform/environments/data-platform/cluster
tflint_exitcode=0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

environments-repository Used to exclude PRs from this repo in our Slack PR update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants