Skip to content

Promote topology-aware deployment hooks to a common location #2166

Description

@sanchezl

Problem

Several topology-aware deployment hooks live in the CSI-specific package pkg/operator/csi/csidrivercontrollerservicecontroller/helpers.go:

  • WithControlPlaneTopologyHook — clears nodeSelector when ControlPlaneTopology == External
  • WithReplicasHook — sets replicas to 2 for HA, 1 otherwise
  • WithObservedProxyDeploymentHook — injects observed proxy config into containers
  • WithCABundleDeploymentHook — projects custom CA bundle ConfigMap into containers
  • WithConfigMapHashAnnotationHook / WithSecretHashAnnotationHook — annotates deployment with hashes to trigger rollouts

These are general-purpose patterns needed by any operator using deploymentcontroller.NewDeploymentController, not just CSI drivers. Because they live in the CSI package, non-CSI operators either import a CSI-specific path or reimplement the logic themselves.

There is also a separate WithReplicasHook in pkg/operator/deploymentcontroller/helpers.go that counts nodes matching the deployment's own nodeSelector — a different approach with the same name.

Ecosystem survey

A survey of DeploymentHookFunc usage across OpenShift operator repos shows widespread reimplementation of these patterns:

Topology / nodeSelector (3+ custom implementations)

Repo Hook What it does
console-operator withNodeSelector() Surgically deletes master key for External topology
service-ca-operator shouldScheduleOnWorkers() Flag-based shift to worker nodes for External
cluster-kube-storage-version-migrator-operator setControlPlaneNodeSelector() Adds control-plane nodeSelector for non-External (PR #157)
csi-operator withHyperShiftNodeSelector() Reads nodeSelector from HostedControlPlane CR

Replicas (6+ custom implementations)

Repo Hook What it does
console-operator withReplicas() 1 for SNO, 2 for HA, handles Arbiter mode
cluster-ingress-operator DetermineReplicas() Chooses based on DefaultPlacement + topology
cluster-cloud-controller-manager-operator IsSingleReplica Config flag from topology
operator-framework-olm getReplicas() 1 for non-HA, 2 for HA
csi-operator withHyperShiftReplicas() Hardcoded replicas for HyperShift
cluster-kube-storage-version-migrator-operator setDesiredReplicas() Counts control-plane nodes, defaults to 2 for External

Proxy injection (5+ custom implementations)

Repo Hook What it does
cluster-olm-operator UpdateDeploymentProxyHook() Reads cluster proxy config, injects env vars
cluster-storage-operator withProxyHook() Custom proxy injection from observedConfig
cert-manager-operator withProxyEnv Proxy env vars for cert-manager containers
csi-operator withClusterWideProxy() Wraps library-go's version
All CSI driver operators Use library-go's CSI-scoped version directly

CA bundle (3+ custom implementations)

Repo Hook What it does
cert-manager-operator withCAConfigMap() Projects trusted CA ConfigMap into containers
aws-ebs-csi-driver-operator withCustomAWSCABundle() AWS-specific CA bundle
All CSI driver operators Use library-go's CSI-scoped version directly

Anti-affinity (2+ custom implementations)

Repo Hook What it does
console-operator withAffinity() Pod anti-affinity for HA, clear for SNO
operator-framework-olm getAntiAffinityConfig() Same pattern

HyperShift-specific hooks (reimplemented per operator)

Repo Hooks
csi-snapshot-controller-operator hyperShiftNodeSelectorHook, hyperShiftControlPlaneIsolationHook, hyperShiftColocationHook, hyperShiftLabelsHook, hyperShiftReplaceNamespaceHook, hyperShiftSetSecurityContext
csi-operator withHyperShiftNodeSelector, withHyperShiftCustomTolerations, withHyperShiftLabels, withHyperShiftControlPlaneImages, withHyperShiftRunAsUser, WithTokenMinter
aws-ebs-csi-driver-operator withHypershiftDeploymentHook, withHypershiftReplicasHook, withNamespaceDeploymentHook

Proposal

Promote the most commonly reimplemented hooks from pkg/operator/csi/csidrivercontrollerservicecontroller/ to pkg/operator/deploymentcontroller/ so they're discoverable and importable without pulling in CSI dependencies:

  1. WithControlPlaneTopologyHook — clear nodeSelector for External topology
  2. WithObservedProxyDeploymentHook — inject proxy config
  3. WithCABundleDeploymentHook — project CA bundle ConfigMap
  4. WithConfigMapHashAnnotationHook / WithSecretHashAnnotationHook — rollout trigger annotations
  5. Topology-aware WithReplicasHook — resolve the naming collision with the existing node-counting variant in deploymentcontroller/helpers.go

Additionally, consider adding hooks for common patterns that don't yet exist in library-go:

  • WithControlPlaneNodeSelectorHook — add control-plane nodeSelector for non-External topology (the inverse of WithControlPlaneTopologyHook)
  • WithTopologyAwareAntiAffinityHook — set pod anti-affinity for HA, clear for SNO

Context

This came up during review of openshift/cluster-kube-storage-version-migrator-operator#157.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions