From c4a44ef6e0f04b37751b544364ee2c89cf4e9de9 Mon Sep 17 00:00:00 2001 From: Thomas Maas Date: Thu, 19 Mar 2026 12:30:23 +0100 Subject: [PATCH] Add cross-repo capability specs for AI-assisted development Adds specification documents that capture how Kuadrant capabilities span multiple repositories, the contracts between components, and shared conventions. These specs enable AI coding assistants to reason about cross-repo changes and serve as human-readable architecture docs. Files added: - docs/specs/repo-map.md: inter-repo dependencies and contracts - docs/specs/capabilities/gateway-policy.md: policy-machinery framework - docs/specs/capabilities/rate-limiting.md: rate limiting (4 repos) - docs/specs/capabilities/auth.md: auth/authz (4 repos) - docs/specs/capabilities/dns.md: DNS management (2 repos) - docs/specs/capabilities/tls.md: TLS certificates - docs/specs/capabilities/cross-cutting.md: shared conventions Co-Authored-By: Claude Opus 4.6 (1M context) --- docs/specs/README.md | 27 ++ docs/specs/capabilities/auth.md | 373 ++++++++++++++++++ docs/specs/capabilities/cross-cutting.md | 387 ++++++++++++++++++ docs/specs/capabilities/dns.md | 349 +++++++++++++++++ docs/specs/capabilities/gateway-policy.md | 457 ++++++++++++++++++++++ docs/specs/capabilities/rate-limiting.md | 286 ++++++++++++++ docs/specs/capabilities/tls.md | 193 +++++++++ docs/specs/repo-map.md | 270 +++++++++++++ 8 files changed, 2342 insertions(+) create mode 100644 docs/specs/README.md create mode 100644 docs/specs/capabilities/auth.md create mode 100644 docs/specs/capabilities/cross-cutting.md create mode 100644 docs/specs/capabilities/dns.md create mode 100644 docs/specs/capabilities/gateway-policy.md create mode 100644 docs/specs/capabilities/rate-limiting.md create mode 100644 docs/specs/capabilities/tls.md create mode 100644 docs/specs/repo-map.md diff --git a/docs/specs/README.md b/docs/specs/README.md new file mode 100644 index 0000000..f87ea39 --- /dev/null +++ b/docs/specs/README.md @@ -0,0 +1,27 @@ +# Kuadrant Capability Specs + +Cross-repository specification documents for Kuadrant. These specs capture how capabilities span multiple repositories, the contracts between components, and shared conventions. + +Designed for use with AI coding assistants (Spec-Driven Development) but equally useful as human-readable architecture documentation. + +## Files + +| File | Description | +|------|-------------| +| [repo-map.md](repo-map.md) | Inter-repo dependencies, CRD ownership, and cross-repo contracts | +| [capabilities/gateway-policy.md](capabilities/gateway-policy.md) | Policy-machinery framework: topology DAG, policy attachment, controller system | +| [capabilities/rate-limiting.md](capabilities/rate-limiting.md) | Rate limiting end-to-end across kuadrant-operator, limitador, limitador-operator, wasm-shim | +| [capabilities/auth.md](capabilities/auth.md) | Auth/authz end-to-end across kuadrant-operator, authorino, authorino-operator, wasm-shim | +| [capabilities/dns.md](capabilities/dns.md) | DNS management across kuadrant-operator and dns-operator, including multi-cluster delegation | +| [capabilities/tls.md](capabilities/tls.md) | TLS certificate management via kuadrant-operator and cert-manager | +| [capabilities/cross-cutting.md](capabilities/cross-cutting.md) | Shared conventions: labels, conditions, errors, images, toolchain, testing | + +## How to use with AI agents + +Point your AI coding assistant at the relevant spec file(s) when working on cross-repo changes. The specs provide: + +- **Data flow diagrams** showing how a user action propagates through components +- **CRD spec structures** with realistic YAML examples +- **Cross-repo contracts** specifying the exact data format at each integration boundary +- **Critical invariants** that must stay in sync across repos +- **Key source file links** for navigating directly to implementation diff --git a/docs/specs/capabilities/auth.md b/docs/specs/capabilities/auth.md new file mode 100644 index 0000000..55bf4f2 --- /dev/null +++ b/docs/specs/capabilities/auth.md @@ -0,0 +1,373 @@ +# Auth + +## Components + +| Component | Repo | Language | Role | +|-----------|------|----------|------| +| AuthPolicy CRD | kuadrant-operator | Go | User-facing policy (v1) attached to Gateway/HTTPRoute | +| Authorino | authorino | Go | Authorization service implementing Envoy ext_authz gRPC | +| Authorino Operator | authorino-operator | Go | Deploys/configures Authorino instances | +| wasm-shim | wasm-shim | Rust | Envoy WASM filter that routes auth requests to Authorino | + +## Data Flow + +``` +User creates AuthPolicy (kuadrant.io/v1) + | + v +kuadrant-operator reconciles + | + +---> EffectiveAuthPolicyReconciler + | merges policies across Gateway/HTTPRoute hierarchy + | + +---> AuthConfigsReconciler + | translates effective AuthPolicy -> AuthConfig CR + | in Authorino's namespace + | + +---> IstioAuthClusterReconciler / EnvoyGatewayAuthClusterReconciler + | configures Envoy cluster pointing to Authorino gRPC endpoint + | + +---> IstioExtensionReconciler / EnvoyGatewayExtensionReconciler + builds ActionSets with auth-service actions + writes to WasmPlugin CR (Istio) or EnvoyExtensionPolicy (EG) + +Request arrives at Envoy + | + v +wasm-shim evaluates ActionSet predicates (hostname, path, method) + | + v +wasm-shim sends gRPC ext_authz Check to Authorino + - scope = AuthConfig name (SHA256 hash) + - Authorino looks up AuthConfig by host in radix-tree index + | + v +Authorino executes 5-phase pipeline: + 1. Authentication (at least ONE must succeed -> 401 if all fail) + 2. Metadata (external data fetching, optional) + 3. Authorization (ALL must succeed -> 403 if any fail) + 4. Response (build headers/dynamic metadata) + 5. Callbacks (fire-and-forget HTTP calls) + | + v +Response: OK (200) or UNAUTHENTICATED (401) or PERMISSION_DENIED (403) +``` + +## CRD Spec Structure + +### AuthPolicy (`kuadrant.io/v1`) + +```yaml +apiVersion: kuadrant.io/v1 +kind: AuthPolicy +metadata: + name: my-auth +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute # or Gateway + name: my-route + + # Top-level conditions (CEL predicates) + when: + - predicate: "request.path.startsWith('/api')" + + # Named reusable patterns + patterns: + internal-source: + - selector: request.headers.x-source + operator: eq + value: internal + + # Auth scheme + authScheme: + authentication: + api-key: + apiKey: + selector: + matchLabels: + app: my-api + credentials: + authorizationHeader: + prefix: APIKEY + jwt: + jwt: + issuerUrl: https://auth.example.com + + metadata: + user-info: + userInfo: + identitySource: jwt + + authorization: + check-role: + patternMatching: + rules: + - selector: auth.identity.roles + operator: incl + value: admin + opa-policy: + opa: + rego: | + allow { input.request.method == "GET" } + allValues: true + + response: + unauthenticated: + code: 401 + message: + value: "Authentication required" + unauthorized: + code: 403 + success: + headers: + x-auth-user: + plain: + selector: auth.identity.username + dynamicMetadata: + auth-data: + json: + properties: + user: + selector: auth.identity.username + + callbacks: + audit-log: + http: + url: https://audit.example.com/log + method: POST + + # OR use defaults/overrides for policy hierarchy + defaults: + strategy: merge # "atomic" (default) or "merge" + authScheme: { ... } + overrides: + strategy: atomic + authScheme: { ... } +``` + +### AuthConfig (`authorino.kuadrant.io/v1beta3`) + +Generated by kuadrant-operator. Users don't create these directly. + +```yaml +apiVersion: authorino.kuadrant.io/v1beta3 +kind: AuthConfig +metadata: + name: + namespace: + labels: + kuadrant.io/managed-by: kuadrant + kuadrant.io/auth: "true" +spec: + hosts: + - # wasm-shim uses this as scope + authentication: { ... } # copied from effective AuthPolicy + metadata: { ... } + authorization: { ... } + response: { ... } + callbacks: { ... } +``` + +## Authentication Methods + +| Method | Spec field | Description | +|--------|-----------|-------------| +| API Key | `apiKey` | Validates keys stored in K8s Secrets (matched by label selector) | +| JWT/OIDC | `jwt` | Validates JWTs via OIDC Discovery or static JWKS | +| OAuth2 Introspection | `oauth2Introspection` | Validates opaque tokens via introspection endpoint | +| K8s TokenReview | `kubernetesTokenReview` | Validates K8s ServiceAccount tokens | +| x509 mTLS | `x509` | Validates client certificates against CA Secrets | +| Plain | `plain` | Extracts pre-authenticated identity from request context | +| Anonymous | `anonymous` | Allows unauthenticated access (no credentials required) | + +**Credential extraction** (where to find the token/key in the request): +- `authorizationHeader.prefix` — `Authorization: ` +- `customHeader.name` — Custom HTTP header +- `queryString.name` — Query parameter +- `cookie.name` — Cookie value + +## Authorization Methods + +| Method | Spec field | Description | +|--------|-----------|-------------| +| Pattern Matching | `patternMatching` | JSON pattern rules with operators (eq, neq, incl, excl, matches) | +| OPA/Rego | `opa` | Open Policy Agent policies (inline or external bundle) | +| K8s SubjectAccessReview | `kubernetesSubjectAccessReview` | Kubernetes RBAC check | +| SpiceDB | `spicedb` | Authzed/SpiceDB permission check | + +## Policy Merging + +Same strategy as RateLimitPolicy (see `capabilities/rate-limiting.md`): + +| Strategy | Defaults behavior | Overrides behavior | +|----------|-------------------|-------------------| +| `atomic` | Target wins if non-empty, else source | Source always wins | +| `merge` | Target rules + source rules not in target | Source rules + target rules not in source | + +Rules are keyed by rule ID within each section: +- `authentication#` +- `metadata#` +- `authorization#` +- `response.success.headers#` +- `response.success.metadata#` +- `callbacks#` + +## Cross-Repo Contracts + +### kuadrant-operator -> Authorino (via AuthConfig CR) + +kuadrant-operator creates AuthConfig CRs in Authorino's namespace: + +- **Name**: `SHA256(pathID)` where pathID = `gateway//|httproute//|` +- **Host**: Same SHA256 hash (used by wasm-shim as the ext_authz scope) +- **Labels**: `kuadrant.io/managed-by: kuadrant`, `kuadrant.io/auth: "true"` +- **Spec**: Auth rules copied from effective AuthPolicy + +Authorino watches AuthConfig CRs, builds evaluators, and indexes by host in a radix tree. + +### kuadrant-operator -> wasm-shim (via WasmPlugin CR) + +Auth actions in the WASM config: + +```json +{ + "actions": [{ + "service": "auth-service", + "scope": "", + "predicates": ["request.path.startsWith('/api')"] + }] +} +``` + +- **Service name**: `auth-service` +- **Scope**: SHA256 hash matching the AuthConfig host +- **Failure mode**: deny (configurable via `AUTH_SERVICE_FAILURE_MODE` env var) +- **Timeout**: 200ms (configurable via `AUTH_SERVICE_TIMEOUT` env var) + +### wasm-shim -> Authorino (gRPC ext_authz) + +- **Protocol**: `envoy.service.auth.v3.Authorization.Check` +- **Port**: 50051 (default) +- **Host**: `-authorino-authorization..svc.cluster.local` +- **Request**: Envoy `CheckRequest` with HTTP attributes (method, path, headers, host) +- **Context extension**: `host` set to AuthConfig name (SHA256 hash) for index lookup + +### kuadrant-operator -> Envoy (cluster config) + +kuadrant-operator creates gateway-provider-specific resources to register Authorino as an Envoy cluster: + +| Provider | Resource | Cluster name | +|----------|----------|-------------| +| Istio | EnvoyFilter (patch) | `kuadrant-auth-` | +| Envoy Gateway | EnvoyPatchPolicy | `kuadrant-auth-` | + +## Authorino Internals + +### Host Index (Radix Tree) + +Authorino indexes AuthConfigs by `spec.hosts` in a radix tree: +- Keys are dot-reversed: `api.example.com` → `.com.example.api` +- Wildcards (`*`) match any suffix at a tree level +- Lookup: deepest exact match first, then traverse up for wildcards + +For kuadrant-managed AuthConfigs, the host is the SHA256 hash (not a real hostname). wasm-shim passes this hash as a context extension so Authorino can look it up. + +### Auth Pipeline + +Each phase groups evaluators by `priority` (default 0). Within a priority group, evaluators run concurrently. + +| Phase | Field | Requirement | Failure code | +|-------|-------|-------------|-------------| +| 1. Authentication | `authentication` | At least ONE succeeds | 401 UNAUTHENTICATED | +| 2. Metadata | `metadata` | Optional (failures ignored) | — | +| 3. Authorization | `authorization` | ALL must succeed | 403 PERMISSION_DENIED | +| 4. Response | `response` | Optional (failures ignored) | — | +| 5. Callbacks | `callbacks` | Fire-and-forget | — | + +### Authorization JSON + +Available to all evaluators as context: + +```json +{ + "context": { }, + "request": { "http": { "method": "GET", "path": "/api", "headers": {} } }, + "auth": { + "identity": { }, + "metadata": { "source-name": { } }, + "authorization": { "policy-name": { } }, + "response": { "item-name": { } }, + "callbacks": { "callback-name": { } } + } +} +``` + +Selectors use GJSON path syntax: `auth.identity.username`, `request.http.headers.authorization` + +Custom modifiers: `@extract`, `@replace`, `@case:upper|lower`, `@base64:encode|decode`, `@strip` + +### Pattern Expression Operators + +| Operator | Description | +|----------|-------------| +| `eq` | Equal to | +| `neq` | Not equal to | +| `incl` | Includes (array contains) | +| `excl` | Excludes (array does not contain) | +| `matches` | Regex match | + +### Authorino Operator CRD + +`Authorino` CR (`operator.authorino.kuadrant.io/v1beta1`) configures deployment: +- `replicas`, `image`, `clusterWide` (all namespaces vs single) +- `listener.ports.grpc` (default 50051), `listener.ports.http` (default 5001) +- `listener.tls` — TLS cert Secret reference +- `oidcServer` — Festival Wristband OIDC Discovery (port 8083) +- `authConfigLabelSelectors` — filter which AuthConfigs to watch +- `secretLabelSelectors` — filter Secrets (default: `authorino.kuadrant.io/managed-by=authorino`) +- `evaluatorCacheSize` — per-evaluator cache in MB (default 1) + +## Action Ordering in WASM Config + +Auth and rate-limit actions coexist in the same ActionSet. The order depends on whether rate-limit rules reference auth data: + +``` +If rate-limit CEL expressions reference auth.* properties: + → auth action FIRST, then rate-limit action (post-auth) +Else: + → rate-limit action FIRST, then auth action (pre-auth) +``` + +This is determined by `hasAuthAccess()` in the WASM config builder. + +## Key Source Files + +### kuadrant-operator +- CRD types: [`api/v1/authpolicy_types.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/api/v1/authpolicy_types.go) +- Merge strategies: [`api/v1/merge_strategies.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/api/v1/merge_strategies.go) +- Effective policy calc: [`internal/controller/effective_auth_policies_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/effective_auth_policies_reconciler.go) +- AuthConfig reconciler: [`internal/controller/authconfigs_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/authconfigs_reconciler.go) +- Auth helpers (naming, action building): [`internal/controller/auth_workflow_helpers.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/auth_workflow_helpers.go) +- WASM types: [`internal/wasm/types.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/wasm/types.go) +- WASM service config: [`internal/wasm/utils.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/wasm/utils.go) +- Istio auth cluster: [`internal/controller/istio_auth_cluster_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/istio_auth_cluster_reconciler.go) +- EG auth cluster: [`internal/controller/envoy_gateway_auth_cluster_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/envoy_gateway_auth_cluster_reconciler.go) +- Data plane workflow: [`internal/controller/data_plane_policies_workflow.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/data_plane_policies_workflow.go) + +### authorino +- AuthConfig CRD types: [`api/v1beta3/auth_config_types.go`](https://github.com/Kuadrant/authorino/blob/main/api/v1beta3/auth_config_types.go) +- Auth service (request handling): [`pkg/service/auth.go`](https://github.com/Kuadrant/authorino/blob/main/pkg/service/auth.go) +- Auth pipeline: [`pkg/service/auth_pipeline.go`](https://github.com/Kuadrant/authorino/blob/main/pkg/service/auth_pipeline.go) +- Host index (radix tree): [`pkg/index/index.go`](https://github.com/Kuadrant/authorino/blob/main/pkg/index/index.go) +- Evaluator config: [`pkg/evaluators/config.go`](https://github.com/Kuadrant/authorino/blob/main/pkg/evaluators/config.go) +- AuthConfig controller: [`controllers/auth_config_controller.go`](https://github.com/Kuadrant/authorino/blob/main/controllers/auth_config_controller.go) +- Status updater: [`controllers/auth_config_status_updater.go`](https://github.com/Kuadrant/authorino/blob/main/controllers/auth_config_status_updater.go) + +### authorino-operator +- Authorino CRD types: [`api/v1beta1/authorino_types.go`](https://github.com/Kuadrant/authorino-operator/blob/main/api/v1beta1/authorino_types.go) +- Controller: [`controllers/authorino_controller.go`](https://github.com/Kuadrant/authorino-operator/blob/main/controllers/authorino_controller.go) + +### wasm-shim +- Auth service client: [`src/services/auth.rs`](https://github.com/Kuadrant/wasm-shim/blob/main/src/services/auth.rs) +- Config parsing: [`src/configuration.rs`](https://github.com/Kuadrant/wasm-shim/blob/main/src/configuration.rs) diff --git a/docs/specs/capabilities/cross-cutting.md b/docs/specs/capabilities/cross-cutting.md new file mode 100644 index 0000000..f8b2702 --- /dev/null +++ b/docs/specs/capabilities/cross-cutting.md @@ -0,0 +1,387 @@ +# Cross-Cutting Conventions + +Shared patterns, naming conventions, and standards used consistently across Kuadrant repositories. + +## API Groups and Versioning + +| API Group | Owner repo | Versions | CRDs | +|-----------|-----------|----------|------| +| `kuadrant.io` | kuadrant-operator | v1, v1beta1, v1alpha1 | AuthPolicy, RateLimitPolicy, DNSPolicy, TLSPolicy, TokenRateLimitPolicy, Kuadrant | +| `kuadrant.io` | dns-operator | v1alpha1 | DNSRecord, DNSHealthCheckProbe | +| `authorino.kuadrant.io` | authorino | v1beta3, v1beta2 | AuthConfig | +| `operator.authorino.kuadrant.io` | authorino-operator | v1beta1 | Authorino | +| `limitador.kuadrant.io` | limitador-operator | v1alpha1 | Limitador | + +**Stability convention**: +- `v1` — stable, no breaking changes +- `v1beta1` — beta, may change +- `v1alpha1` — alpha, expect breaking changes + +No conversion webhooks — version upgrades are handled in-place. + +## Labels + +### Well-known labels (`kuadrant.io/`) + +| Label | Value | Purpose | Used by | +|-------|-------|---------|---------| +| `kuadrant.io/managed-by` | `kuadrant` | Marks operator-managed resources | kuadrant-operator | +| `kuadrant.io/auth` | `true` | Auth-related resources (AuthConfig) | kuadrant-operator | +| `kuadrant.io/ratelimit` | `true` | Rate-limit resources | kuadrant-operator | +| `kuadrant.io/tokenratelimit` | `true` | Token rate-limit resources | kuadrant-operator | +| `kuadrant.io/tracing` | `true` | Tracing resources | kuadrant-operator | +| `kuadrant.io/listener-name` | `` | Listener name on DNSRecords | kuadrant-operator | +| `kuadrant.io/topology` | — | Topology-related objects | kuadrant-operator | +| `kuadrant.io/observability` | — | Observability resources | kuadrant-operator | +| `kuadrant.io/developerportal` | — | Developer portal resources | kuadrant-operator | +| `kuadrant.io/default-provider` | `true` | Default DNS provider Secret | dns-operator | +| `kuadrant.io/multicluster-kubeconfig` | `true` | Multi-cluster connection Secrets | dns-operator | +| `kuadrant.io/authoritative-record` | `true` | Authoritative DNSRecord (primary) | dns-operator | +| `kuadrant.io/health-probes-owner` | `` | Links probes to DNSRecords | dns-operator | +| `authorino.kuadrant.io/managed-by` | `authorino` | Secrets managed by Authorino | authorino | + +### Standard Kubernetes labels + +Applied via `CommonLabels()` in kuadrant-operator: + +```yaml +app: kuadrant +app.kubernetes.io/component: kuadrant +app.kubernetes.io/managed-by: kuadrant-operator +app.kubernetes.io/instance: kuadrant +app.kubernetes.io/name: kuadrant +app.kubernetes.io/part-of: kuadrant +``` + +### Policy-affected condition labels + +Dynamic label pattern on Gateway API resources: `kuadrant.io/{PolicyKind}Affected` (e.g., `kuadrant.io/AuthPolicyAffected`). + +## Annotations + +| Annotation | Purpose | Used by | +|-----------|---------|---------| +| `kuadrant.io/delete` | Marks resource for deletion cleanup | kuadrant-operator | +| `extensions.kuadrant.io/trigger-time` | Extension trigger timestamp | kuadrant-operator | +| `extensions.kuadrant.io/trigger-reason` | Extension trigger reason | kuadrant-operator | + +## Finalizers + +**Naming pattern**: `kuadrant.io/{resource-name}` + +| Finalizer | Resource | Repo | +|-----------|----------|------| +| `kuadrant.io/dns-record` | DNSRecord | dns-operator | +| `kuadrant.io/dnshealthcheckprobe` | DNSHealthCheckProbe | dns-operator | +| `kuadrant.io/extensions` | Extension resources | kuadrant-operator | +| `kuadrant.io/developerportal` | Developer portal | kuadrant-operator | + +All use `controllerutil.AddFinalizer()` / `controllerutil.RemoveFinalizer()`. + +## Status Conditions + +### Standard condition types + +| Type | Meaning | Used by | +|------|---------|---------| +| `Accepted` | Policy validated and attached to target | All policy CRDs | +| `Enforced` | Sub-resources created and ready | All policy CRDs | +| `SubResourcesHealthy` | Health probes passing | DNSPolicy | +| `Ready` | Resource fully reconciled | AuthConfig, Limitador, Authorino, DNSRecord | +| `Available` | Resource has linked hosts | AuthConfig | +| `Healthy` | Health checks passing | DNSRecord | +| `ReadyForDelegation` | Ready for multi-cluster delegation | DNSRecord | +| `Active` | Member of active failover group | DNSRecord | + +### Standard condition reasons + +Defined in `internal/kuadrant/conditions.go`: + +```go +PolicyReasonEnforced // fully enforced +PolicyReasonOverridden // overridden by another policy +PolicyReasonUnknown // unknown state +PolicyReasonMissingDependency // required CRD/operator not installed +PolicyReasonMissingResource // required resource not found +PolicyReasonInvalidCelExpression // CEL validation failure +``` + +From Gateway API: +```go +PolicyReasonTargetNotFound // targetRef doesn't resolve +PolicyReasonInvalid // validation error +PolicyReasonConflicted // older policy already controls target +``` + +### Condition helpers + +```go +AcceptedCondition(policy, err) *metav1.Condition +EnforcedCondition(policy, err, fullyEnforced) *metav1.Condition +meta.SetStatusCondition(&conditions, condition) +meta.IsStatusConditionTrue(conditions, "Type") +``` + +## Error Handling + +### PolicyError interface + +All policy errors implement `PolicyError` with a `Reason()` method mapping to Gateway API condition reasons: + +| Error type | Reason | When | +|-----------|--------|------| +| `ErrTargetNotFound` | TargetNotFound | Policy target missing from topology | +| `ErrInvalid` | Invalid | Validation failure (e.g., issuer not found) | +| `ErrConflict` | Conflicted | Older policy already controls same target | +| `ErrDependencyNotInstalled` | MissingDependency | Required CRD not installed | +| `ErrSystemResource` | MissingResource | Required system resource missing | +| `ErrCelValidation` | InvalidCelExpression | CEL expression validation failure | +| `ErrOverridden` | Overridden | Policy overridden by higher-priority policy | +| `ErrNoRoutes` | Unknown | Policy not in path to any routes | +| `ErrOutOfSync` | Unknown | Components not synced | + +### Wrapping pattern + +```go +var policyErr PolicyError +if !errors.As(err, &policyErr) { + policyErr = NewErrUnknown(p.Kind(), err) +} +``` + +## Container Images + +### Registry + +All images published to `quay.io/kuadrant/`: + +| Image | Repo | +|-------|------| +| `quay.io/kuadrant/kuadrant-operator` | kuadrant-operator | +| `quay.io/kuadrant/authorino` | authorino | +| `quay.io/kuadrant/authorino-operator` | authorino-operator | +| `quay.io/kuadrant/limitador` | limitador | +| `quay.io/kuadrant/limitador-operator` | limitador-operator | +| `quay.io/kuadrant/dns-operator` | dns-operator | +| `quay.io/kuadrant/wasm-shim` | wasm-shim | +| `quay.io/kuadrant/console-plugin` | console-plugin | + +**Bundle images**: `{image}-bundle:TAG` +**Catalog images**: `{image}-catalog:TAG` + +### Tagging convention + +```makefile +# Semantic version → v-prefixed tag +VERSION=1.2.3 → IMAGE_TAG=v1.2.3 + +# Non-semantic → dev tag +VERSION=main → IMAGE_TAG=dev (or latest) + +# Default (no version) → latest +VERSION=0.0.0 → IMAGE_TAG=latest +``` + +## Toolchain + +### Go repositories + +| Convention | Value | +|-----------|-------| +| Go version | 1.25.5 | +| Controller framework | controller-runtime | +| Test framework | Ginkgo v2 + Gomega | +| Linter | golangci-lint | +| Code generation | controller-gen (kubebuilder) | +| Build | `go build` with ldflags for version injection | + +### Rust repositories (limitador, wasm-shim) + +| Convention | Value | +|-----------|-------| +| Workspace | Multi-crate workspace with resolver v2 | +| Release profile | LTO enabled, `codegen-units = 1` | +| Test | `cargo test --all-features` | +| Benchmarks | Criterion | + +## Makefile Targets + +Standard across all operator repos: + +| Target | Purpose | +|--------|---------| +| `build` | Compile binary | +| `test` / `test-unit` | Unit tests | +| `test-integration` | Integration tests (requires cluster) | +| `test-e2e` | End-to-end tests | +| `generate` | Generate DeepCopy/CRD code | +| `manifests` | Generate CRD/RBAC manifests | +| `fmt` / `vet` / `lint` | Code quality | +| `docker-build` / `docker-push` | Container image build/push | +| `install` / `uninstall` | CRD install/remove | +| `deploy` / `undeploy` | Operator deploy/remove | +| `bundle` | Generate OLM bundle | +| `helm-build` | Generate Helm chart | +| `local-setup` / `local-cleanup` | Kind cluster lifecycle | +| `verify-manifests` | CI: ensure manifests up-to-date | +| `verify-bundle` | CI: ensure bundle up-to-date | +| `verify-fmt` | CI: check code formatting | + +## Environment Variables + +### kuadrant-operator + +| Variable | Default | Purpose | +|----------|---------|---------| +| `LOG_LEVEL` | `info` | Logging level | +| `LOG_MODE` | `production` | Log format (production/development) | +| `OTEL_EXPORTER_OTLP_ENDPOINT` | (empty) | OpenTelemetry collector endpoint | +| `OTEL_EXPORTER_OTLP_INSECURE` | `false` | Allow insecure OTLP connection | +| `OTEL_SERVICE_NAME` | `kuadrant-operator` | Service name for telemetry | +| `AUTH_SERVICE_TIMEOUT` | `200ms` | WASM auth service timeout | +| `AUTH_SERVICE_FAILURE_MODE` | `deny` | WASM auth failure behavior | +| `DNS_DEFAULT_TTL` | `300` | Default DNS record TTL (seconds) | +| `DNS_DEFAULT_LB_TTL` | `60` | Default load-balanced DNS TTL (seconds) | + +### Common operator flags + +| Flag | Default | Purpose | +|------|---------|---------| +| `--metrics-bind-address` | `:8080` | Prometheus metrics endpoint | +| `--health-probe-bind-address` | `:8081` | Health/readiness probes | +| `--leader-elect` | `false` | Enable leader election | + +## Observability + +### Metrics (Prometheus) + +```go +kuadrant_policies_total{kind} // gauge: total policies by kind +kuadrant_policies_enforced{kind, status} // gauge: enforced policies by kind and status +``` + +Exposed on `:8080/metrics` via controller-runtime. + +### Tracing (OpenTelemetry) + +- W3C Trace Context propagation +- OTLP exporter (gRPC) when `OTEL_EXPORTER_OTLP_ENDPOINT` set +- Spans on reconciliation workflows via `traceReconcileFunc()` +- Span attributes include: git SHA, dirty state, Go version + +### Logging + +- Dual output: Zap (console) + OTLP (remote, if configured) +- Structured JSON in production mode +- Standard `logr.Logger` interface throughout + +## CEL (Common Expression Language) + +### Root bindings (available in all CEL expressions) + +```go +request // request attributes (method, path, headers, host, etc.) +source // source attributes +destination // destination attributes +connection // connection attributes +``` + +### Wasm-shim specific functions + +``` +requestBodyJSON("/path") // parse request body JSON +responseBodyJSON("/path") // parse response body JSON (TokenRateLimitPolicy) +queryMap(request.query) // parse query string to map +``` + +### Validation + +CEL expressions are validated at reconcile time. Invalid expressions produce `ErrCelValidation` errors with reason `InvalidCelExpression`. + +## Conflict Resolution + +All policies follow the same conflict resolution rule: **oldest policy wins** (by `metadata.creationTimestamp`). Newer conflicting policies get `Accepted=False` with reason `Conflicted`. + +## Owner References + +All generated sub-resources set `ownerReferences` to the creating policy/operator resource. This enables: +- Automatic garbage collection on parent deletion +- Ownership tracking in topology +- `utils.IsOwnedBy()` checks for link resolution + +## Project Layout (Go operators) + +``` +├── api/ +│ ├── v1/ # stable CRD types +│ ├── v1beta1/ # beta CRD types +│ └── v1alpha1/ # alpha CRD types +├── cmd/main.go # entrypoint, flag parsing, logger setup +├── internal/ +│ ├── controller/ # reconcilers, workflows, state_of_the_world +│ └── {component}/ # component-specific helpers +├── pkg/ # shared packages (exported) +├── config/ +│ ├── crd/ # generated CRD manifests +│ ├── manager/ # operator Deployment +│ ├── rbac/ # RBAC manifests +│ └── samples/ # example CRs +├── bundle/ # OLM bundle +├── charts/ # Helm charts +├── tests/ +│ ├── common/ # shared test helpers +│ ├── istio/ # Istio-specific integration tests +│ ├── envoygateway/ # EnvoyGateway-specific integration tests +│ └── bare_k8s/ # pure K8s tests +├── Makefile +├── go.mod +└── Dockerfile +``` + +## Test Conventions + +### Timeouts + +```go +TimeoutShort = 5s +TimeoutMedium = 10s +TimeoutLongerMedium = 15s +TimeoutLong = 30s +RetryIntervalMedium = 250ms +``` + +### Async assertions + +```go +Eventually(func(g Gomega) { + obj := &v1.SomeResource{} + g.Expect(client.Get(ctx, key, obj)).To(Succeed()) + g.Expect(obj.Status.Conditions).To(ContainElement( + MatchFields(IgnoreExtras, Fields{ + "Type": Equal("Ready"), + "Status": Equal(metav1.ConditionTrue), + }), + )) +}).WithContext(ctx).Should(Succeed()) +``` + +### Build tags + +```go +//go:build integration // integration tests +//go:build unit // unit tests (if separated) +``` + +## Build Info Injection + +All Go operators inject build metadata via ldflags: + +```go +var ( + gitSHA string // git commit SHA + dirty string // "true" if uncommitted changes + version string // semantic version +) +``` + +Used in OTEL resource attributes and operator status reporting. diff --git a/docs/specs/capabilities/dns.md b/docs/specs/capabilities/dns.md new file mode 100644 index 0000000..f816482 --- /dev/null +++ b/docs/specs/capabilities/dns.md @@ -0,0 +1,349 @@ +# DNS + +## Components + +| Component | Repo | Language | Role | +|-----------|------|----------|------| +| DNSPolicy CRD | kuadrant-operator | Go | User-facing policy (v1) attached to Gateway | +| DNS Operator | dns-operator | Go | Reconciles DNSRecord CRs into cloud DNS provider records | +| DNSRecord CRD | dns-operator | Go | Internal CR representing DNS endpoints for a listener hostname | +| DNSHealthCheckProbe CRD | dns-operator | Go | Health check probe for individual DNS endpoints | + +## Data Flow + +``` +User creates DNSPolicy (kuadrant.io/v1) + targets a Gateway (optionally a specific Listener via sectionName) + | + v +kuadrant-operator reconciles + | + +---> DNSPoliciesValidator + | validates target, checks for conflicts, verifies dependencies + | + +---> EffectiveDNSPoliciesReconciler + | for each targeted listener with a hostname: + | builds DNSRecord CR with endpoints from Gateway status addresses + | applies load balancing config (weight, geo, defaultGeo) + | sets health check spec, provider ref, delegation flag + | + +---> DNSPolicyStatusUpdater + propagates DNSRecord readiness back to DNSPolicy status + +dns-operator reconciles DNSRecord + | + +---> Discovers DNS zone from provider (suffix-matches rootHost) + +---> Assigns ownerID (from spec or UID hash) + +---> Creates TXT registry records for ownership tracking + +---> Publishes A/AAAA/CNAME endpoints to DNS provider + +---> Creates DNSHealthCheckProbe CRs for each endpoint target + +---> Filters unhealthy endpoints from publishing +``` + +### Multi-Cluster Delegation Flow + +``` +Secondary cluster Primary cluster + DNSRecord (delegate: true) RemoteDNSRecordReconciler + with spec.endpoints watches secondary cluster DNSRecords + | | + v v + Sets ReadyForDelegation Creates/updates authoritative DNSRecord + condition when ready merging endpoints from all secondaries + | + v + Authoritative DNSRecord published + to actual DNS provider + | + v + Remote status flows back to secondary + via status.remoteRecordStatuses[clusterID] +``` + +## CRD Spec Structure + +### DNSPolicy (`kuadrant.io/v1`) + +```yaml +apiVersion: kuadrant.io/v1 +kind: DNSPolicy +metadata: + name: my-dns +spec: + targetRef: + group: gateway.networking.k8s.io + kind: Gateway # Gateway only (not HTTPRoute) + name: prod-gateway + sectionName: api # optional: target specific listener + + # DNS provider credentials (max 1) + providerRefs: + - name: my-aws-credentials # Secret in same namespace + + # Load balancing for multi-cluster / geo routing (optional) + loadBalancing: + weight: 120 # traffic weight (default 120) + geo: "GEO-EU" # geographic code (provider-specific) + defaultGeo: false # catch-all for unmatched geos (one per hostname) + + # Health checks (optional) + healthCheck: + port: 443 # 80, 443, or 1024-49151 + protocol: HTTPS # HTTP or HTTPS + path: /healthz + interval: 5m # check frequency + failureThreshold: 5 # consecutive failures before unhealthy + additionalHeadersRef: # optional: Secret with custom headers + name: health-headers + + # Addresses to exclude from DNS records (optional, max 20) + excludeAddresses: + - 10.0.0.0/8 # CIDR blocks, IPs, or hostnames + + # Multi-cluster delegation mode (immutable) + delegate: false +``` + +**Note**: DNSPolicy does NOT support defaults/overrides — it attaches directly to Gateway listeners. No policy merging hierarchy. + +### DNSRecord (`kuadrant.io/v1alpha1`) + +Generated by kuadrant-operator. Users don't create these directly. + +```yaml +apiVersion: kuadrant.io/v1alpha1 +kind: DNSRecord +metadata: + name: prod-gateway-api # - + namespace: my-app + labels: + kuadrant.io/listener-name: api + ownerReferences: + - kind: DNSPolicy # owned by the DNSPolicy +spec: + rootHost: api.example.com # from listener.hostname (immutable) + + providerRef: + name: my-aws-credentials # from DNSPolicy.spec.providerRefs[0] + + delegate: false # immutable, mutually exclusive with providerRef + + ownerID: "a1b2c3d4" # immutable, defaults to UID hash (8 chars, base36) + + endpoints: # built from Gateway status addresses + load balancing + - dnsName: api.example.com + targets: ["1.2.3.4"] + recordType: A + recordTTL: 300 + - dnsName: a1b2c3.klb.api.example.com # multi-cluster: clusterHash.klb.domain + targets: ["1.2.3.4"] + recordType: A + recordTTL: 60 + providerSpecific: + - name: weight + value: "120" + - name: geo-code + value: "GEO-EU" + + healthCheck: # copied from DNSPolicy + port: 443 + protocol: HTTPS + path: /healthz + interval: 5m + failureThreshold: 5 +``` + +### DNSHealthCheckProbe (`kuadrant.io/v1alpha1`) + +Generated by dns-operator. One per unique endpoint target IP. + +```yaml +apiVersion: kuadrant.io/v1alpha1 +kind: DNSHealthCheckProbe +metadata: + name: + labels: + kuadrant.io/health-probes-owner: + ownerReferences: + - kind: DNSRecord +spec: + hostname: api.example.com # rootHost from parent DNSRecord + address: 1.2.3.4 # target IP to probe + port: 443 + protocol: HTTPS + path: /healthz + interval: 5m + failureThreshold: 5 +status: + healthy: true + consecutiveFailures: 0 + lastCheckedAt: "2026-03-19T10:00:00Z" +``` + +## Cross-Repo Contracts + +### kuadrant-operator -> dns-operator (via DNSRecord CR) + +kuadrant-operator creates DNSRecord CRs: + +- **Name**: `-` +- **Namespace**: Same as DNSPolicy +- **Owner**: DNSPolicy (via OwnerReference) +- **Labels**: `kuadrant.io/listener-name: ` +- **RootHost**: From `listener.hostname` +- **ProviderRef**: From `DNSPolicy.spec.providerRefs[0]` +- **HealthCheck**: Copied from DNSPolicy +- **Endpoints**: Built from Gateway `status.addresses`, filtered by `excludeAddresses` + +### Endpoint Building (single vs multi-cluster) + +**Simple (no loadBalancing)**: +``` +api.example.com → [gateway-address-1, gateway-address-2] +TTL: 300s (DNS_DEFAULT_TTL env, default 300) +``` + +**Multi-cluster (with loadBalancing)**: +``` +api.example.com + └─ CNAME → klb.api.example.com (TTL: 300s) + └─ geo-specific CNAME → .klb.api.example.com (TTL: 60s) + └─ weighted A → -.klb.api.example.com (TTL: 60s) + └─ targets: [1.2.3.4] + weight: 120, geo-code: GEO-EU +``` + +Cluster hash: 6-char base36 hash of Kubernetes cluster UID (`utils.GetClusterUID()`). + +## DNS Provider System + +### Supported Providers + +| Provider | Secret type | Required keys | Geo codes | +|----------|------------|---------------|-----------| +| AWS Route53 | `kuadrant.io/aws` | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | `GEO-` prefix continents | +| Google Cloud DNS | `kuadrant.io/gcp` | `GOOGLE` (JSON), `PROJECT_ID` | Region/zone codes | +| Azure DNS | `kuadrant.io/azure` | `azure.json` (service principal) | Geographic region codes | +| CoreDNS | `kuadrant.io/coredns` | (optional `ZONES`) | N/A | +| In-memory | `kuadrant.io/inmemory` | (optional `INMEM_INIT_ZONES`) | N/A | + +Provider is determined by the Secret's `type` field referenced by `spec.providerRef`. + +### Default Provider + +If `spec.providerRef` is omitted, dns-operator looks for a Secret labeled `kuadrant.io/default-provider=true`. + +## Ownership and TXT Registry + +dns-operator tracks record ownership via TXT records: + +- **TXT prefix**: `kuadrant-` +- **TXT content**: ownerID + endpoint labels + targets +- **Wildcard handling**: `*.api.example.com` → `kuadrant-wildcard-api.example.com` +- **Multi-owner zones**: Multiple DNSRecords can share the same DNS zone safely + +Each DNSRecord gets an ownerID: +- From `spec.ownerID` if set (immutable, 6-36 chars) +- Otherwise generated from UID hash (8 chars, base36) + +Tracked in `status.domainOwners` — list of all owners for the root domain. + +## Health-Aware DNS + +### How probes affect publishing + +1. dns-operator creates DNSHealthCheckProbe per unique endpoint target +2. Probes run at configured `interval`, tracking `consecutiveFailures` +3. When failures >= `failureThreshold`, probe marks target unhealthy +4. `healthCheckAdapter` wraps DNSRecord, filtering unhealthy endpoints from `GetEndpoints()` +5. Next reconciliation publishes only healthy endpoints to DNS provider +6. Wildcard hosts (`*.example.com`) skip health checks + +### Status conditions + +| Condition | True when | False when | +|-----------|-----------|------------| +| `Healthy` | All probes pass (`AllChecksPassed`) | All probes fail (`HealthChecksFailed`) | +| | Some probes pass (`SomeChecksPassed`) | | + +## Multi-Cluster Delegation + +### Roles + +| Role | Flag | Responsibilities | +|------|------|-----------------| +| Primary | `--delegation-role=primary` | Watches all clusters, publishes to DNS provider, manages authoritative DNSRecords | +| Secondary | `--delegation-role=secondary` | Creates delegating DNSRecords, does NOT talk to DNS provider | + +### ReadyForDelegation requirements + +A secondary DNSRecord must meet all conditions before primary will accept it: +- Finalizer set (`kuadrant.io/dns-record`) +- ownerID assigned +- Validation passed +- Health probes created +- Group assigned + +### Authoritative DNSRecord + +On the primary cluster, delegation creates an internal `AuthoritativeDNSRecord`: +- Labeled: `kuadrant.io/authoritative-record=true` +- Aggregates endpoints from all delegating secondaries +- Published to actual DNS provider via endpoint provider +- Remote status flows back to secondaries via `status.remoteRecordStatuses[clusterID]` + +### Multi-cluster connection + +Primary discovers secondaries via Secrets labeled `kuadrant.io/multicluster-kubeconfig=true`. + +## DNSPolicy Status + +### Conditions + +| Condition | Description | +|-----------|-------------| +| `Accepted` | Policy validated and attached to target Gateway | +| `Enforced` | All owned DNSRecords are ready | +| `SubResourcesHealthy` | All health probes passing (only if healthCheck configured) | + +### Per-record conditions + +`status.recordConditions` maps rootHost to per-DNSRecord conditions, propagated from dns-operator. + +`status.totalRecords` tracks the count of managed DNSRecords. + +## Immutable Fields + +| Field | CRD | Why | +|-------|-----|-----| +| `spec.rootHost` | DNSRecord | DNS zone binding is permanent | +| `spec.ownerID` | DNSRecord | Ownership tracking integrity | +| `spec.delegate` | DNSRecord | Cannot switch delegation mode | +| `spec.delegate` | DNSPolicy | Propagated to DNSRecord | + +## Key Source Files + +### kuadrant-operator +- CRD types: [`api/v1/dnspolicy_types.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/api/v1/dnspolicy_types.go) +- DNS workflow: [`internal/controller/dns_workflow.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/dns_workflow.go) +- Validator: [`internal/controller/dnspolicies_validator.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/dnspolicies_validator.go) +- Effective policies reconciler: [`internal/controller/effective_dnspolicies_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/effective_dnspolicies_reconciler.go) +- DNSRecord builder: [`internal/controller/dnspolicy_dnsrecords.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/dnspolicy_dnsrecords.go) +- Status updater: [`internal/controller/dnspolicy_status_updater.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/dnspolicy_status_updater.go) + +### dns-operator +- DNSRecord CRD types: [`api/v1alpha1/dnsrecord_types.go`](https://github.com/Kuadrant/dns-operator/blob/main/api/v1alpha1/dnsrecord_types.go) +- DNSHealthCheckProbe CRD types: [`api/v1alpha1/dnshealthcheckprobe_types.go`](https://github.com/Kuadrant/dns-operator/blob/main/api/v1alpha1/dnshealthcheckprobe_types.go) +- Provider types: [`api/v1alpha1/provider_types.go`](https://github.com/Kuadrant/dns-operator/blob/main/api/v1alpha1/provider_types.go) +- Conditions: [`api/v1alpha1/conditions.go`](https://github.com/Kuadrant/dns-operator/blob/main/api/v1alpha1/conditions.go) +- DNSRecord controller: [`internal/controller/dnsrecord_controller.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/controller/dnsrecord_controller.go) +- Base reconciler: [`internal/controller/base_dnsrecord_reconciler.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/controller/base_dnsrecord_reconciler.go) +- Remote controller: [`internal/controller/remote_dnsrecord_controller.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/controller/remote_dnsrecord_controller.go) +- Health check reconciler: [`internal/controller/dnshealthcheckprobe_reconciler.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/controller/dnshealthcheckprobe_reconciler.go) +- Health check adapter: [`internal/controller/dnsrecord_healthchecks.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/controller/dnsrecord_healthchecks.go) +- Provider interface: [`internal/provider/provider.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/provider/provider.go) +- Provider factory: [`internal/provider/factory.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/provider/factory.go) +- Endpoint provider: [`internal/provider/endpoint/endpoint.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/provider/endpoint/endpoint.go) +- Authoritative provider: [`internal/provider/endpoint/authoritative_dnsrecord.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/provider/endpoint/authoritative_dnsrecord.go) +- TXT registry: [`internal/external-dns/registry/txt.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/external-dns/registry/txt.go) +- Group registry: [`internal/external-dns/registry/group.go`](https://github.com/Kuadrant/dns-operator/blob/main/internal/external-dns/registry/group.go) diff --git a/docs/specs/capabilities/gateway-policy.md b/docs/specs/capabilities/gateway-policy.md new file mode 100644 index 0000000..02518e7 --- /dev/null +++ b/docs/specs/capabilities/gateway-policy.md @@ -0,0 +1,457 @@ +# Gateway Policy Framework + +## Components + +| Component | Repo | Language | Role | +|-----------|------|----------|------| +| policy-machinery | policy-machinery | Go | Shared library: topology DAG, policy attachment, merge strategies, controller framework | +| kuadrant-operator | kuadrant-operator | Go | Consumes policy-machinery to build topology, compute effective policies, run reconciliation workflows | + +## What policy-machinery provides + +policy-machinery implements the [Gateway API Policy Attachment](https://gateway-api.sigs.k8s.io/geps/gep-713/) (GEP-713) pattern as a reusable library. It provides: + +1. **Topology** — a DAG of Gateway API resources with policies attached +2. **Policy merging** — defaults/overrides with pluggable merge strategies +3. **Effective policy computation** — walk topology paths to produce merged policies per route rule +4. **Controller framework** — event-driven reconciliation with subscriptions and workflows + +## Core Interfaces + +All defined in `machinery/types.go`: + +```go +// Base interface for all topology objects +type Object interface { + schema.ObjectKind + GetNamespace() string + GetName() string + GetLocator() string // unique ID in topology, e.g. "Gateway:default/my-gw" +} + +// Resources that can have policies attached (GEP-713) +type Targetable interface { + Object + SetPolicies([]Policy) + Policies() []Policy +} + +// Policy objects with merge logic +type Policy interface { + Object + GetTargetRefs() []PolicyTargetReference + GetMergeStrategy() MergeStrategy + Merge(Policy) Policy +} + +// Function type for custom merge logic +type MergeStrategy func(source, target Policy) Policy +``` + +## Topology DAG + +### Structure + +The topology is a directed acyclic graph where nodes are Kubernetes objects and edges represent relationships: + +``` +Kuadrant + | +GatewayClass + | +Gateway ──────────────────── [Policies: DNSPolicy, TLSPolicy] + | +Listener ─────────────────── [Policies: DNSPolicy (sectionName), TLSPolicy (sectionName)] + | +HTTPRoute ────────────────── [Policies: AuthPolicy, RateLimitPolicy] + | +HTTPRouteRule ────────────── [Policies: AuthPolicy (sectionName), RateLimitPolicy (sectionName)] + | +Service / ServicePort +``` + +Side links to generated resources: +``` +Listener ──→ DNSRecord, Certificate +HTTPRouteRule ──→ AuthConfig +Gateway ──→ WasmPlugin (Istio) / EnvoyExtensionPolicy (EG) +Kuadrant ──→ Limitador, Authorino +``` + +### Built-in Gateway API types + +Wrapper types in `machinery/gateway_api_types.go` implement `Targetable`: + +| Type | Wraps | Locator format | Parent field | +|------|-------|----------------|-------------| +| `GatewayClass` | `*gwapiv1.GatewayClass` | `GatewayClass:name` | — | +| `Gateway` | `*gwapiv1.Gateway` | `Gateway:ns/name` | — | +| `Listener` | `*gwapiv1.Listener` | `Gateway:ns/name#listener` | `Gateway` | +| `HTTPRoute` | `*gwapiv1.HTTPRoute` | `HTTPRoute:ns/name` | — | +| `HTTPRouteRule` | `*gwapiv1.HTTPRouteRule` | `HTTPRoute:ns/name#rule-idx` | `HTTPRoute` | +| `GRPCRoute` | `*gwapiv1.GRPCRoute` | `GRPCRoute:ns/name` | — | +| `Service` | `*core.Service` | `Service:ns/name` | — | +| `ServicePort` | `*core.ServicePort` | `Service:ns/name#port` | `Service` | + +### LinkFunc — defining edges + +Each edge in the DAG is defined by a `LinkFunc`: + +```go +type LinkFunc struct { + From schema.GroupKind // parent kind + To schema.GroupKind // child kind + Func func(child Object) []Object // given child, return its parents +} +``` + +**Built-in links** (in `machinery/gateway_api_topology.go`): +- `LinkGatewayClassToGatewayFunc` — Gateway references GatewayClass by `spec.gatewayClassName` +- `LinkGatewayToListenerFunc` — Gateway contains Listeners (expansion) +- `LinkListenerToHTTPRouteFunc` — HTTPRoute `spec.parentRefs` with sectionName +- `LinkGatewayToHTTPRouteFunc` — HTTPRoute `spec.parentRefs` (no listener expansion) +- `LinkHTTPRouteToHTTPRouteRuleFunc` — HTTPRoute contains rules (expansion) +- `LinkHTTPRouteRuleToServiceFunc` — BackendRefs in rules +- Similar for GRPCRoute, TCPRoute, TLSRoute, UDPRoute + +### Policy attachment + +When the topology is built, policies are attached to targetables by matching locators: + +```go +// In NewTopology(): +policiesByTargetRef[targetRef.GetLocator()] = append(..., policy) + +// Then: +targetable.SetPolicies(policiesByTargetRef[targetable.GetLocator()]) +``` + +A policy's `GetTargetRefs()` returns locators like `Gateway:default/my-gw` or `Gateway:default/my-gw#https` (with sectionName). + +### Topology options + +```go +type GatewayAPITopologyOptions struct { + GatewayClasses []*GatewayClass + Gateways []*Gateway + HTTPRoutes []*HTTPRoute + Services []*Service + Policies []Policy + Objects []Object + Links []LinkFunc + + // Expansion flags — create sub-nodes for sections + ExpandGatewayListeners bool // Gateway → Listener nodes + ExpandHTTPRouteRules bool // HTTPRoute → HTTPRouteRule nodes + ExpandServicePorts bool // Service → ServicePort nodes +} +``` + +kuadrant-operator enables all expansion flags to get the most granular topology. + +### DAG validation + +The topology builder validates the graph is acyclic using Kahn's algorithm (topological sort). If a cycle is detected, topology creation fails (unless `AllowLoops` is set). + +## Effective Policy Computation + +### Algorithm + +`EffectivePolicyForPath[T Policy](path []Targetable, predicate func(Policy) bool) *T`: + +1. **Gather** — collect all policies of type `T` from every targetable in the path, filtered by predicate +2. **Sort** — order from least specific (GatewayClass) to most specific (HTTPRouteRule) +3. **Merge** — `lo.ReduceRight()` from most-specific to least-specific, calling `policy.Merge(other)` at each step +4. **Return** — the single merged effective policy + +### Path computation + +Paths are computed via depth-first search: + +```go +// All paths from any GatewayClass to a specific HTTPRouteRule +paths := topology.Targetables().Paths(gatewayClass, httpRouteRule) +``` + +Each path is a slice: `[GatewayClass, Gateway, Listener, HTTPRoute, HTTPRouteRule]` + +### How kuadrant-operator uses it + +For each policy type, a dedicated reconciler computes effective policies: + +```go +// EffectiveAuthPolicyReconciler +for _, gatewayClass := range gatewayClasses { + for _, httpRouteRule := range httpRouteRules { + paths := targetables.Paths(gatewayClass, httpRouteRule) + for i := range paths { + effectivePolicy := EffectivePolicyForPath[*AuthPolicy]( + paths[i], + isAuthPolicyAcceptedAndNotDeletedFunc(state), + ) + if effectivePolicy != nil { + effectivePolicies[pathID] = EffectiveAuthPolicy{ + Path: paths[i], + Spec: *effectivePolicy, + SourcePolicies: SourcePoliciesFromEffectivePolicy(*effectivePolicy), + } + } + } + } +} +state.Store(StateEffectiveAuthPolicies, effectivePolicies) +``` + +Effective policy reconcilers exist for: +- `EffectiveAuthPolicyReconciler` → `StateEffectiveAuthPolicies` +- `EffectiveRateLimitPolicyReconciler` → `StateEffectiveRateLimitPolicies` +- `EffectiveTokenRateLimitPolicyReconciler` → `StateEffectiveTokenRateLimitPolicies` +- `EffectiveDNSPoliciesReconciler` (uses listener-level, not route-rule-level) +- `EffectiveTLSPoliciesReconciler` (uses listener-level) + +### Merge strategies (implemented in kuadrant-operator) + +| Strategy | Defaults | Overrides | +|----------|----------|-----------| +| `atomic` | Target wins if non-empty, else source | Source always wins | +| `merge` | Target rules + source rules not in target | Source rules + target rules not in source | + +Each rule carries a `Source` field (`kind/namespace/name`) for tracing which policy contributed it. + +## Controller Framework + +### Architecture + +```go +type Controller struct { + client *dynamic.DynamicClient + manager ctrlruntime.Manager + cache *CacheStore // watchable object store + topology *gatewayAPITopologyBuilder + runnables map[string]Runnable // resource watchers + reconcile ReconcileFunc // main reconciliation function +} +``` + +### Event flow + +``` +Resource watcher detects change (via SharedInformer) + | + v +CacheStore updated (watchable.Map) + | + v +Controller detects diff (old vs new snapshot) + - CreateEvent: UID not in old state + - UpdateEvent: UID in both, objects differ + - DeleteEvent: UID in old, not in new + | + v +Topology rebuilt from current cache + | + v +ReconcileFunc called with (events, topology, state) + | + v +Subscription matchers filter events per reconciler + | + v +Matching reconcilers execute +``` + +### Subscription + +Each reconciler declares which events it cares about: + +```go +type Subscription struct { + ReconcileFunc ReconcileFunc + Events []ResourceEventMatcher +} + +type ResourceEventMatcher struct { + Kind *schema.GroupKind // filter by resource kind + EventType *EventType // Create, Update, Delete (nil = any) + ObjectNamespace string // filter by namespace (empty = any) + ObjectName string // filter by name (empty = any) +} +``` + +### Workflow + +Workflows compose reconcilers into phases: + +```go +type Workflow struct { + Precondition ReconcileFunc // runs first (validation) + Tasks []ReconcileFunc // run concurrently + Postcondition ReconcileFunc // runs last (status updates) +} +``` + +### ReconcileFunc signature + +```go +type ReconcileFunc func( + ctx context.Context, + resourceEvents []ResourceEvent, + topology *machinery.Topology, + err error, + state *sync.Map, // shared state between reconcilers +) error +``` + +## kuadrant-operator Topology Wiring + +### Object and policy registration + +In `state_of_the_world.go`: + +```go +// Policy kinds (attached to targetables in topology) +controller.WithPolicyKinds( + DNSPolicyGroupKind, + TLSPolicyGroupKind, + AuthPolicyGroupKind, + RateLimitPolicyGroupKind, + TokenRateLimitPolicyGroupKind, +) + +// Non-policy, non-targetable objects tracked in topology +controller.WithObjectKinds( + KuadrantGroupKind, + ConfigMapGroupKind, + DeploymentGroupKind, +) +``` + +### Custom links (kuadrant-operator specific) + +| Link | From | To | Purpose | +|------|------|----|---------| +| `LinkKuadrantToGatewayClasses` | Kuadrant | GatewayClass | Root of topology | +| `LinkKuadrantToLimitador` | Kuadrant | Limitador | Component dependency | +| `LinkKuadrantToAuthorino` | Kuadrant | Authorino | Component dependency | +| `LinkLimitadorToDeployment` | Limitador | Deployment | Readiness tracking | +| `LinkListenerToDNSRecord` | Listener | DNSRecord | DNS capability | +| `LinkDNSPolicyToDNSRecord` | DNSPolicy | DNSRecord | Ownership | +| `LinkListenerToCertificate` | Listener | Certificate | TLS capability | +| `LinkTLSPolicyToIssuer` | TLSPolicy | Issuer | Issuer validation | +| `LinkTLSPolicyToClusterIssuer` | TLSPolicy | ClusterIssuer | Issuer validation | +| `LinkHTTPRouteRuleToAuthConfig` | HTTPRouteRule | AuthConfig | Auth capability | +| `LinkGatewayToWasmPlugin` | Gateway | WasmPlugin | Istio WASM injection | +| `LinkGatewayToEnvoyFilter` | Gateway | EnvoyFilter | Istio cluster config | +| `LinkGatewayToEnvoyPatchPolicy` | Gateway | EnvoyPatchPolicy | EG cluster config | +| `LinkGatewayToEnvoyExtensionPolicy` | Gateway | EnvoyExtensionPolicy | EG WASM injection | +| `LinkKuadrantToPeerAuthentication` | Kuadrant | PeerAuthentication | Istio mTLS | +| `LinkKuadrantToServiceMonitor` | Kuadrant | ServiceMonitor | Observability | +| `LinkKuadrantToPodMonitor` | Kuadrant | PodMonitor | Observability | + +### Conditional registration + +Links and watchers are registered conditionally based on installed CRDs: + +```go +// state_of_the_world.go — BootOptionsBuilder methods +getGatewayAPIOptions() // always required +getIstioOptions() // if Istio CRDs present +getEnvoyGatewayOptions() // if EnvoyGateway CRDs present +getCertManagerOptions() // if cert-manager CRDs present +getDNSOperatorOptions() // if dns-operator CRDs present +getLimitadorOperatorOptions() // if limitador-operator CRDs present +getAuthorinoOperatorOptions() // if authorino CRDs present +``` + +This allows the single operator binary to adapt to different cluster configurations. + +## Main Reconciliation Workflow + +``` +Event arrives + | + v +[Init Workflow] (Precondition) +├── EventLogger +└── TopologyReconciler (writes topology to ConfigMap) + | + v +[Parallel Tasks] +├── DNS Workflow +│ ├── Precondition: DNSPoliciesValidator +│ ├── Task: EffectiveDNSPoliciesReconciler +│ └── Postcondition: DNSPolicyStatusUpdater +│ +├── TLS Workflow +│ ├── Precondition: TLSPoliciesValidator +│ ├── Task: EffectiveTLSPoliciesReconciler +│ └── Postcondition: TLSPolicyStatusUpdater +│ +├── Data Plane Workflow +│ ├── Precondition: Validators (Auth, RateLimit, TokenRateLimit) +│ ├── Tasks: +│ │ ├── Precondition: Effective policy computation (Auth, RL, TokenRL) +│ │ ├── AuthConfigsReconciler +│ │ ├── LimitadorLimitsReconciler +│ │ ├── Provider-specific cluster + extension reconcilers +│ │ └── Istio integration reconcilers +│ └── Postcondition: Status updaters (Auth, RL, TokenRL) +│ +├── Observability Workflow +├── Developer Portal Workflow +└── Console Plugin Workflow + | + v +[Finalize Workflow] (Postcondition) +├── KuadrantStatusUpdater +├── PolicyMetricsReconciler +├── GatewayPolicyDiscoverabilityReconciler +└── ExtensionReconcilers +``` + +## State Passing Between Reconcilers + +Reconcilers communicate via `state *sync.Map`: + +| Key | Type | Producer | Consumer | +|-----|------|----------|----------| +| `StateEffectiveAuthPolicies` | `map[string]EffectiveAuthPolicy` | EffectiveAuthPolicyReconciler | AuthConfigsReconciler, ExtensionReconcilers | +| `StateEffectiveRateLimitPolicies` | `map[string]EffectiveRateLimitPolicy` | EffectiveRateLimitPolicyReconciler | LimitadorLimitsReconciler, ExtensionReconcilers | +| `StateEffectiveTokenRateLimitPolicies` | `map[string]EffectiveTokenRateLimitPolicy` | EffectiveTokenRateLimitPolicyReconciler | LimitadorLimitsReconciler, ExtensionReconcilers | +| `StateDNSPolicyAcceptedKey` | `map[string]error` | DNSPoliciesValidator | EffectiveDNSPoliciesReconciler | +| `StateTLSPolicyAcceptedKey` | `map[string]error` | TLSPoliciesValidator | EffectiveTLSPoliciesReconciler | +| `StateAuthPolicyValid` | `map[string]error` | AuthPolicyValidator | EffectiveAuthPolicyReconciler | +| `StateRateLimitPolicyValid` | `map[string]error` | RateLimitPolicyValidator | EffectiveRateLimitPolicyReconciler | +| `StateLimitadorLimitsModified` | `bool` | LimitadorLimitsReconciler | (status reporting) | + +## Key Source Files + +### policy-machinery +- Core interfaces: [`machinery/types.go`](https://github.com/Kuadrant/policy-machinery/blob/main/machinery/types.go) +- Topology builder: [`machinery/topology.go`](https://github.com/Kuadrant/policy-machinery/blob/main/machinery/topology.go) +- Gateway API topology: [`machinery/gateway_api_topology.go`](https://github.com/Kuadrant/policy-machinery/blob/main/machinery/gateway_api_topology.go) +- Gateway API types: [`machinery/gateway_api_types.go`](https://github.com/Kuadrant/policy-machinery/blob/main/machinery/gateway_api_types.go) +- Core K8s types: [`machinery/core_types.go`](https://github.com/Kuadrant/policy-machinery/blob/main/machinery/core_types.go) +- Controller: [`controller/controller.go`](https://github.com/Kuadrant/policy-machinery/blob/main/controller/controller.go) +- Workflow: [`controller/workflow.go`](https://github.com/Kuadrant/policy-machinery/blob/main/controller/workflow.go) +- Subscription: [`controller/subscriber.go`](https://github.com/Kuadrant/policy-machinery/blob/main/controller/subscriber.go) +- Events: [`controller/events.go`](https://github.com/Kuadrant/policy-machinery/blob/main/controller/events.go) +- Cache: [`controller/cache.go`](https://github.com/Kuadrant/policy-machinery/blob/main/controller/cache.go) +- Topology builder (controller): [`controller/topology_builder.go`](https://github.com/Kuadrant/policy-machinery/blob/main/controller/topology_builder.go) +- Runnable/watcher: [`controller/runnable.go`](https://github.com/Kuadrant/policy-machinery/blob/main/controller/runnable.go) + +### kuadrant-operator +- State of the world (wiring): [`internal/controller/state_of_the_world.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/state_of_the_world.go) +- Merge strategies: [`api/v1/merge_strategies.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/api/v1/merge_strategies.go) +- Topology links (Kuadrant root): [`api/v1beta1/topology.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/api/v1beta1/topology.go) +- Data plane workflow: [`internal/controller/data_plane_policies_workflow.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/data_plane_policies_workflow.go) +- DNS workflow: [`internal/controller/dns_workflow.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/dns_workflow.go) +- TLS workflow: [`internal/controller/tls_workflow.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/tls_workflow.go) +- Effective auth policies: [`internal/controller/effective_auth_policies_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/effective_auth_policies_reconciler.go) +- Effective RL policies: [`internal/controller/effective_ratelimit_policies_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/effective_ratelimit_policies_reconciler.go) +- Effective token RL policies: [`internal/controller/effective_tokenratelimit_policies_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/effective_tokenratelimit_policies_reconciler.go) +- Effective DNS policies: [`internal/controller/effective_dnspolicies_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/effective_dnspolicies_reconciler.go) +- Effective TLS policies: [`internal/controller/effective_tls_policies_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/effective_tls_policies_reconciler.go) +- Istio utils (links): [`internal/istio/utils.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/istio/utils.go) +- EnvoyGateway utils (links): [`internal/envoygateway/utils.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/envoygateway/utils.go) +- Authorino utils (links): [`internal/authorino/utils.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/authorino/utils.go) diff --git a/docs/specs/capabilities/rate-limiting.md b/docs/specs/capabilities/rate-limiting.md new file mode 100644 index 0000000..7d90b54 --- /dev/null +++ b/docs/specs/capabilities/rate-limiting.md @@ -0,0 +1,286 @@ +# Rate Limiting + +## Components + +| Component | Repo | Language | Role | +|-----------|------|----------|------| +| RateLimitPolicy CRD | kuadrant-operator | Go | User-facing policy (v1) attached to Gateway/HTTPRoute | +| TokenRateLimitPolicy CRD | kuadrant-operator | Go | Token-based variant (v1alpha1) for AI/LLM workloads | +| Limitador | limitador | Rust | Rate-limiting service with counter storage | +| Limitador Operator | limitador-operator | Go | Deploys/configures Limitador instances | +| wasm-shim | wasm-shim | Rust | Envoy WASM filter that extracts descriptors and calls Limitador | + +## Data Flow + +``` +User creates RateLimitPolicy (kuadrant.io/v1) + | + v +kuadrant-operator reconciles + | + +---> LimitadorLimitsReconciler + | writes limit definitions to Limitador CR spec.limits + | limitador-operator syncs to ConfigMap -> mounted into Limitador pods + | + +---> IstioExtensionReconciler / EnvoyGatewayExtensionReconciler + builds ActionSets with CEL predicates + descriptor keys + writes to WasmPlugin CR (Istio) or EnvoyExtensionPolicy (EG) + wasm-shim parses config at filter init + +Request arrives at Envoy + | + v +wasm-shim evaluates ActionSet predicates (hostname, path, method, headers) + | + v +wasm-shim evaluates Action predicates (limit-level When conditions) + | + v +wasm-shim builds gRPC RateLimitRequest (scope + descriptors + hits_addend) + | + v +Limitador receives request + - matches descriptors to limits by namespace + condition predicates + - resolves counter variables to partition counters (e.g. per-user) + - checks counter against max_value/seconds window + | + v +Response: OK (200) or OverLimit (429) +``` + +### TokenRateLimitPolicy (dual-phase) + +``` +Request phase: + wasm-shim -> Limitador CheckRateLimit (hits_addend=0, check only) + If over limit -> reject immediately (429) + +Response phase: + wasm-shim -> Limitador Report (hits_addend = responseBodyJSON("/usage/total_tokens")) + Increments counter with actual token consumption from LLM response body +``` + +## CRD Spec Structure + +### RateLimitPolicy (`kuadrant.io/v1`) + +```yaml +apiVersion: kuadrant.io/v1 +kind: RateLimitPolicy +metadata: + name: my-rlp +spec: + targetRef: + group: gateway.networking.k8s.io + kind: HTTPRoute # or Gateway + name: my-route + + # Top-level conditions (CEL, all must be true) + when: + - predicate: "request.method == 'POST'" + + # Limit definitions (map keyed by unique name) + limits: + per-user-limit: + rates: + - limit: 100 # max requests + window: 1m # time window (Gateway API duration: h/m/s/ms) + counters: + - expression: "auth.identity.username" # CEL: partition counter per user + when: + - predicate: "request.path.startsWith('/api')" # limit-level condition + + global-limit: + rates: + - limit: 1000 + window: 1h + + # OR use defaults/overrides for policy hierarchy + # (mutually exclusive with bare limits) + defaults: + strategy: atomic # "atomic" (default) or "merge" + limits: { ... } + overrides: + strategy: atomic + limits: { ... } +``` + +### TokenRateLimitPolicy (`kuadrant.io/v1alpha1`) + +Same structure as RateLimitPolicy but: +- Limits use token-based counting (dual-phase check/report) +- `hits_addend` derived from response body (`responseBodyJSON("/usage/total_tokens")`) + +## Policy Merging + +Policies attach at Gateway or HTTPRoute level. When multiple policies apply to the same request path, they are merged from most specific to least specific. + +### Strategies + +| Strategy | Defaults behavior | Overrides behavior | +|----------|-------------------|-------------------| +| `atomic` | Target wins if non-empty, else source | Source always wins | +| `merge` | Target limits + source limits not in target | Source limits + target limits not in source | + +### Hierarchy + +``` +Gateway-level RateLimitPolicy (least specific) + | + v merge/override +HTTPRoute-level RateLimitPolicy (most specific) + | + v +Effective policy (what gets reconciled) +``` + +Each limit rule carries a `source` field tracking which policy contributed it (format: `kind/namespace/name`). + +## Cross-Repo Contracts + +### kuadrant-operator -> Limitador (via limitador-operator) + +kuadrant-operator writes to `Limitador.Spec.Limits`: + +```yaml +# Limitador CR spec.limits entry +- namespace: "default/my-route" # / + name: per-user-limit # limit name from RateLimitPolicy + maxValue: 100 + seconds: 60 + conditions: + - "descriptors[0]['limit.per_user_limit__a1b2c3d4'] == '1'" + variables: + - "descriptors[0]['auth.identity.username']" +``` + +limitador-operator syncs these to a ConfigMap mounted at `/home/limitador/etc/limitador-config.yaml`. + +### kuadrant-operator -> wasm-shim (via WasmPlugin CR) + +kuadrant-operator writes ActionSets into WasmPlugin `spec.pluginConfig`: + +```json +{ + "actionSets": [{ + "name": "route-rule-1", + "routeRuleConditions": { + "hostnames": ["api.example.com"], + "predicates": ["request.url_path.startsWith('/api')"] + }, + "actions": [{ + "service": "ratelimit-service", + "scope": "default/my-route", + "conditionalData": [{ + "predicates": ["request.method == 'POST'"], + "data": [ + { "key": "limit.per_user_limit__a1b2c3d4", "value": "1" }, + { "key": "auth.identity.username", "value": "auth.identity.username" } + ] + }] + }] + }] +} +``` + +### wasm-shim -> Limitador (gRPC) + +wasm-shim sends `RateLimitRequest`: + +```protobuf +RateLimitRequest { + domain: "default/my-route" // matches Limitador limit namespace + descriptors: [{ + entries: [ + { key: "limit.per_user_limit__a1b2c3d4", value: "1" }, + { key: "auth.identity.username", value: "alice" } + ] + }] + hits_addend: 1 +} +``` + +### Critical invariant + +The **descriptor key** in wasm-shim config (`limit.per_user_limit__a1b2c3d4`) MUST match the **condition** in Limitador limits (`descriptors[0]['limit.per_user_limit__a1b2c3d4'] == '1'`). Both are generated by kuadrant-operator using the same `LimitNameToLimitadorIdentifier()` function. + +**Identifier format**: `limit.__<4-byte-SHA256-hex>` +- Sanitized: only alphanumeric and `_` +- Hash input: `//` +- TokenRateLimitPolicy uses prefix `tokenlimit.` instead of `limit.` + +## Limitador Internals + +### Limit matching algorithm + +1. Filter limits by namespace (= `scope` from gRPC request) +2. For each limit, evaluate all `conditions` as CEL predicates against the `descriptors` context +3. For each limit, resolve all `variables` — if any variable expression returns null, the limit does not apply +4. Matching limits produce Counters keyed by: `(limit_id, resolved_variable_values)` +5. First counter exceeding `max_value` in its `seconds` window -> `OverLimit` + +### Counter identity + +A counter is uniquely identified by: +- The limit it belongs to (namespace + conditions + seconds + variables) +- The resolved variable values (e.g., `user_id=alice`) + +### Storage backends + +| Backend | Use case | Flag | +|---------|----------|------| +| `memory` | Dev/test, single replica | (default) | +| `disk` | Persistent, single replica only | `disk` | +| `redis` | Production, multi-replica | `redis ` | +| `redis_cached` | Production, high throughput | `redis_cached ` | + +### gRPC services + +| Service | Method | Purpose | +|---------|--------|---------| +| Kuadrant RLS | `CheckRateLimit` | Check + increment counter | +| Kuadrant RLS | `Report` | Increment counter only (fire-and-forget) | +| Envoy RLS v3 | `ShouldRateLimit` | Standard Envoy compatibility | + +Proto: `limitador-server/proto/kuadrantrls.proto` + +## WASM Service Names + +| Service name | Used by | Calls | +|--------------|---------|-------| +| `ratelimit-service` | RateLimitPolicy | Limitador `CheckRateLimit` | +| `ratelimit-check-service` | TokenRateLimitPolicy (request phase) | Limitador `CheckRateLimit` | +| `ratelimit-report-service` | TokenRateLimitPolicy (response phase) | Limitador `Report` | + +## Key Source Files + +### kuadrant-operator +- CRD types: [`api/v1/ratelimitpolicy_types.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/api/v1/ratelimitpolicy_types.go) +- Token CRD types: [`api/v1alpha1/tokenratelimitpolicy_types.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/api/v1alpha1/tokenratelimitpolicy_types.go) +- Merge strategies: [`api/v1/merge_strategies.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/api/v1/merge_strategies.go) +- Effective policy calc: [`internal/controller/effective_ratelimit_policies_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/effective_ratelimit_policies_reconciler.go) +- Limitador limits reconciler: [`internal/controller/limitador_limits_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/limitador_limits_reconciler.go) +- WASM action building: [`internal/controller/ratelimit_workflow_helpers.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/ratelimit_workflow_helpers.go) +- WASM types: [`internal/wasm/types.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/wasm/types.go) +- Rate limit index: [`internal/ratelimit/index.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/ratelimit/index.go) + +### limitador +- Limit struct: [`limitador/src/limit.rs`](https://github.com/Kuadrant/limitador/blob/main/limitador/src/limit.rs) +- CEL evaluation: [`limitador/src/limit/cel.rs`](https://github.com/Kuadrant/limitador/blob/main/limitador/src/limit/cel.rs) +- Counter struct: [`limitador/src/counter.rs`](https://github.com/Kuadrant/limitador/blob/main/limitador/src/counter.rs) +- Storage keys: [`limitador/src/storage/keys.rs`](https://github.com/Kuadrant/limitador/blob/main/limitador/src/storage/keys.rs) +- Core rate limiter: [`limitador/src/lib.rs`](https://github.com/Kuadrant/limitador/blob/main/limitador/src/lib.rs) +- Kuadrant gRPC service: [`limitador-server/src/envoy_rls/kuadrant_service.rs`](https://github.com/Kuadrant/limitador/blob/main/limitador-server/src/envoy_rls/kuadrant_service.rs) +- Proto definition: [`limitador-server/proto/kuadrantrls.proto`](https://github.com/Kuadrant/limitador/blob/main/limitador-server/proto/kuadrantrls.proto) +- Example limits: [`limitador-server/examples/limits.yaml`](https://github.com/Kuadrant/limitador/blob/main/limitador-server/examples/limits.yaml) + +### limitador-operator +- CRD types: [`api/v1alpha1/limitador_types.go`](https://github.com/Kuadrant/limitador-operator/blob/main/api/v1alpha1/limitador_types.go) +- Deployment options: [`pkg/limitador/deployment_options.go`](https://github.com/Kuadrant/limitador-operator/blob/main/pkg/limitador/deployment_options.go) +- K8s object builders: [`pkg/limitador/k8s_objects.go`](https://github.com/Kuadrant/limitador-operator/blob/main/pkg/limitador/k8s_objects.go) +- Controller: [`controllers/limitador_controller.go`](https://github.com/Kuadrant/limitador-operator/blob/main/controllers/limitador_controller.go) + +### wasm-shim +- Rate limit task: [`src/kuadrant/pipeline/tasks/ratelimit.rs`](https://github.com/Kuadrant/wasm-shim/blob/main/src/kuadrant/pipeline/tasks/ratelimit.rs) +- Rate limit service client: [`src/services/rate_limit.rs`](https://github.com/Kuadrant/wasm-shim/blob/main/src/services/rate_limit.rs) +- Config parsing: [`src/configuration.rs`](https://github.com/Kuadrant/wasm-shim/blob/main/src/configuration.rs) diff --git a/docs/specs/capabilities/tls.md b/docs/specs/capabilities/tls.md new file mode 100644 index 0000000..83b4cc6 --- /dev/null +++ b/docs/specs/capabilities/tls.md @@ -0,0 +1,193 @@ +# TLS + +## Components + +| Component | Repo | Language | Role | +|-----------|------|----------|------| +| TLSPolicy CRD | kuadrant-operator | Go | User-facing policy (v1) attached to Gateway | +| cert-manager | (external) | Go | Issues and renews TLS certificates | + +TLS is simpler than other capabilities — kuadrant-operator translates TLSPolicy directly into cert-manager Certificate CRs. No additional Kuadrant components involved. + +## Data Flow + +``` +User creates TLSPolicy (kuadrant.io/v1) + targets a Gateway (optionally a specific Listener via sectionName) + | + v +kuadrant-operator reconciles + | + +---> TLSPoliciesValidator + | validates target, checks for conflicts + | verifies cert-manager Issuer/ClusterIssuer exists + | + +---> EffectiveTLSPoliciesReconciler + | for each targeted listener with TLS termination: + | validates listener has hostname, TLS config, certificateRefs + | creates cert-manager Certificate CR + | Certificate DNSNames = listener hostname + | Certificate secretName = listener certificateRef + | + +---> TLSPolicyStatusUpdater + checks Issuer readiness + Certificate readiness + sets Accepted + Enforced conditions + +cert-manager reconciles Certificate + | + v +Issues certificate from Issuer/ClusterIssuer +Stores TLS cert+key in Kubernetes Secret +Gateway controller picks up Secret for TLS termination +``` + +## CRD Spec Structure + +### TLSPolicy (`kuadrant.io/v1`) + +```yaml +apiVersion: kuadrant.io/v1 +kind: TLSPolicy +metadata: + name: my-tls +spec: + targetRef: + group: gateway.networking.k8s.io + kind: Gateway # Gateway only (not HTTPRoute) + name: prod-gateway + sectionName: https # optional: target specific listener + + # Issuer reference (required) + issuerRef: + name: letsencrypt-prod + kind: ClusterIssuer # "Issuer" (namespaced) or "ClusterIssuer" + # If kind omitted or "Issuer", must be in same namespace as TLSPolicy + + # Certificate options (all optional) + commonName: "api.example.com" # X.509 CN (max 64 chars) + duration: 2160h # certificate lifetime (default 90 days) + renewBefore: 720h # renew this long before expiry (default 2/3 of duration, min 5m) + revisionHistoryLimit: 3 # max CertificateRequest revisions kept (≥1 or nil) + + usages: # X.509 key usages (default: digital signature + key encipherment) + - digital signature + - key encipherment + + privateKey: # private key configuration + algorithm: RSA # RSA, ECDSA, or Ed25519 + encoding: PKCS1 # PKCS1 or PKCS8 + size: 2048 # key size in bits + rotationPolicy: Always # Never or Always +``` + +**Note**: TLSPolicy does NOT support defaults/overrides — it attaches directly to Gateway listeners. No policy merging hierarchy. + +## Listener Requirements + +TLSPolicy only creates Certificates for listeners that meet ALL conditions: + +| Requirement | Field | Value | +|-------------|-------|-------| +| TLS configured | `listener.tls` | Must be non-nil | +| TLS mode | `listener.tls.mode` | Must be `Terminate` | +| Certificate refs | `listener.tls.certificateRefs` | At least one ref | +| Ref group | `certificateRef.group` | `""` or `"core"` (K8s Secret) | +| Ref kind | `certificateRef.kind` | `""` or `"Secret"` | +| Ref namespace | `certificateRef.namespace` | Same as Gateway (no cross-namespace) | +| Hostname | `listener.hostname` | Should be set (falls back to `*`) | + +**Shared secrets not supported**: Multiple listeners referencing the same TLS Secret will cause an `IncorrectCertificate` error. Each listener needs its own Secret. + +## Cross-Repo Contract + +### kuadrant-operator -> cert-manager (via Certificate CR) + +kuadrant-operator creates cert-manager Certificate CRs: + +- **Name**: `-` (e.g., `prod-gateway-https`) +- **Namespace**: Same as the certificateRef Secret namespace +- **Owner**: TLSPolicy (via OwnerReference) +- **Labels**: Kuadrant operator common labels + +Generated Certificate spec: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: prod-gateway-https + ownerReferences: + - kind: TLSPolicy +spec: + dnsNames: + - api.example.com # from listener.hostname + secretName: api-example-tls # from listener.tls.certificateRefs[0].name + issuerRef: + name: letsencrypt-prod # from TLSPolicy + kind: ClusterIssuer + commonName: api.example.com # from TLSPolicy (optional) + duration: 2160h # from TLSPolicy (optional) + renewBefore: 720h # from TLSPolicy (optional) + usages: # from TLSPolicy (default: digital signature + key encipherment) + - digital signature + - key encipherment + privateKey: # from TLSPolicy (optional) + algorithm: RSA + encoding: PKCS1 + size: 2048 + rotationPolicy: Always + revisionHistoryLimit: 3 # from TLSPolicy (optional) +``` + +## Validation + +### Preconditions + +1. **Gateway API installed** — CRD must be present +2. **cert-manager installed** — CRD must be present +3. **Target found** — Gateway (and optional sectionName) must exist in topology +4. **No conflicts** — Only one TLSPolicy per targetRef (oldest wins) +5. **Issuer exists** — Referenced Issuer/ClusterIssuer must exist + +### Conflict resolution + +When multiple TLSPolicies target the same Gateway (or same listener), the **oldest policy** (by creation timestamp) wins. Newer policies get `Accepted=False` with reason `Conflicting`. + +## TLSPolicy Status + +### Conditions + +| Condition | True | False | +|-----------|------|-------| +| `Accepted` | Policy validated, target found, no conflicts | Target not found, issuer missing, dependency missing, conflict | +| `Enforced` | Issuer ready AND all Certificates ready | Issuer not ready, certificate missing or not ready | + +`Enforced` is only evaluated when `Accepted` is true. + +### Enforcement checks + +1. **Issuer readiness**: Looks up Issuer/ClusterIssuer, checks for `Ready=True` condition +2. **Certificate readiness**: For each listener, looks up expected Certificate, checks for `Ready=True` condition + +## Workflow Position + +TLS runs as a parallel task alongside DNS and data-plane (auth/ratelimit) workflows: + +``` +Main reconciliation loop + ├── DNS workflow + ├── TLS workflow ← runs in parallel + ├── Data plane workflow (auth, rate limiting) + ├── Observability + └── Developer portal +``` + +## Key Source Files + +### kuadrant-operator +- CRD types: [`api/v1/tlspolicy_types.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/api/v1/tlspolicy_types.go) +- TLS workflow: [`internal/controller/tls_workflow.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/tls_workflow.go) +- Validator: [`internal/controller/tlspolicies_validator.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/tlspolicies_validator.go) +- Certificate reconciler: [`internal/controller/effective_tls_policies_reconciler.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/effective_tls_policies_reconciler.go) +- Status updater: [`internal/controller/tlspolicy_status_updater.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/tlspolicy_status_updater.go) +- State of the world: [`internal/controller/state_of_the_world.go`](https://github.com/Kuadrant/kuadrant-operator/blob/main/internal/controller/state_of_the_world.go) diff --git a/docs/specs/repo-map.md b/docs/specs/repo-map.md new file mode 100644 index 0000000..634d901 --- /dev/null +++ b/docs/specs/repo-map.md @@ -0,0 +1,270 @@ +# Kuadrant Repo Map + +Cross-repository dependency map for the Kuadrant project. Describes what each repo owns, how repos relate, and the contracts between them. + +## Dependency Flow + +``` +gateway-api (external) + | +policy-machinery (library) + | +kuadrant-operator (orchestrator) + | + +-- authorino-operator --> authorino + +-- limitador-operator --> limitador + +-- dns-operator + +-- wasm-shim (injected as Envoy WASM filter) + +-- cert-manager (external) + +-- console-plugin (OpenShift UI) +``` + +## Repositories + +### kuadrant-operator +**Role**: Central orchestrator. Translates user-facing policy CRDs into component-specific resources. + +**Defines CRDs** (`kuadrant.io`): +| Kind | API Version | Targets | +|------|-------------|---------| +| AuthPolicy | v1 | Gateway, HTTPRoute | +| RateLimitPolicy | v1 | Gateway, HTTPRoute | +| DNSPolicy | v1 | Gateway | +| TLSPolicy | v1 | Gateway | +| TokenRateLimitPolicy | v1alpha1 | Gateway, HTTPRoute | +| Kuadrant | v1beta1 | (cluster config) | + +**Consumes CRDs**: +| Kind | API Group | From Repo | +|------|-----------|-----------| +| AuthConfig | authorino.kuadrant.io/v1beta3 | authorino | +| Limitador | limitador.kuadrant.io/v1alpha1 | limitador-operator | +| DNSRecord | kuadrant.io/v1alpha1 | dns-operator | +| Certificate | cert-manager.io/v1 | cert-manager (external) | +| Gateway, HTTPRoute | gateway.networking.k8s.io/v1 | gateway-api (external) | +| WasmPlugin | extensions.istio.io | istio (external) | + +**Go module imports**: +| Module | Version | +|--------|---------| +| github.com/kuadrant/authorino | v0.22.0 | +| github.com/kuadrant/authorino-operator | v0.21.0 | +| github.com/kuadrant/limitador-operator | v0.15.0 | +| github.com/kuadrant/dns-operator | (commit-pinned) | +| github.com/kuadrant/policy-machinery | v0.7.1 | + +**Key controllers**: +- AuthConfigsReconciler: AuthPolicy -> AuthConfig +- LimitadorLimitsReconciler: RateLimitPolicy -> Limitador limits (via ConfigMap) +- IstioExtensionReconciler: Policies -> WasmPlugin (Istio) +- EnvoyGatewayExtensionReconciler: Policies -> EnvoyGateway extensions +- DNSPolicyReconciler: DNSPolicy -> DNSRecord +- TLSPolicyReconciler: TLSPolicy -> cert-manager Certificate + +--- + +### authorino +**Role**: Authorization service. Envoy ext_authz gRPC server. + +**Defines CRDs** (`authorino.kuadrant.io`): +| Kind | API Version | +|------|-------------| +| AuthConfig | v1beta3 (storage), v1beta2 (legacy) | + +**Protocol**: Envoy `ext_authz` gRPC on port 50051, raw HTTP on port 5001. + +**Auth methods**: API key, JWT/OIDC, OAuth2 introspection, K8s TokenReview, x509 mTLS, OPA/Rego, SpiceDB, pattern matching. + +**Go module imports**: No Kuadrant dependencies. Imports envoyproxy/go-control-plane, OPA, SpiceDB SDKs. + +--- + +### authorino-operator +**Role**: Deploys and manages Authorino instances. + +**Defines CRDs** (`operator.authorino.kuadrant.io`): +| Kind | API Version | +|------|-------------| +| Authorino | v1beta1 | + +**Manages**: Deployment, Services (auth/OIDC/metrics), ServiceAccount, ClusterRole/RoleBinding for Authorino. + +**Go module imports**: No Kuadrant dependencies. Manages Authorino purely as a container image. + +--- + +### limitador +**Role**: Rate-limiting service (Rust). Implements Envoy RLS protocol. + +**gRPC API** (port 8081): +- `CheckRateLimit` / `Report` (Kuadrant RLS extension: `kuadrantrls.proto`) +- Standard Envoy `envoy.service.ratelimit.v3.RateLimitService` + +**HTTP API** (port 8080): +- `/check_and_report`, `/check`, `/report` (rate limit operations) +- `/limits/{namespace}`, `/counters/{namespace}` (inspection) +- `/status`, `/metrics` + +**Config format** (YAML, hot-reloadable): +```yaml +- namespace: example.org + max_value: 100 + seconds: 60 + conditions: ["descriptors[0]['req.method'] == 'POST'"] + variables: ["descriptors[0]['user_id']"] +``` + +**Storage backends**: memory, disk, redis, redis_cached. + +--- + +### limitador-operator +**Role**: Deploys and manages Limitador instances. + +**Defines CRDs** (`limitador.kuadrant.io`): +| Kind | API Version | +|------|-------------| +| Limitador | v1alpha1 | + +**Manages**: Deployment, Service (headless), ConfigMap (limits YAML), PVC (disk storage), PDB. + +**Translates**: CRD spec -> Limitador CLI args + mounted ConfigMap at `/home/limitador/etc/limitador-config.yaml`. + +**Go module imports**: No Kuadrant dependencies. Manages Limitador purely as a container image. + +--- + +### dns-operator +**Role**: Manages DNS records across cloud providers (AWS Route53, Google Cloud DNS, Azure DNS, CoreDNS). + +**Defines CRDs** (`kuadrant.io`): +| Kind | API Version | +|------|-------------| +| DNSRecord | v1alpha1 | +| DNSHealthCheckProbe | v1alpha1 | + +**Features**: Multi-cluster delegation, health-aware DNS, TXT registry for ownership. + +**Go module imports**: `sigs.k8s.io/external-dns` (Kuadrant fork), `sigs.k8s.io/multicluster-runtime`. + +--- + +### policy-machinery +**Role**: Shared Go library. Provides the policy attachment framework used by kuadrant-operator. + +**Exports**: +- `Policy` interface (policies that target objects and merge) +- `Targetable` interface (Gateways, HTTPRoutes) +- `MergeStrategy` function type +- Topology builder (DAG of Gateway -> HTTPRoute -> Policy relationships) + +**Consumed by**: kuadrant-operator only. + +**Go module imports**: `sigs.k8s.io/gateway-api` v1.2.1, controller-runtime. + +--- + +### wasm-shim +**Role**: Proxy-Wasm Rust module (compiled to WASM). Deployed as an Envoy filter via Istio WasmPlugin or EnvoyGateway extension. + +**Config format** (JSON, injected by kuadrant-operator): +```json +{ + "actionSets": [{ + "name": "...", + "routeRuleConditions": { + "hostnames": ["api.example.com"], + "predicates": ["request.method == 'GET'"] + }, + "actions": [{ + "service": "ratelimit-service", + "scope": "my-namespace/my-rlp", + "predicates": ["..."], + "conditionalData": [{"key": "...", "expression": "..."}] + }] + }] +} +``` + +**Dispatches gRPC to**: +- limitador: `CheckRateLimit` / `Report` (Kuadrant RLS) +- authorino: Envoy `ext_authz` `Check` + +**CEL predicates**: Evaluated per-request to determine which actions fire. + +--- + +### console-plugin +**Role**: OpenShift Console dynamic plugin (React/TypeScript). Provides UI for Kuadrant resources. + +**Displays CRDs**: +- Gateway, HTTPRoute (gateway-api) +- AuthPolicy, RateLimitPolicy, DNSPolicy, TLSPolicy (kuadrant.io/v1) +- PlanPolicy, APIProduct, APIKeyRequest (extensions.kuadrant.io/v1alpha1) + +**Features**: Policy topology visualization, form/YAML editors, gateway health dashboard. + +--- + +### architecture +**Role**: RFCs and design documents. Central coordination for cross-repo architecture decisions. + +**Key RFCs**: Policy Machinery (0011), RLP v2 (0001), DNSPolicy (0003/0005), mTLS (0012), DNS Failover (0014). + +--- + +## Cross-Repo Contracts + +### kuadrant-operator <-> authorino +- **Object**: `AuthConfig` CR (authorino.kuadrant.io/v1beta3) +- **Producer**: kuadrant-operator (AuthConfigsReconciler) +- **Consumer**: authorino (watches AuthConfig, builds in-memory index) +- **Contract**: kuadrant-operator creates AuthConfig CRs with host matching, auth rules, and metadata. Authorino enforces them on ext_authz requests. + +### kuadrant-operator <-> limitador-operator +- **Object**: Limitador CR (limitador.kuadrant.io/v1alpha1) + limits ConfigMap +- **Producer**: kuadrant-operator (LimitadorLimitsReconciler) +- **Consumer**: limitador-operator (reconciles Deployment + ConfigMap) +- **Contract**: kuadrant-operator writes rate limit definitions to the Limitador CR's `spec.limits`. limitador-operator syncs these to a ConfigMap mounted into Limitador pods. + +### kuadrant-operator <-> wasm-shim +- **Object**: WasmPlugin CR (extensions.istio.io) containing JSON config +- **Producer**: kuadrant-operator (IstioExtensionReconciler) +- **Consumer**: wasm-shim (parses JSON config at filter init) +- **Contract**: kuadrant-operator serializes ActionSets with CEL predicates, service references, and conditional data into the WasmPlugin's `spec.pluginConfig`. wasm-shim uses these to route auth/ratelimit calls. +- **Invariant**: Descriptor keys in wasm-shim config MUST match limit definitions written to Limitador. Scope values MUST match between wasm-shim actions and Limitador limit namespaces. + +### kuadrant-operator <-> dns-operator +- **Object**: `DNSRecord` CR (kuadrant.io/v1alpha1) +- **Producer**: kuadrant-operator (DNSPolicyReconciler) +- **Consumer**: dns-operator (reconciles DNS records with cloud providers) +- **Contract**: kuadrant-operator creates DNSRecord CRs with endpoints, provider references, and health check config. dns-operator syncs these to the configured DNS provider. + +### kuadrant-operator <-> cert-manager +- **Object**: `Certificate` CR (cert-manager.io/v1) +- **Producer**: kuadrant-operator (TLSPolicyReconciler) +- **Consumer**: cert-manager (issues/renews TLS certificates) +- **Contract**: kuadrant-operator creates Certificate CRs referencing an Issuer/ClusterIssuer. cert-manager provisions certificates and stores them in Secrets. + +### wasm-shim <-> limitador +- **Protocol**: gRPC (Kuadrant RLS extension) +- **Service**: `kuadrant.service.ratelimit.v1.RateLimitService` +- **Methods**: `CheckRateLimit`, `Report` +- **Data**: Scope + descriptors (key-value pairs from CEL expressions) + hits addend + +### wasm-shim <-> authorino +- **Protocol**: gRPC (Envoy ext_authz) +- **Service**: `envoy.service.auth.v3.Authorization` +- **Method**: `Check` +- **Data**: HTTP request attributes (method, path, headers, host) + +## Gateway Provider Integration + +kuadrant-operator supports two gateway providers, each with its own extension mechanism: + +| Provider | Extension CRD | Reconciler | wasm-shim Injection | +|----------|--------------|------------|---------------------| +| Istio | WasmPlugin (extensions.istio.io) | IstioExtensionReconciler | Via Istio WasmPlugin CR | +| Envoy Gateway | EnvoyExtensionPolicy | EnvoyGatewayExtensionReconciler | Via EG extension policy | + +Both paths produce equivalent wasm-shim configurations; only the delivery mechanism differs.