feat: OPA server + Authentication server

[Manuthor]

Summary

• Integrates Open Policy Agent (OPA) as an optional RBAC authorization layer with three modes: disabled (default), exclusive (OPA is the sole decision maker), and enforcing (both OPA and native KMS permissions must allow).
• Adds --opa-url / --opa-mode CLI flags (env vars KMS_OPA_URL, KMS_OPA_MODE) to point the KMS at an OPA sidecar.
• Extracts JWT roles and as_domain claims from incoming requests into a per-request OpaUserContext (now backed by tokio::task_local! for correct async isolation) and propagates them to OPA without threading through all call signatures.
• Adds a domain column to the objects table (SQLite, PostgreSQL, MySQL, Redis-findex) with automatic schema migration; objects are stamped with the creator's domain at creation time.
• Ships a reference kms.rego Rego policy with role-based, domain-scoped rules for CryptoOfficer, Auditor, DomainAdmin, and User roles; super_admin is intentionally decoupled from OPA (native KMS gate only).
• Restructures the authorization documentation into four focused pages (overview, mode 1 native, mode 2 exclusive OPA, mode 3 enforcing dual-gate) with sequence and flowchart diagrams.
• Adds 11 OPA integration test vectors (allowed and denied paths) covering all three modes, plus a setup_auth_server_for_opa() helper that provisions a live Cosmian auth server with five test users via REST.
• Fixes JWT middleware to accept tokens without an aud claim and to fall back to sub when email is absent, enabling compatibility with the Cosmian auth server.
• Aligns argon2 dev-dependency in test_kms_server to the workspace version (0.5) and adds the password-hash+alloc features needed for password hashing in test helpers.
• Fixes operation name casing in kms.rego to match KmipOperation::Display lowercase snake_case output (e.g. "get_attributes" not "GetAttributes").

[Copilot]

Pull request overview

This PR introduces an OPA (Open Policy Agent) sidecar integration to add JWT role/domain-based RBAC to the KMS authorization path, alongside a documentation restructuring and new integration test vectors to validate the three authorization modes (native-only, OPA-exclusive, OPA+native enforcing).

Changes:

• Add OPA client/config/input/context plumbing and wire it into user_has_permission() with exclusive and enforcing modes.
• Add domain stamping to objects (new domain column across DB backends + ObjectWithMetadata.domain) to support domain-scoped RBAC decisions.
• Add documentation + mkdocs navigation restructure for authorization modes, plus OPA integration test vectors and auth-server provisioning helpers.

Reviewed changes

Copilot reviewed 55 out of 56 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
OPA-middleware.md	Design/implementation plan and rationale for OPA RBAC integration and domain model.
documentation/mkdocs.yml	Updates nav to new authorization section structure (overview + mode pages).
documentation/docs/configuration/authorization/index.md	New authorization overview page (modes, role model, domain model, JWT claims, OPA input).
documentation/docs/configuration/authorization/mode1.md	New Mode 1 documentation for native KMS permission system.
documentation/docs/configuration/authorization/mode2.md	New Mode 2 documentation for exclusive OPA RBAC behavior and operations.
documentation/docs/configuration/authorization/mode3.md	New Mode 3 documentation for dual-gate (OPA then native KMS) behavior.
documentation/docs/configuration/authorization.md	Removes old single-page authorization doc in favor of the new multi-page structure.
docker-compose.yml	Adds an opa service to compose for local/integration usage with kms.rego.
crate/test_kms_server/src/vector_runner.rs	Adds JWT identity support and substantial OPA/auth-server provisioning + new OPA test vectors.
crate/test_kms_server/README.md	Documents OPA vectors and auth-server provisioning flow for local runs.
crate/test_kms_server/Cargo.toml	Adds dev-deps and reqwest features needed for auth-server provisioning in tests.
crate/server/src/tests/test_set_attribute.rs	Updates object creation calls to include the new domain parameter.
crate/server/src/tests/test_modify_attribute.rs	Updates object creation calls to include the new domain parameter.
crate/server/src/middlewares/mod.rs	Extends AuthenticatedUser to carry roles and domain.
crate/server/src/middlewares/tls_auth.rs	Populates AuthenticatedUser with empty roles/domain for mTLS auth.
crate/server/src/middlewares/jwt/jwt_config.rs	Adds JWT roles and as_domain (alias) parsing into UserClaim.
crate/server/src/middlewares/jwt/jwt_token_auth.rs	Uses email or sub as username; propagates roles/domain into AuthenticatedUser.
crate/server/src/middlewares/ensure_auth.rs	Ensures default user injection also sets empty roles/domain.
crate/server/src/middlewares/api_token/api_token_middleware.rs	Ensures API-token auth sets empty roles/domain.
crate/server/src/main.rs	Updates test config initialization to include default OpaConfig.
crate/server/src/core/mod.rs	Exposes new core::opa module.
crate/server/src/core/opa/mod.rs	Introduces OPA integration module exports (client/config/context/input).
crate/server/src/core/opa/client.rs	Adds reqwest-based OPA client with fail-closed behavior.
crate/server/src/core/opa/config.rs	Adds OpaMode and OpaParams configuration types.
crate/server/src/core/opa/context.rs	Adds per-request OPA user context storage (roles/domain).
crate/server/src/core/opa/input.rs	Adds OpaInput struct used as the OPA decision input document.
crate/server/src/core/kms/mod.rs	Stores optional opa_client on the KMS struct and initializes it when configured.
crate/server/src/core/kms/permissions.rs	Sets per-request OPA context as a side-effect of KMS::get_user().
crate/server/src/core/retrieve_object_utils.rs	Wires OPA decision into user_has_permission() with mode-dependent behavior.
crate/server/src/core/operations/create.rs	Stamps created objects with the creator’s domain (from OPA context).
crate/server/src/core/operations/derive_key.rs	Updates DB create call to include domain parameter.
crate/server/src/core/operations/key_ops/mod.rs	Updates ObjectWithMetadata::new test helpers for new domain field.
crate/server/src/config/command_line/mod.rs	Registers new CLI OPA config module.
crate/server/src/config/command_line/opa_config.rs	Adds --opa-url / --opa-mode CLI + env var config.
crate/server/src/config/command_line/clap_config.rs	Plumbs OpaConfig into the top-level clap config.
crate/server/src/config/params/server_params.rs	Converts CLI config into runtime opa_params and exposes it via debug output.
crate/interfaces/src/stores/objects_store.rs	Extends ObjectsStore::create() with a domain parameter.
crate/interfaces/src/stores/object_with_metadata.rs	Adds domain field + getter, and updates Display.
crate/interfaces/src/hsm/hsm_store.rs	Updates HSM object creation paths for new domain parameter (ignored).
crate/server_database/src/core/database_objects.rs	Plumbs domain through Database::create() to underlying stores.
crate/server_database/src/core/unwrapped_cache.rs	Updates tests for new domain parameter.
crate/server_database/src/tests/tagging_tests.rs	Updates tests for new domain parameter.
crate/server_database/src/tests/owner_test.rs	Updates tests for new domain parameter.
crate/server_database/src/tests/list_uids_for_tags_test.rs	Updates tests for new domain parameter.
crate/server_database/src/tests/json_access_test.rs	Updates tests for new domain parameter.
crate/server_database/src/tests/find_attributes_test.rs	Updates tests for new domain parameter.
crate/server_database/src/tests/database_tests.rs	Updates tests for new domain parameter.
crate/server_database/src/stores/sql/query.sql	Adds domain column and updates insert/select queries (non-MySQL SQL).
crate/server_database/src/stores/sql/query_mysql.sql	Adds domain column and updates insert/select queries (MySQL SQL).
crate/server_database/src/stores/sql/sqlite.rs	Adds sqlite migration and CRUD updates for domain column.
crate/server_database/src/stores/sql/pgsql.rs	Updates Postgres insert/select to include domain.
crate/server_database/src/stores/sql/mysql.rs	Updates MySQL insert/select to include domain.
crate/server_database/src/stores/redis/redis_with_findex.rs	Updates Redis-findex store signatures and object conversion for domain (currently empty).
CHANGELOG/rbac_rego.md	Adds a detailed changelog entry for the OPA RBAC feature branch.
Cargo.lock	Updates lockfile for new deps/features used by tests and OPA integration.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[serene-kitfisto-8899]

--opa-url and --opa-mode is confusing :
I can start the KMS with opa-mode as exclusive without opa-url :
$ cargo r --bin cosmian_kms -- --opa-mode exclusive
...
opa: OpaConfig {
opa_url: None,
opa_mode: "exclusive",
},
I think that opa_url should be mandatory when opa_mode is exclusive.
The KMS should not start, inmho.

[serene-kitfisto-8899]

🔍 OPA Integration — Testing session notes (2026-06-23)

OPA server setup

Start via docker-compose:

docker compose up -d opa

The docker-compose.yml runs openpolicyagent/opa:edge-static-debug on port 8181, mounting test_data/opa/kms.rego read-only.

⚠️ test_data/opa/kms.rego must be a file — if Docker auto-creates it as a directory (because it didn't exist), OPA loads no policy and returns {}. Fix: sudo rm -rf test_data/opa/kms.rego then restart.

Verify OPA is serving the policy:

curl -s http://localhost:8181/v1/policies   # should show kms package

Test OPA policy (use lowercase operation names — matches KmipOperation::to_string()):

# CryptoOfficer create → true
curl -s http://localhost:8181/v1/data/kms/allow -H "Content-Type: application/json" \
  -d '{"input":{"user":"alice","user_domain":"acme.com","roles":["CryptoOfficer"],"operation":"create","object_uid":"*","object_domain":"acme.com","is_owner":false}}'
# → {"result":true}

# User create → false (create not in user_ops)
curl -s http://localhost:8181/v1/data/kms/allow -H "Content-Type: application/json" \
  -d '{"input":{"user":"bob","user_domain":"acme.com","roles":["User"],"operation":"create","object_uid":"*","object_domain":"acme.com","is_owner":false}}'
# → {"result":false}

# Debug: why was it denied?
curl -s http://localhost:8181/v1/data/kms/reasons -H "Content-Type: application/json" -d '{...}'

Key gotcha: operation names are lowercase ("create", "get_attributes") — confirmed in crate/kmip/src/kmip_2_1/mod.rs. Sending "Create" (PascalCase) returns {"result":false}.

---

Running KMS with OPA

cargo run --bin cosmian_kms -- \
  --database-type sqlite --sqlite-path /tmp/kms-data \
  --opa-url http://localhost:8181 \
  --opa-mode enforcing

# For local dev without a JWT auth server, bypass OPA for admin:
cargo run --bin cosmian_kms -- \
  --database-type sqlite --sqlite-path /tmp/kms-data \
  --opa-url http://localhost:8181 \
  --opa-mode enforcing \
  --privileged-users admin

ckms CLI (--url is a global flag, must go before the subcommand):

cargo run --bin ckms -- server version
# → 5.23.0 (OpenSSL 3.6.2 7 Apr 2026-FIPS)

cargo run --bin ckms -- sym keys create   # uses KMS_DEFAULT_URL or ~/.cosmian/ckms.toml
export KMS_DEFAULT_URL=http://localhost:9998

---

🐛 Bug fixed: OPA fail-closed not enforced on create/import/register/create_key_pair

Root cause: In create.rs, import.rs, register.rs, create_key_pair.rs, the user_has_permission() call (which invokes OPA) was gated on if let Some(users) = privileged_users. When --privileged-users was not set, privileged_users was None, the entire block was skipped, and OPA was never consulted for object-creation operations — even in Enforcing/Exclusive mode.

Fix: OPA is now checked whenever kms.opa_client.is_some(), independent of privileged_users:

let opa_active = kms.opa_client.is_some();
if opa_active || privileged_users.is_some() {
    let has_permission = user_has_permission(owner, None, &KmipOperation::Create, kms).await?;
    let is_privileged = privileged_users.as_deref().is_some_and(|users| users.iter().any(|u| u == owner));
    if !has_permission && !is_privileged {
        kms_bail!(KmsError::Unauthorized("User does not have create access-right."))
    }
}

Files changed: create.rs, import.rs, register.rs, create_key_pair.rs

---

Rights Matrix — OPA Modes × Roles × Operations

Legend: ✅ Allowed · ❌ Denied · ⊕ Same-domain only · Owner override always ✅

Mode 1 — Disabled

OPA not called. Native KMS permissions only. All users have open access unless --privileged-users is set.

Modes 2 (Exclusive) and 3 (Enforcing) — OPA decision per role

Operation	SuperAdmin	DomainAdmin	CryptoOfficer	Auditor	User	[] no role
create / create_key_pair / import / register	✅	⊕	⊕	❌	❌	❌
get / export	✅	⊕	⊕	⊕ get only	❌	❌
locate / get_attributes	✅	⊕	⊕	⊕	⊕	❌
set/modify/delete/add_attribute	✅	⊕	⊕	❌	❌	❌
activate / revoke / archive / recover / destroy	✅	⊕	⊕	❌	❌	❌
rekey / rekey_key_pair	✅	⊕	⊕	❌	❌	❌
encrypt / decrypt / sign / verify	✅	⊕	❌	❌	⊕	❌
mac / derive_key	✅	⊕	❌	❌	⊕	❌
mac_verify	✅	⊕	❌	⊕	⊕	❌
list_access / query_access	✅	⊕	❌	⊕	❌	❌

OPA down → fail-closed: all requests denied in Modes 2 & 3 (unwrap_or(false)).

Mode 3 extra gate: after OPA allows, the native KMS grant system also runs — OPA allows the operation class, KMS checks per-object access rights.

---

Notes generated from a Copilot CLI debugging session.

[serene-kitfisto-8899]

⚠️ Known Limitation: TTLV Socket clients cannot use OPA RBAC

Root cause

The KMIP 2.1 wire protocol defines an Authentication structure inside every RequestMessage (§9.4) that can carry client credentials:

RequestMessage
  └─ RequestHeader
       └─ Authentication (OPTIONAL)
            └─ Credential
                 ├─ UsernameAndPassword { username, password }
                 ├─ Ticket { ticket_type, ticket_value }   ← could carry a JWT
                 └─ ...

The Cosmian KMS parses but does not use this field. Identity, roles, and domain are established exclusively from the HTTP transport layer (Authorization: Bearer <JWT> header or TLS client certificate CN).

Impact

Client type	Identity	roles	domain	OPA Modes 2/3
HTTPS + JWT Bearer	JWT sub	JWT roles	JWT as_domain	✅ Full RBAC
HTTPS + mTLS cert	Cert CN	[]	""	❌ Fail-closed
HTTPS + API token	Token id	[]	""	❌ Fail-closed
TCP socket (PyKMIP, Synology DSM)	Cert CN	[]	""	❌ Fail-closed
KMIP Authentication.Credential	Ignored	Ignored	Ignored	—

Socket-mode clients (PyKMIP, Synology DSM, PKCS#11) are denied all operations in Modes 2 and 3 unless they own the object.

Workarounds

1. Migrate to HTTPS /kmip endpoint — send binary TTLV as Content-Type: application/octet-stream with Authorization: Bearer <JWT>. Full spec-compliant RBAC, no code change needed server-side.

2. Use Mode 1 (Disabled) for socket clients — rely on network-level access control (firewall, VPN) and native KMS grants for object-level access.

Future improvement

Implement Credential.Ticket extraction from the KMIP Authentication header in the message body. This would allow socket clients to embed a JWT inside the KMIP protocol itself, closing the gap without requiring HTTP transport. The Ticket credential type (KMIP 2.1 Table 442) is designed for exactly this use case (Kerberos tokens, opaque security tokens).

Also noted: operation names are lowercase

Operation names sent to OPA in input.operation are lowercase snake_case ("create", "get_attributes") — not PascalCase as in the KMIP spec text. OPA policies must use lowercase or evaluation silently returns false.

---

Documented in documentation/docs/configuration/authorization/index.md § Known limitations.

[serene-kitfisto-8899]

📋 KMIP Profiles v2.1 — Compliance Analysis vs. OPA RBAC

Checked against KMIP Profiles v2.1 OS (https://docs.oasis-open.org/kmip/kmip-profiles/v2.1/os/kmip-profiles-v2.1-os.html).

---

Authentication Suites defined (§3)

KMIP Profiles v2.1 defines exactly two authentication suites:

Suite	Transport	Auth mechanism
3.1 Basic	TCP/TLS (port 5696)	Mutual TLS (client cert)
3.2 HTTPS	HTTP over TLS (RFC 2818)	Delegates to §3.1

Neither suite mentions JWT, Bearer tokens, or any HTTP-level authorization header. These are implementation extensions.

---

🔴 Compliance gap found — §3.1.3 Basic Authentication Client Authenticity

The spec says (verbatim):

"Conformant KMIP servers SHALL use the identity derived from the channel mutual authentication to determine the client identity if the KMIP client requests do not contain an Authentication object."

"Conformant KMIP servers SHALL use the identity derived from the Credential information to determine the client identity if the KMIP client requests contain an Authentication object."

Current behavior: Cosmian KMS ignores RequestMessage.Authentication.Credential entirely. Identity always comes from the HTTP Authorization: Bearer header or TLS CN — even when a client sends a Credential in the KMIP body.

This is a SHALL violation. When a KMIP client includes an Authentication structure in its request, the server MUST use that credential for identity determination.

---

No RBAC in KMIP Profiles either

§3.1.3 explicitly punts: "The exact mechanisms determining the client identity are outside the scope of this specification."

No profile in KMIP Profiles v2.1 defines:

• Authorization roles
• Access control rules
• What the server does once it knows who the client is

RBAC is entirely implementation-defined at all levels of the KMIP standard stack (core spec + profiles). The OPA RBAC system in this PR is a valid and non-conflicting extension.

---

Full compliance matrix

Requirement	Source	Cosmian KMS	Gap?
TLS 1.3 (SHALL), TLS 1.2 (SHOULD)	§3.1.1	✅ via OpenSSL	None
mTLS for client auth	§3.1.3	✅ socket server + HTTP	None
Use TLS CN as identity (no Credential)	§3.1.3	✅	None
Use Credential when present in message	§3.1.3 SHALL	❌ ignored	YES
HTTPS transport	§3.2	✅	None
Port 5696	§3.1.4 SHOULD	⚠️ default 9998	Minor
Authorization / RBAC	Not defined	OPA RBAC (extension)	N/A
JWT Bearer auth	Not mentioned	✅ (extension)	N/A

---

Recommended fix (closes both gaps)

Implement Authentication.Credential extraction in crate/server/src/core/operations/message.rs:

1. If RequestMessage.request_header.authentication is Some(auth):
   • Credential::UsernameAndPassword → use username as the client identity
   • Credential::Ticket { ticket_value } → attempt to parse as JWT → extract sub, roles, as_domain → inject into OpaUserContext
2. If absent → fall back to current behavior (TLS CN or HTTP Bearer)

This would:

• Close the §3.1.3 SHALL compliance gap
• Allow socket-mode clients (PyKMIP, Synology) to participate in OPA RBAC by embedding a JWT as a Ticket credential — without requiring HTTP transport
• Be fully backward-compatible (existing clients not sending Authentication are unaffected)

---

Analysis based on KMIP Profiles v2.1 OS, section 3 (Authentication Suites). The full profiles spec is at https://docs.oasis-open.org/kmip/kmip-profiles/v2.1/os/kmip-profiles-v2.1-os.html.

[serene-kitfisto-8899]

📋 KMIP Profiles v2.1 Compliance Audit — Authentication & Authorization

Checked against KMIP Profiles v2.1 OS (https://docs.oasis-open.org/kmip/kmip-profiles/v2.1/os/kmip-profiles-v2.1-os.html), specifically Section 3 (Authentication Suites).

---

Section 3 — Two Authentication Suites defined

3.1 Basic Authentication Suite (TCP/TLS — used by socket server):

• SHALL: TLS v1.3; SHOULD: TLS v1.2; SHALL NOT: TLS ≤1.1 or SSL
• SHALL: mutual TLS (mTLS) for client authenticity
• Port 5696 (IANA assigned)

3.2 HTTPS Authentication Suite (used by HTTP endpoint):

• SHALL: HTTP over TLS (RFC 2818)
• All TLS/cipher/auth requirements delegate back to §3.1

---

🔴 Compliance gap found — §3.1.3 Client Identity

The spec normatively states:

"Conformant KMIP servers SHALL use the identity derived from the channel mutual authentication to determine the client identity if the KMIP client requests do not contain an Authentication object."

"Conformant KMIP servers SHALL use the identity derived from the Credential information to determine the client identity if the KMIP client requests contain an Authentication object."

The Cosmian KMS currently ignores the KMIP Authentication.Credential in the RequestMessage. Identity is always taken from the HTTP Authorization: Bearer <JWT> header or TLS certificate CN, regardless of whether the client sends a Credential in the KMIP body.

Impact: A client conformant to the Basic Authentication Suite that embeds its identity in Authentication.UsernameAndPassword or Authentication.Ticket gets its credential silently discarded. This violates the second SHALL above.

---

No RBAC in profiles either

§3.1.3 explicitly defers: "The exact mechanisms determining the client identity are outside the scope of this specification."

No KMIP profile defines authorization roles, RBAC, or what the server does after determining client identity. The OPA RBAC layer in this PR is a fully spec-compliant implementation-defined extension — KMIP does not constrain it.

---

Full compliance matrix

Requirement	Source	Cosmian KMS	Gap?
TLS 1.3 (SHALL) / 1.2 (SHOULD)	§3.1.1	✅ OpenSSL/Actix	None
mTLS for client auth	§3.1.3	✅ socket + HTTP mTLS	None
Use TLS CN as identity (no Credential in message)	§3.1.3	✅	None
Use Authentication.Credential when present	§3.1.3 SHALL	❌ ignored	YES
HTTPS transport	§3.2	✅	None
Port 5696	§3.1.4 SHOULD	⚠️ default 9998	Minor
Authorization / RBAC	Not defined anywhere	OPA RBAC	N/A (extension)
JWT Bearer token	Not mentioned	✅ (RFC 9068 extension)	N/A (extension)

---

Recommended fix

Implement Credential extraction from the KMIP RequestMessage.Authentication header. This would:

1. Close the §3.1.3 compliance gap — use credential identity when present
2. Solve the TTLV socket RBAC problem — socket clients could embed a JWT as a Credential.Ticket (KMIP Table 442 Ticket type is designed for Kerberos tokens and opaque security tokens, i.e. JWTs)
3. Give UsernameAndPassword clients a path to participate in RBAC by mapping usernames to roles at KMS configuration time

Priority order for implementation:

1. Credential.Ticket { ticket_type, ticket_value } — treat ticket_value as a Bearer JWT → extract roles + as_domain
2. Credential.UsernameAndPassword — map username to a configured domain/role set

---

Spec reference: KMIP Profiles v2.1 OS §3.1.3 — https://docs.oasis-open.org/kmip/kmip-profiles/v2.1/os/kmip-profiles-v2.1-os.html