Local ONNX inference (embeddings + first-class classify) and AI-gateway OpenTelemetry improvements

[rickcrawford]

Summary

Two threads of work for the AI gateway:

1. Local ONNX inference. Run the embedding semantic cache and prompt-injection classify on local models (no per-call API cost, no prompt egress, low loopback latency, air-gap capable). Sidecar by default for process isolation; in-process opt-in for a single binary.
2. Best-of-class OpenTelemetry. Richer, standards-conformant AI spans so traffic renders as full LLM trajectories in LLM-native backends.

Plus three cross-cutting fixes: multi-tenant attribution on the new metrics and logs, and a CI Node-20 deprecation cleanup.

Local inference

• New OnnxEmbedder on the pure-Rust tract engine (mean-pool + L2-normalize), with all-MiniLM-L6-v2 pinned as the default embedding model (Apache-2.0).
• The classifier sidecar now implements the Embed RPC (--embed-model), and the client gained embed().
• The existing embedding semantic cache gained a source switch: provider (default, unchanged), sidecar (local, free, no egress), and inprocess (single binary, behind the inprocess-embed build feature which the released binary enables). Any local-source failure degrades to a cache miss, never wedging a request.
• Prompt-injection classify is now first-class: an opt-in in-process ONNX detector (detector: inprocess) with a max_model_bytes guard, alongside the existing sidecar detector.
• New operator guide: docs/local-inference.md.

OpenTelemetry

• Derived USD cost on the AI request span (gen_ai.usage.cost + llm.usage.total_cost), stamped at the billing choke point so the span and the cost metric agree.
• AI error span semantics: failures set otel.status_code = ERROR plus a typed error.type (guardrail block, rate limit, provider error, content filter), wired at the rate-limit returns and the input-guardrail block.
• A GenAI semantic-convention conformance test that pins the attribute set so emitted spans cannot silently drift off-spec.
• docs/observability.md documents the AI-span attributes and a compatible LLM-native backend matrix (Phoenix, Langfuse, Jaeger, Tempo, Datadog, Honeycomb).

Usage tracking and multi-tenancy

• New metrics: semantic-cache results, local-inference call counts and latency, and tokens/cost saved by a cache hit (using the same cost table as spent cost, so saved and spent reconcile). Cataloged in docs/metrics-stability.md.
• The cache-result and savings metrics, and the new dispatch log lines, carry a tenant label/field for multi-tenant attribution.

CI

• Bumped the GitHub Actions that still ran on Node 20 to their node24 successors (Swatinem/rust-cache, peter-evans/create-pull-request); the others already use node24 majors. Remaining node20 actions have no node24 release yet.

Testing

• Per-crate unit tests for the embedder math, the sidecar/client embed round-trip, the cache source config, the in-process detector, the metric registration, the cost/error span helpers, and the semconv conformance.
• Validated locally: cargo fmt --check, cargo clippy --workspace --all-targets -D warnings (both feature states), and the workspace test suite.

Follow-ups (not in this PR)

A few observability items need live-OTLP-backend verification (a collector plus Phoenix/Langfuse), so they are scoped separately rather than shipped unverified: prompt/completion content events (capture-gated, redacted), a verified Phoenix/Langfuse reference compose stack, cost-aware head sampling wiring (the config already exists), and a span-arrival end-to-end test. The local-inference e2e (gated on a downloaded model) and an example config are also follow-ups.