[Feature][KubeAI] Enabling Observability of Gaudi

[joshuayao]

Priority

Undecided

OS type

Ubuntu

Hardware type

Xeon-GNR

Running nodes

Single Node

Description

Enabling Observability for Gaudi

[joshuayao]

Hi @eero-t Just confirm if we can close this issue after #1054 get merged?

[eero-t]

PR #1054 was minimal support.

After PR #993 is merged (needed to integrate OPEA applications with KubeAI), I'll try that, look into current application specific (perf + scaling) dashboards, and what needs to be done so that they provide also metrics for relevant KubeAI scaled integration engines (that run in the same cluster).

=> They don't work fully with KubeAI, as dashboards cannot anymore identify inferencing engine instances used by the given application, as being in the same namespace & using the same pod/service name prefix. Applications use specific models though, so I could potentially use metrics from vLLM instances using the same models, if I can get "remote" model info reasonably from app Helm chart to the dashboard spec.

Another option is just a separate scaling dashboard for KubeAI (vLLM instances). That's needed also in case KubeAI is in completely separate cluster from the OPEA application.

=> Re-opening. I'll add PR for separate KubeAI (vLLM) scaling dashboard soon, the potential OPEA app dashboards update comes later, if their (same cluster) KubeAI support turns out feasible.