Production-grade observability for Fabric Data Agent (FDA) NL-to-DAX interactions, backed by a Fabric Eventhouse (KQL) database.
It captures the full interaction trail — question → reasoning → grounding → generated DAX → answer — plus user, latency, tokens, downstream execution telemetry, governance events, and cost/usage, into a queryable, retention-managed sink. The module proxies the FDA published-endpoint call (the only first-party path that links the natural-language question to the generated DAX in one call), records the entire payload with optional PII redaction, and correlates it with Workspace Monitoring DAX traces and M365 Unified Audit governance events.
📖 Full documentation: https://patrickgallucci.github.io/PSFabricDataAgentObservability/
- PowerShell 7.2+ (Windows / Linux / macOS)
- A Fabric workspace with permission to create or point at an Eventhouse
- Permission to enable Workspace Monitoring on the semantic model (for executed-DAX correlation)
- For governance sync: an app registration with
ActivityFeed.Readonmanage.office.com - A Service Principal, Managed Identity, or interactive user that can reach FDA + Eventhouse
- For UserDelegated (browser) sign-in: a public client app present in your tenant — see the one-time admin prerequisite below
Browser sign-in (-AuthMethod UserDelegated) uses a public client app that must exist in your tenant. The least-friction option is to have a tenant admin instantiate the Azure CLI well-known app once:
Connect-MgGraph -Scopes "Application.ReadWrite.All"
New-MgServicePrincipal -AppId "04b07795-8ddb-461a-bbee-02f9e1bf7b46" # Azure CLIOnce that service principal exists, Connect-FDAObservability works with config.json ClientId left null. Why this is easiest: the Azure CLI app already carries broad pre-consented delegated permissions (Fabric/Power BI, ARM, Kusto), so there's nothing else to configure.
No admin access? Register your own public-client app (Authentication → Allow public client flows = Yes, redirect URI http://localhost), grant it the delegated permissions above, and put its app id in config.json ClientId (or pass -ClientId). The symptom of a missing/unprovisioned app is AADSTS700016 at sign-in.
Install-Module PSFabricDataAgentObservability -Scope CurrentUserOr import from source:
Import-Module ./PSFabricDataAgentObservability.psd1# 1. Sign in. UserDelegated uses the browser (auth-code + PKCE): a window opens,
# you sign in, the tenant is discovered from the token and returned — no
# tenant prompt, no code to paste. (See the admin prerequisite above.)
$TenantId = Connect-FDAObservability -AuthMethod UserDelegated
# 2. One-time provisioning (tables, mappings, policies, functions, seed levels).
# The Workspace / Eventhouse / Database default to config.json
# (WorkspaceName 'FUAM PUB', EventhouseName 'FDAObservability',
# DatabaseName 'FDAObs') and are CREATED if they don't already exist.
Initialize-FDAObservability -TenantId $TenantId
# …or override any of the names (still create-if-missing):
Initialize-FDAObservability -TenantId $TenantId `
-WorkspaceName 'My WS' -EventhouseName 'FDAObservabilityProd'
# …or target an existing workspace/Eventhouse explicitly:
Initialize-FDAObservability -WorkspaceId '<workspace-guid>' -EventhouseId '<eventhouse-guid>'
# 3. Replace direct FDA calls with the proxy.
$answer = Invoke-FDAQuery -AgentEndpoint 'https://<fda-endpoint>' `
-Question 'Revenue by region last quarter?'
# 4. Query what was captured.
Get-FDAInteraction -Last 24h -Top 20
Get-FDAAuthEvent -Last 7d -Outcome Failure
New-FDAObservabilityReport -Report DailyOps| Area | Cmdlets |
|---|---|
| Setup | Initialize-FDAObservability, Connect-FDAObservability, Disconnect-FDAObservability, Set-FDAObservabilityConfig, Get-FDAObservabilityConfig |
| Levels | Register-FDALogLevel, Unregister-FDALogLevel, Get-FDALogLevel |
| Capture | Invoke-FDAQuery, Write-FDALog, Sync-FDAGovernanceLog |
| Read | Search-FDALog, Get-FDAInteraction, Get-FDAExecutionTelemetry, Get-FDAAuthEvent, Get-FDACostUsage, New-FDAObservabilityReport |
| Health | Test-FDAObservability |
See the cmdlet index for details.
Invoke-Pester -Path ./tests
Invoke-ScriptAnalyzer -Path . -Recurse -Settings ./PSScriptAnalyzerSettings.psd1
Test-ModuleManifest ./PSFabricDataAgentObservability.psd1docs/README.md— overview, quick start, cmdlet indexdocs/ARCHITECTURE.md— architecture, design choices, trade-offsdocs/SCHEMA.md— Eventhouse table referencedocs/OPERATIONS.md— runbook, dashboards, retention, alertsCHANGELOG.md— release notes
MIT © Patrick Gallucci