fix(desktop): structured error_kind on chat_agent_error telemetry (#9342)#9348
Open
kodjima33 wants to merge 1 commit into
Open
fix(desktop): structured error_kind on chat_agent_error telemetry (#9342)#9348kodjima33 wants to merge 1 commit into
kodjima33 wants to merge 1 commit into
Conversation
…#9342) The macOS chat 'AI not available: bridge failed to start' error dominates beta chat failures (~21.8% of active beta users), but every non-carded failure — non-BridgeError launch errors plus BridgeError cases that ChatErrorState.from() maps to nil (.agentError, .agentRuntimeFailure, .quotaExceeded, .encodingError, .requestAlreadyActive, .stopped) — is flattened into one opaque 'error' string, so PostHog cannot tell a missing install from a launch failure, a crash, or a provider/config error. Add BridgeError.telemetryKind(for:) and attach it as a low-cardinality 'error_kind' property on both chat_agent_error emit sites. Non-BridgeError launch failures surface the NSError domain+code (POSIX ENOENT = missing executable / install problem), distinct from crashes. Addresses issue #9342 acceptance criterion #3 (structured fields to distinguish failure modes). No behavior change; existing 'error' string preserved. Regression test: testTelemetryKindDistinguishesFailureModes. Verified: xcrun swift build -c debug (Build complete) + swift test --filter ChatErrorStateMappingTests (3/3 pass).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
Issue #9342 (p1, reliability): macOS beta chat requests fail with
AI not available: bridge failed to start— the dominant beta chat error (~21.8% of active beta/newer users in the last 24h). Acceptance criterion #3 asks for telemetry structured enough to distinguish missing install, launch failure, crash, and provider/config errors.Root cause (observability gap)
In
ChatProvider's twochat_agent_erroremit paths, every failure that isn't rendered as aChatErrorStatecard is flattened into a single opaqueerrorstring. This bucket mixes:BridgeErrorlaunch failures (the actualProcessspawn errors behind "bridge failed to start"), andBridgeErrorcases thatChatErrorState.from()maps tonil:.agentError,.agentRuntimeFailure,.quotaExceeded,.encodingError,.requestAlreadyActive,.stopped.So PostHog can't tell a missing install from a launch failure, a crash, or a provider/config error — exactly what the issue needs to triage this p1.
Fix
BridgeError.telemetryKind(for:)— a stable, low-cardinality category for anyError.BridgeErrorcases map tobridge.*kinds; non-BridgeErrorlaunch failures surface theNSErrordomain+code (e.g. POSIXENOENT= missing executable / install problem, distinct from a crash).error_kindproperty on bothchat_agent_erroremit sites via a new optionalchatAgentError(errorKind:)param.error/raw_errorfields are unchanged, so current dashboards keep working.No user-facing behavior change — telemetry enrichment only.
Tests / verification
testTelemetryKindDistinguishesFailureModes(distinct causes → distinct kinds; missing-binary ≠ crash).xcrun swift build -c debug --package-path Desktop→ Build complete.xcrun swift test --filter ChatErrorStateMappingTests→ 3/3 pass.🤖 automated by hourly watchdog; opened for review, not merged. This is a diagnostics change — it does not itself resolve the underlying bridge-start failure, so it is intentionally not auto-merged.