Skip to content

Add PostHog error tracking, richer telemetry, docs analytics, and release annotations#86

Open
leoisadev1 wants to merge 9 commits into
tembo/posthog-mcp-features-overviewfrom
posthog-code/analytics-error-tracking
Open

Add PostHog error tracking, richer telemetry, docs analytics, and release annotations#86
leoisadev1 wants to merge 9 commits into
tembo/posthog-mcp-features-overviewfrom
posthog-code/analytics-error-tracking

Conversation

@leoisadev1

Copy link
Copy Markdown
Member

Builds on #80 (stacked PR — merging this lands the changes into the #80 branch; merge #80 afterward to ship everything to main).

What's new

1. Error tracking (feat(telemetry))

  • captureException() in the telemetry module sends PostHog $exception events through the same zero-dependency transport.
  • Privacy-first sanitization: stack-frame paths reduced to package-relative names or bare basenames (never home dirs/project layouts); messages scrubbed of emails, URLs, quoted text, long tokens, and home directories; cause chains capped at 3; frames capped at 20.
  • Per-process guard rails: dedupe by error object and error class, max 5 reports per process.
  • Stable issue grouping via $exception_fingerprint = ErrorName:error_code for EmailSdkErrors.
  • Captured at: failed send() calls (handled; usage errors like validation/unknown-adapter excluded) and unexpected CLI crashes (unhandled).
  • New analytics context: source (sdk|cli) on all events, email batch sent summary event, ci_vendor detection (GitHub Actions, GitLab, CircleCI, Jenkins, Travis, Buildkite, Vercel, generic).
  • README/notice disclosures updated; changeset included. All existing opt-outs unchanged.

2. Docs site analytics (feat(docs))

  • posthog-js on email-sdk.dev: SPA pageviews (history-change), exception autocapture, web vitals. Vercel Analytics stays. Session replay is off by default (one-line flip in apps/fumadocs/src/lib/posthog.ts).
  • CSP in vercel.json allows https://*.posthog.com (script-src + connect-src).
  • Privacy policy discloses PostHog and links the package telemetry docs.

3. Release annotations (ci(release))

  • After a successful changesets publish, posts a "vX.Y.Z released" annotation to PostHog so release markers appear on every dashboard chart. Requires a POSTHOG_PERSONAL_API_KEY repo secret (scope: annotation:write, project 468042); skips gracefully without it.

PostHog workspace (already live, built via MCP)

  • New Reliability & Errors dashboard (failure rates, error codes, exceptions by type/source/version, latency p95, batch health) + 7 new usage insights (version adoption, platform split, adapter market share, dry-run vs real, CI vs local, activation funnel, SDK vs CLI) + 3 docs insights (adapter-page views for sponsor value, getting-started, blog).
  • Alerts: send-failure rate >20%/day and exceptions >25/day → email.
  • Action "Adapter docs viewed"; dashboard headers updated; starter dashboard unpinned.

Verification

  • bun run release:ci passes (types, 117 package tests + workspace tests, registry checks, full build, pack dry-runs).
  • Live smoke test from the built CLI: all 6 events ingested (client created, failed email sent, $exception, 3× cli command run), and Error Tracking created the issue EmailProviderError — "Resend failed with 401: API key is invalid" with fully resolved, sanitized frames (cli.js, core.js, resend.js — no user paths).

Created with PostHog Code

…d CI context

- captureException sends sanitized $exception events for PostHog error
  tracking: package-relative stack frames, scrubbed messages (emails, URLs,
  quoted text, tokens, home dirs), per-process dedupe and a 5-report cap
- tag client created / email sent / cli command run with source (sdk|cli);
  the CLI reports unexpected crashes as unhandled exceptions
- new email batch sent summary event alongside per-item events
- detect ci_vendor (GitHub Actions, GitLab, CircleCI, Jenkins, Travis,
  Buildkite, Vercel, generic) and derive ci from it
- disclose error reports in the notice and both READMEs; add changeset

Generated-By: PostHog Code
Task-Id: da0ebd32-eea6-4c6f-aa27-9eff2d00808f
…dk.dev

- posthog-js init on hydration with history-change pageviews, exception
  autocapture, and web vitals; session replay stays off (one-line flip)
- keep Vercel Analytics alongside
- allow https://*.posthog.com in the CSP script-src and connect-src
- disclose PostHog and package telemetry in the privacy policy

Generated-By: PostHog Code
Task-Id: da0ebd32-eea6-4c6f-aa27-9eff2d00808f
Posts a project annotation for @opencoredev/email-sdk releases so version
cohorts are visible on dashboards. Skips gracefully when the
POSTHOG_PERSONAL_API_KEY secret is absent and never fails the release.

Generated-By: PostHog Code
Task-Id: da0ebd32-eea6-4c6f-aa27-9eff2d00808f
@vercel

vercel Bot commented Jun 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
email-sdk-fumadocs Ready Ready Preview, Comment Jun 13, 2026 2:59am

@greptile-apps

greptile-apps Bot commented Jun 13, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds PostHog error tracking, enriched telemetry (source tagging, CI detection, batch summaries), browser analytics for the docs site, and a release annotation step to the publish workflow. The core new surface is captureException() in the SDK telemetry module, which manually constructs Sentry-style $exception payloads through the existing zero-dependency HTTP transport, with a thorough privacy-first sanitization pipeline.

  • SDK/CLI error tracking: captureException() redacts messages (emails, URLs, tokens, home directories), reduces stack frames to package-relative basenames, deduplicates by object identity and error class, and caps reports at 5 per process; captured for provider failures in core.ts and unhandled CLI crashes.
  • Telemetry enrichment: telemetrySource ("sdk"|"cli"), ci_vendor detection for 6 CI systems, and a new email batch sent summary event on sendBatch.
  • Docs site analytics: posthog-js initialized in a useEffect on the root TanStack Router route with session replay disabled; CSP updated to specific PostHog hosts rather than a wildcard.
  • Release annotations: Post-publish step calls the PostHog annotations API; skips gracefully when the secret is absent.

Confidence Score: 5/5

Safe to merge — all new telemetry paths are fire-and-forget with comprehensive try/catch guards, existing opt-outs are unchanged, and the privacy sanitization is well-tested.

The error tracking implementation is thorough: hostile error shapes (throwing getters, non-Error throws, circular cause chains) are all handled, deduplication prevents spamming PostHog, and every redaction rule has explicit test coverage. The core send/sendBatch paths are unchanged in behavior — telemetry is always void-and-ignored. The only new observable side-effects are the PostHog HTTP calls, bounded by the 3s timeout and a 5-report-per-process cap.

No files require special attention for merge safety. The minor observations in core.ts (empty-batch event), telemetry.ts (dedup classKey design), and the workflow step (version guard) are all non-blocking.

Important Files Changed

Filename Overview
packages/email-sdk/src/telemetry.ts Major addition: captureException() with privacy-first sanitization, buildExceptionList, parseStackFrames, redactErrorMessage, detectCiVendor, isReportableSendError. Well-tested and guarded; minor dedup/ordering design trade-offs noted.
packages/email-sdk/src/core.ts Adds telemetrySource tagging, captureException on reportable send failures, and a sendBatch summary event. Correct logic; empty-batch event fires unconditionally (P2).
packages/email-sdk/src/telemetry.test.ts Thorough test coverage for exception capture: sanitization, deduplication, cause chains, hostile stacks, CI detection, and disabled state — all new paths well-covered.
packages/email-sdk/src/core.test.ts New createEmailClient telemetry suite tests source tagging, exception reporting, usage-error exclusion, batch summary, mixed-adapter, and fallback adapter resolution — comprehensive.
apps/fumadocs/src/lib/posthog.ts New PostHog browser init for docs site with defaults: '2026-01-30' and capture_exceptions: true; no explicit capture_console_errors: false and no reverse proxy — both noted in prior review comments.
.github/workflows/release.yml Adds graceful PostHog release annotation step; bun -e failure could silently produce a vundefined annotation or fail the step with no guard (P2).
vercel.json CSP updated with specific PostHog hosts (us-assets.i.posthog.com for script-src, us.i.posthog.com + us-assets.i.posthog.com for connect-src) — tighter than a wildcard.
packages/email-sdk/src/cli.ts Adds telemetrySource: 'cli', source: 'cli' on run events, and unhandled-crash exception reporting before the CLI run summary; dedup correctly excludes already-reported SDK errors.

Sequence Diagram

sequenceDiagram
    participant CLI as cli.ts
    participant Core as core.ts (send/sendBatch)
    participant Tel as telemetry.ts
    participant PH as PostHog API

    CLI->>Core: "createEmailClient({telemetrySource:"cli"})"
    Core->>Tel: "capture("client created", {source:"cli"})"
    Tel-->>PH: POST /capture/ (queued)

    CLI->>Core: client.send(message)
    Core->>Core: sendWithAdapters()
    alt success
        Core->>Tel: "capture("email sent", {success:true, source})"
    else provider failure (isReportableSendError)
        Core->>Tel: "capture("email sent", {success:false})"
        Core->>Tel: "captureException(error, {handled:true})"
        Tel->>Tel: buildExceptionList() → redactErrorMessage() + parseStackFrames()
        Tel->>Tel: dedup check (seenObjects, seenClasses, budget)
        Tel-->>PH: POST /capture/ ($exception event)
    end

    CLI->>Tel: "capture("cli command run", {source:"cli"})"
    Note over CLI,Tel: captureCliRun() → flush()
    Tel->>Tel: flush() — await all pending

    alt unhandled crash (not CliFailure/EmailSdkError)
        CLI->>Tel: "captureException(error, {handled:false})"
        Tel-->>PH: POST /capture/ ($exception)
    end

    Note over PH: Release workflow
    CLI->>PH: POST /api/projects/468042/annotations/ (vX.Y.Z released)
Loading

Reviews (7): Last reviewed commit: "fix(telemetry): stop tagging Vercel prod..." | Re-trigger Greptile

Comment thread packages/email-sdk/src/telemetry.ts Outdated
Comment thread vercel.json
Addresses review findings on the error-tracking change:
- redact home directories before token matching so a long alphanumeric
  username collapses to "~" instead of leaking as "/home/<token>"
- guard captureException against hostile error shapes: ignore non-string
  stack values and wrap the build path so a throwing stack getter can never
  make reporting throw
- build the PostHog annotation body with jq so a crafted version string
  can't break out of the JSON or override the annotation scope

Generated-By: PostHog Code
Task-Id: da0ebd32-eea6-4c6f-aa27-9eff2d00808f
Greptile review follow-ups:
- redact tokens with lookaround anchors instead of \b so base64 secrets
  ending in "=" padding are scrubbed whole rather than leaking the tail
- mark an error object "seen" only after it passes the class and budget
  guards, so a bailed-out report never silently consumes a distinct error

Generated-By: PostHog Code
Task-Id: da0ebd32-eea6-4c6f-aa27-9eff2d00808f
… batch adapter

Greptile review follow-ups (pass 2):
- redact any scheme://… so SMTP/AMQP/DB connection strings with embedded
  credentials are scrubbed, and run URL redaction before email so a
  "scheme://user:pass@host" collapses whole
- reduce project-relative stack frames (tsx/bun-from-source) to basenames so
  reports never leak the project directory layout; node: frames kept intact
- report adapter "mixed" on email batch sent when items use different
  adapters instead of always showing the batch default

Generated-By: PostHog Code
Task-Id: da0ebd32-eea6-4c6f-aa27-9eff2d00808f
Greptile review follow-ups (pass 3):
- pin the docs-site CSP to PostHog's actual hosts (us-assets.i.posthog.com
  for scripts, us.i.posthog.com + us-assets for connect) instead of the
  https://*.posthog.com wildcard, so only PostHog's known origins run/serve
- record the adapter that actually delivered each batch item (response
  provider, which fallbacks change) rather than the intended primary, so the
  batch summary's adapter matches the per-item events

Generated-By: PostHog Code
Task-Id: da0ebd32-eea6-4c6f-aa27-9eff2d00808f
…paths

Greptile review follow-ups (pass 4):
- set posthog-js person_profiles: "identified_only" so anonymous docs
  traffic never creates person profiles, matching the SDK's server-side
  $process_person_profile: false and the privacy policy
- extend the home-path redaction to match backslash separators so another
  user's Windows home path (\Users\name) is scrubbed, not just Unix paths

Generated-By: PostHog Code
Task-Id: da0ebd32-eea6-4c6f-aa27-9eff2d00808f
Greptile review follow-up (pass 5): VERCEL=1 is injected into Vercel's
production serverless runtime, not just CI builds, so an SDK deployed in a
Vercel function would report every live send as ci=true. Drop the VERCEL
heuristic; Vercel CI builds still set CI=1 and resolve to "generic", while
production runtimes correctly report no CI.

Generated-By: PostHog Code
Task-Id: da0ebd32-eea6-4c6f-aa27-9eff2d00808f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant