Skip to content

Add anonymous usage telemetry to Email SDK and CLI#80

Open
tembo[bot] wants to merge 2 commits into
mainfrom
tembo/posthog-mcp-features-overview
Open

Add anonymous usage telemetry to Email SDK and CLI#80
tembo[bot] wants to merge 2 commits into
mainfrom
tembo/posthog-mcp-features-overview

Conversation

@tembo

@tembo tembo Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Add anonymous usage telemetry to the Email SDK and CLI using PostHog. Captures events: client created, email sent, and cli command run with adapter names, command names, success/failure, error codes, duration, and recipient counts. No sensitive data (email content, addresses, headers, credentials) is collected. Includes opt-out via environment variables (EMAIL_SDK_TELEMETRY=0, DO_NOT_TRACK=1) or client config (telemetry: false). Telemetry is disabled automatically in test environments. Adds a one-time opt-out notice on first use.

  • Introduce telemetry module with PostHog integration.
  • Capture telemetry events in core client and CLI.
  • Normalize adapter names to mask custom adapters.
  • Add telemetry opt-out and notice logic.
  • Update README with telemetry documentation.
  • Add tests for telemetry functionality and opt-out.

Want tembo to make any changes? Add a comment with @tembo and i'll get back to work!

View on Tembo View Agent Settings

@tembo tembo Bot added the tembo Pull request created by Tembo label Jun 12, 2026
@tripwire-prod

tripwire-prod Bot commented Jun 12, 2026

Copy link
Copy Markdown

Tripwire — This PR was automatically closed.

Reason: @tembo[bot] has 0 public repos (minimum: 3).

Think this was a mistake? Request a review as @tembo[bot].

@tembo

tembo Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Requesting review from @leoisadev1 who has experience with the following files modified in this PR:

  • bun.lock
  • README.md
  • packages/email-sdk/README.md
  • packages/email-sdk/src/cli.ts
  • packages/email-sdk/src/core.ts
  • packages/email-sdk/src/types.ts

@tembo tembo Bot requested a review from leoisadev1 June 12, 2026 21:28
@tripwire-prod tripwire-prod Bot closed this Jun 12, 2026
@vercel

vercel Bot commented Jun 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
email-sdk-fumadocs Ready Ready Preview, Comment Jun 13, 2026 1:19am

@leoisadev1 leoisadev1 reopened this Jun 13, 2026
@leoisadev1 leoisadev1 marked this pull request as ready for review June 13, 2026 01:09
@greptile-apps

greptile-apps Bot commented Jun 13, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds anonymous usage telemetry to the Email SDK and CLI via a custom PostHog HTTP integration, capturing client created, email sent, and cli command run events with adapter names, success/failure, duration, and recipient counts. Opt-out is supported via env vars (EMAIL_SDK_TELEMETRY=0, DO_NOT_TRACK=1), a per-client telemetry: false option, and automatic disabling in NODE_ENV=test.

  • Introduces telemetry.ts with a singleton + disabled-stub pattern, flush() for pre-exit draining, a stable anonymous ID persisted to ~/.config/email-sdk/telemetry.json, and a one-time opt-out notice.
  • Refactors cli.ts to pass pre-parsed flags into main(), adds captureCliRun that awaits flush() before any process.exit(1), and wraps fail() with a CliFailure class to keep error paths clean.
  • Instruments core.ts send() with per-call telemetry events; documentation (both READMEs and changeset) correctly lists all collected fields.

Confidence Score: 5/5

Safe to merge — telemetry is purely additive, all opt-out paths work, and the CLI refactor correctly flushes events before process.exit.

The core email sending path is unchanged and all telemetry calls are fire-and-forget or properly awaited before exit. No blocking or correctness issues were found in the changed code paths.

packages/email-sdk/src/telemetry.ts (notice persistence in read-only environments) and packages/email-sdk/src/core.ts (sendBatch N-event fan-out).

Important Files Changed

Filename Overview
packages/email-sdk/src/telemetry.ts New custom PostHog integration module; well-structured with disabled-stub, flush, and singleton patterns. Notice can repeat in read-only environments; raw HTTP sends one request per event with no batching.
packages/email-sdk/src/core.ts Instruments createEmailClient and send() with telemetry; sendBatch generates N parallel telemetry requests for N messages via individual send() delegation.
packages/email-sdk/src/cli.ts Refactored main() to accept parsed args so captureCliRun can reuse them; flush() is called before process.exit(1) to prevent event loss — correctly addresses the previous fire-and-forget concern.
packages/email-sdk/src/telemetry.test.ts New test file with good coverage of opt-out paths, stable anonymous ID, flush gate, error resilience, and notice deduplication.
packages/email-sdk/src/types.ts Adds optional telemetry boolean field to EmailClientOptions with clear inline documentation.

Sequence Diagram

sequenceDiagram
    participant CLI as cli.ts (top-level await)
    participant Main as main()
    participant Core as createEmailClient / send()
    participant Tel as telemetry.ts (singleton)
    participant PH as PostHog /capture/

    CLI->>Main: main(command, flags)
    Main->>Core: createEmailClient(options)
    Core->>Tel: getTelemetry()
    Tel-->>Core: Telemetry instance
    Core->>Tel: void capture("client created")
    Tel--)PH: POST /capture/ (fire-and-forget)

    Main->>Core: client.send(message)
    Core->>Tel: void capture("email sent")
    Tel--)PH: POST /capture/ (fire-and-forget)
    Core-->>Main: response

    Main-->>CLI: resolved

    CLI->>Tel: captureCliRun → capture("cli command run")
    Tel->>PH: POST /capture/ (awaited)
    PH-->>Tel: 200 OK
    CLI->>Tel: flush() → Promise.all(pending)
    Tel->>PH: await remaining in-flight requests
    PH-->>Tel: settled
    Tel-->>CLI: flushed

    alt error path
        CLI->>Tel: captureCliRun (success: false)
        Tel->>PH: POST /capture/
        CLI->>Tel: flush()
        CLI->>CLI: process.exit(1)
    end
Loading

Reviews (2): Last reviewed commit: "fix(telemetry): address review feedback ..." | Re-trigger Greptile

Comment on lines +116 to +118
const messageFacts = {
recipients: arrayify(message.to).length,
has_attachments: (message.attachments?.length ?? 0) > 0,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 has_attachments collected but not disclosed in README

has_attachments is sent with every email sent event but is not mentioned anywhere in either README under "What is collected". Both READMEs list "recipient counts, SDK version, OS, and Node.js version" but omit this field entirely. Users who read the documentation to understand what is tracked will not see this field. Given this is opt-in-by-default telemetry, every collected property should be explicitly listed. Score: 4/5

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Codex

Comment thread packages/email-sdk/src/core.ts Outdated
Comment on lines +120 to +124
const doNotTrack = env.DO_NOT_TRACK?.toLowerCase();

if (doNotTrack === "1" || doNotTrack === "true") {
return true;
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 DO_NOT_TRACK only honors "1" and "true", misses the plain "1" standard

The existing behaviour is reasonable, but a comment explaining the deliberate choice would prevent future confusion. Score: 2/5

Suggested change
const doNotTrack = env.DO_NOT_TRACK?.toLowerCase();
if (doNotTrack === "1" || doNotTrack === "true") {
return true;
}
// Honour the standard DNT value "1" as well as the common "true" alias.
// Any other value (including "0") is treated as not opting out.
const doNotTrack = env.DO_NOT_TRACK?.toLowerCase();
if (doNotTrack === "1" || doNotTrack === "true") {
return true;
}

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Codex

Comment thread packages/email-sdk/src/core.ts
Comment thread packages/email-sdk/src/telemetry.ts
Comment thread packages/email-sdk/src/cli.ts Outdated
…CLI parsing

- count cc and bcc recipients in the email sent event and disclose
  has_attachments plus the full recipient definition in both READMEs
- track in-flight captures and add Telemetry.flush() so the CLI settles
  fire-and-forget events from core.ts before process.exit(1)
- add resetTelemetry() to clear the cached shared instance
- document the deliberate DO_NOT_TRACK value handling
- parse process.argv once in the CLI entrypoint and pass command/flags
  to main() and captureCliRun()

Co-authored-by: null <>
@tembo

tembo Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor Author

Greptile Summary

This PR introduces anonymous telemetry to the Email SDK and CLI via direct PostHog HTTP capture (no SDK dependency). Three event types are captured — client created, email sent, and cli command run — with adapters normalized to mask custom names. Opt-out is supported via EMAIL_SDK_TELEMETRY=0, DO_NOT_TRACK=1, telemetry: false on the client, or automatically when NODE_ENV=test.

  • has_attachments is sent with every email sent event but is not disclosed in either README under "What is collected", creating a transparency gap in the documented telemetry inventory.
  • recipients only counts message.to, while both READMEs describe it as "recipient counts", which users would reasonably interpret as the full addressee total (to + cc + bcc).
  • The void telemetry captures in core.ts are fire-and-forget and can be silently dropped when the CLI's error path calls process.exit(1) before those HTTP requests settle.

Confidence Score: 3/5

Safe to merge functionally — telemetry never breaks email delivery — but the transparency gap in what data is disclosed needs fixing before this ships to users.

Two issues in the data disclosure story: has_attachments is collected silently (not listed in either README's "What is collected" section), and recipients is documented as a full recipient count but only reflects the to field. Both affect user trust in an opt-out-by-default feature. Beyond transparency, fire-and-forget captures in core.ts can be silently dropped in the CLI error path after process.exit(1), and the module-level telemetry singleton cannot be cleared between calls in long-running or test processes.

packages/email-sdk/src/core.ts (undisclosed has_attachments field and partial recipients count) and both README.md / packages/email-sdk/README.md (telemetry disclosure sections need updating).

Important Files Changed

Filename Overview
packages/email-sdk/src/telemetry.ts New telemetry module using direct PostHog HTTP capture; correctly gates on env vars and NODE_ENV=test, but the module-level singleton is never cleared and DO_NOT_TRACK handling could use a clarifying comment.
packages/email-sdk/src/core.ts Adds telemetry capture to createEmailClient and send; has_attachments is collected but undisclosed in the README, and recipients only counts the to field despite docs implying a full count.
packages/email-sdk/src/cli.ts Refactors fail() to throw CliFailure and adds captureCliRun; fire-and-forget captures from core.ts may be dropped on process.exit(1) in the error path, and captureCliRun re-parses process.argv unnecessarily.
packages/email-sdk/src/telemetry.test.ts Comprehensive tests for opt-out paths, anonymous ID persistence, delivery failure resilience, and notice deduplication; uses createTelemetry() directly to avoid singleton issues.
packages/email-sdk/src/types.ts Adds optional telemetry boolean to EmailClientOptions with clear JSDoc; straightforward and non-breaking.
README.md Adds telemetry section documenting opt-out and what is/isn't collected; does not mention has_attachments in the collected fields list.
packages/email-sdk/README.md Same telemetry section as root README; same omission of has_attachments from disclosed fields.

Fix All in Codex

Reviews (1): Last reviewed commit: "feat(telemetry): add anonymous usage ana..." | Re-trigger Greptile

addressed all 6 greptile review issues on pr #80 in commit 44008da:

  1. has_attachments is now disclosed in the telemetry section of both readmes
  2. recipients now counts to + cc + bcc, and the readmes spell out that definition
  3. added a comment in telemetry.ts explaining the deliberate DO_NOT_TRACK value handling
  4. added a flush() method to telemetry that tracks in-flight captures; the cli now flushes before it can hit process.exit(1), so the fire-and-forget events from core.ts are no longer dropped
  5. added resetTelemetry() to clear the cached shared instance for long-running processes and tests
  6. the cli entrypoint now parses process.argv once and passes command/flags into main() and captureCliRun(), no re-parsing

validated with bun test (89 pass, including a new flush test), check-types, oxlint/oxfmt, and a cli smoke test of the help, dry-run send, and error paths.

pr: #80

@tembo

tembo Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor Author

these inline review comments are from greptile's review of the older commit (7bbf9b9) — all six were already fixed in commit 44008da, which is the current head of the pr branch. i verified each one is in place there:

  1. has_attachments is disclosed in both readmes' telemetry sections
  2. recipients counts to + cc + bcc, with the definition spelled out in the readmes
  3. the DO_NOT_TRACK comment from the suggestion is in telemetry.ts
  4. telemetry has a flush() that the cli awaits before process.exit(1), so fire-and-forget captures from core.ts aren't dropped
  5. resetTelemetry() clears the cached shared instance
  6. the cli parses process.argv once and passes command/flags into main() and captureCliRun()

re-verified on the branch head: bun test passes (89 tests) and check-types is clean. nothing new to push — re-triggering greptile on the latest commit should resolve these.

pr: #80

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tembo Pull request created by Tembo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant