Add Tracks analytics (Phase 1) alongside MC Stats by wojtekn · Pull Request #4162 · Automattic/studio

wojtekn · 2026-07-10T11:29:50Z

Related issues

Related to STU-2036

How AI was used in this PR

I used Claude for codebase exploration, to create an initial implementation plan, and for the initial implementation, following a few iterations to create the desired architecture.

Proposed Changes

This is Phase 1 of adopting Automattic Tracks in Studio. Today the only behavioral analytics is MC Stats — anonymous, aggregated, counters with no identity and no event properties, so we can't build funnels or attach context. This PR sets up a Tracks pipeline that runs in parallel with MC Stats (nothing is removed) so we can validate the two agree before migrating dashboards in a later phase.

What it adds for users and the team:

Two structured events to start: studio_app_launch (desktop) and studio_site_start, the latter reported consistently whether a site is started from the legacy UI, the agentic UI, or the standalone CLI — counted exactly once.
A single anonymous install identity shared by the desktop app and the CLI. No PII is attached (events are _ut=anon; no site names, paths, or WordPress.com user id).
An opt-out ("Help improve Studio by sharing anonymous usage statistics"), enabled by default, available in both Settings and the onboarding screen. Turning it off stops Tracks everywhere; MC Stats (aggregated stats) and Sentry (crash reports) are unaffected.

Events carry the data team's standardized shared properties (channel, is_a11n, platform, arch, app_version) plus a Studio-specific ui_version to distinguish the legacy vs agentic renderer.

Design and rationale are documented in docs/design-docs/analytics-tracks.md (linked from AGENTS.md), including a living event catalog and a "how to add an event" checklist.

Current UI	Agentic UI	Onboarding

Testing Instructions

studio_app_launch (desktop): npm start, then check the npm start terminal for Would have recorded… studio_app_launch { channel: 'studio-ui', ui_version: 'v1', is_a11n, is_first_launch } (dev no-ops the network send). Add enableAgenticUi to see ui_version: v2.
studio_site_start (CLI): run the CLI directly against an offline site (a running site short-circuits start before the event; watch the terminal, not a log file). Stop it first with node apps/cli/dist/cli/main.mjs stop --path <site>, then STUDIO_FORCE_CLI_TELEMETRY=1 STUDIO_DEBUG_TRACKS=1 node apps/cli/dist/cli/main.mjs start --path <site> and look for Tracks event URL: …_en=studio_site_start…&channel=studio-cli. STUDIO_FORCE_CLI_TELEMETRY=1 enables the CLI path on a dev build without rebuilding; set STUDIO_TRACKS_ORIGIN=studio-ui:v2 to see the UI variant.
Opt-out: toggle it in Settings (both renderers) and onboarding; confirm the Tracks log stops while MC Stats continues.
Live pixel to pixel.wp.com only fires in a shipped npm/prod build — verify server-side once events are registered.

Pre-merge Checklist

Have you checked for TypeScript, React or other console errors?

🤖 Generated with Claude Code

Introduce Automattic Tracks event logging running in parallel with the existing MC Stats counters. Phase 1 wires two events — studio_app_launch (desktop) and studio_site_start (CLI, the sole emitter for both UI- and CLI-initiated starts) — with an anonymous install id and opt-out shared across the desktop app and CLI, plus an opt-out toggle in Settings and onboarding. Events flow from the legacy renderer, the agentic UI, and the CLI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Do the check-and-mint for analyticsInstallId inside a single lockSharedConfig critical section with a re-read, so concurrent first-launch callers (desktop boot plus the spawned CLI) converge on one install id instead of racing to mint two. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Fold analyticsEnabled into apps/ui UserPreferences so the agentic renderer's Settings > Preferences exposes the same opt-out as the legacy renderer and onboarding, reusing the existing preferences query/mutation and the shared getAnalyticsEnabled/saveAnalyticsEnabled IPC handlers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

DataForm's default boolean renderer doesn't bind the checkbox to the record value, so the toggle showed unchecked regardless of the saved opt-out state. Supply a custom Edit control (matching the other boolean fields in this app) that reads and writes data.analyticsEnabled. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The persisted React Query cache from before analyticsEnabled existed served a UserPreferences object without the field, so the toggle rendered unchecked on upgrade despite the user being opted in. Bump the persister buster to discard stale caches, and default analyticsEnabled to true in toFormData as a safety net. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Define TracksChannel and TracksUiVersion once in record-tracks-event.ts and re-export them from the desktop and CLI wrappers, instead of declaring the same union in both, to prevent the vocabulary drifting between the two. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

_via is not a recognized reserved Tracks pixel param and would be ignored server-side. Origin/context is already carried by the channel event property (studio-ui / studio-cli), so remove _via entirely. Document the verified reserved param set (_en/_ut/_ui/_ts) and that identity is anonymous-only in Phase 1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The recordAnalyticsEvent IPC handler is the one untrusted entry point into the Tracks path. Add an isTracksEventName runtime guard and reject unknown event names with a warning, so a renderer bug or typo can't record events with names outside the known set. (URLSearchParams already encodes values, so this is data hygiene rather than an injection fix.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Spell out that studio_site_start counts every start exactly once regardless of --avoid-telemetry, and that channel separates UI from CLI starts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Distinguish --avoid-telemetry (runtime double-count guard, ignored by Tracks) from __ENABLE_CLI_TELEMETRY__ (build-time dev-vs-shipped switch, honored), and note that a dev build therefore logs no studio_site_start by design. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Clarify what fires in which build: unit tests as the primary check, app_launch observable in a dev run, site_start requiring a temporary __ENABLE_CLI_TELEMETRY__ flip (not observable in a plain dev run), and the live pixel needing a shipped build. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

recordTracksEvent was fire-and-forget (void), but its wrapper awaits several config file reads before sending. A short-lived `studio start` process could exit before those resolved, silently dropping studio_site_start. Await it, like the sibling recordSiteRuntimeUsage call, so the event reliably fires. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The manual site-start check silently produces no output if the site is already running (start no-ops) or if run via the desktop app (child-process stdout). Spell out the required steps: rebuild with the flag, stop the site first, run the CLI directly with STUDIO_DEBUG_TRACKS. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The CLI Tracks gate is a build-time flag (__ENABLE_CLI_TELEMETRY__), so testing site-start against a dev build required flipping the flag and rebuilding — and a rebuild with it off dead-code-eliminates the wrapper body. Add a runtime env override so a stock dev build can emit for testing without any rebuild, and rewrite the doc's testing steps around it (plus the offline-site and watch-the-terminal gotchas). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

github-actions Bot assigned wojtekn Jul 10, 2026

wojtekn and others added 15 commits July 10, 2026 16:21

Document is_first_launch coupling to MC Stats lastBumpStats

6fa5f26

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Trim Legal note and open-items section from analytics design doc

c5b4cfe

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Clarify site-start counting rule for data consumers

a8d76e2

Spell out that studio_site_start counts every start exactly once regardless of --avoid-telemetry, and that channel separates UI from CLI starts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

wojtekn marked this pull request as ready for review July 10, 2026 16:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add Tracks analytics (Phase 1) alongside MC Stats#4162

Add Tracks analytics (Phase 1) alongside MC Stats#4162
wojtekn wants to merge 16 commits into
trunkfrom
add-tracks-analytics

wojtekn commented Jul 10, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

wojtekn commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related issues

How AI was used in this PR

Proposed Changes

Testing Instructions

Pre-merge Checklist

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

wojtekn commented Jul 10, 2026 •

edited

Loading