Skip to content

feat(telemetry): send actual.adr.created from adr new and adr create - #960

Open
benw5483 wants to merge 1 commit into
gate-resolved-eventfrom
adr-created-event
Open

benw5483 wants to merge 1 commit into
gate-resolved-eventfrom
adr-created-event

Conversation

@benw5483

@benw5483 benw5483 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Each run of actual adr new and actual adr create now sends one OpenTelemetry event, actual.adr.created, from the same emitter module as actual.adr.discovered and actual.adr.reviewed. It says where the decision went (local for new, cloud for create), whether it came from flags or create --file, the status it was written or saved with, its policy count, whether a create --file save left its scope or publish step unfinished, and the saved decision's id, then the shared outcome, exit code, terminal flag and duration. Outcomes are succeeded, invalid, duplicate, not_signed_in and failed.

The decision id goes only in a request carrying the sign-in's token, as actual.adr.reviewed's does. adr new never sends one, since its id is a fresh local UUID. No title, context, policy text, scope glob or file path is sent, and the telemetry switch stops it. actual.event.schema_version goes from 7 to 9; version 8 is the web app's actual.adr.reviewed producer, which changed nothing the CLI sends. PRIVACY.md and the CHANGELOG say so.

Where the risk sits: exec_create now builds its result before printing so the event can see it, and write_new/run_create gained _noting variants that record what they learn. The open create --all PRs (#943, #950, #951) touch the same functions; whichever lands second adapts the event to one row per run with a count.

Stacked on #958 (v7), which is stacked on #953. Companion PRs: the catalog is architecture #39 and the web and pump producers are sprintreview #4627.

Attributes and test plan

Attributes

Attribute adr new adr create
actual.adr.create.target local cloud
actual.adr.create.source flags flags or file
actual.adr.create.status as written (or asked for, when it stopped first) as saved; a --file save that couldn't finish is a draft
actual.adr.create.policies count, 0 before they were read same
actual.adr.create.partial false true when scope or publish failed after a --file save
actual.adr.decision_id empty the saved id, signed in only

Test plan

  • cargo test --all-features --lib: 5013 passed, 1 ignored.
  • New tests: create_reports_its_target_and_each_outcome (every outcome, both targets, the exit codes), new_sends_one_event_with_counts_and_no_content and create_sends_one_event_and_none_of_the_content (end to end through exec_new and exec_create, no title, policy text or path in what's sent). The otlp spool and token tests now cover the created event's signed-in-only decision id, and the off-switch and debug-mode tests include it.
  • cargo +1.99.1 clippy -- -D warnings, cargo +1.99.1 clippy --locked --no-default-features --lib --bins -- -D warnings, cargo +1.99.1 clippy --all-features --tests -- -D warnings, cargo +1.99.1 fmt --check: clean.

Generated by the operator's software factory.
• City: factory-main · Agent: local-core.builder-1
• On behalf of: @benw5483

@actual-factory actual-factory Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review by austinborn's review page (review-austinborn).

Recommend approval · Risk: Low · Confidence: High · Reviewed head: 5596288fe9c6

I found no P0-P2 defect. The change adds one actual.adr.created event to adr new and adr create, and what the code sends matches what the docs promise: counts, outcomes, and a saved-decision id that only a signed-in request carries, with no decision text or paths. The base requires no checks and none ran at this head, so build and test status here comes from reading the code and its tests, not from a run.

The one finding, a P3 on test coverage of the create --file path, is posted inline; there is nothing else to list here.

Review evidence: enumeration, open questions, checks and coverage ledger

Enumeration (each item the change modifies, the other references to it at the reviewed head, and whether this change updated them)

  1. Shared symbol, the event schema version (7 to 9). Eight references: the one production read (the shared record builder), five tests that pin the value (three in src/telemetry/otlp.rs, two in src/cli/commands/adr/telemetry.rs), the PRIVACY.md row and the changelog entry. Updated 8, not updated 0. The other schema_version fields in the tree belong to the ADR document format and are unrelated.
  2. Shared symbol, run_create and write_new, now test-only wrappers over new _noting forms. Before the change they had 2 production callers (exec_create, exec_new) and 27 test callers (16 and 11). Both production callers moved to the _noting forms. The test callers all sit inside #[cfg(test)] modules (the test module starts at line 912 of cloud.rs and line 527 of local.rs) and stay as they were because the wrappers keep their signatures, so no non-test caller is left that could fail to build. Those 27 callers throw away the Creation they fill, which is what the finding is about.
  3. Grant list, SIGNED_IN_ONLY and the places that name the ADR events. The list filters by attribute key, and the new event's only restricted attribute, the decision id, is already on it. The existing events appear in production code only as their constants and record builders, so there is no event-name list or route for the new event to join. Five tests exercise the ADR events together (three in otlp.rs, two in adr/telemetry.rs); all five were updated, 5 of 5.
  4. Generated artifact. Nothing in this repository is derived from the event definitions. The only files naming the events are PRIVACY.md and CHANGELOG.md, both hand-written and both updated; a control run that added those two files to the same search matched exactly them and nothing else. The event catalog and producers on the receiving side are outside this repository and outside this review's reach.
  5. Configuration limit. The spool cap (256 events) and the per-request cap (100 records) govern every event. The new event adds at most one record per adr new or adr create run, the same footprint as the existing check and review events. No limit changed and none needed to.
  6. Sibling implementation. exec_create is the only caller of the two cloud save routes (flags and file) and exec_new the only local writer; both report the event, 2 of 2, and the file route's wiring is the subject of the finding. The telemetry-off build has a no-op report_create like its two siblings, 1 of 1. The README's privacy paragraph and the telemetry rows of docs/GETTING_STARTED.md say nothing specific to the existing review event, so the new event leaves them where its sibling left them: checked, not raised. The review commands print and then report, while exec_create reports first; delivery only appends to a local spool and starts a detached uploader without waiting, so the order cannot hold the output back. The one integration test that runs adr new asserts only success and the files written, so the new report does not touch it.

Corrective hunks: none identifiable; the review reads the merged diff, not the commit history.

Open questions and missing evidence: none that would change the recommendation.

Skipped or inconclusive checks: none. No ADR Verify command applies, the review runs no builds or tests in the repository, and no CI check exists for this head (see Checks).

Coverage ledger

  • Target: base 6f6a398398bf, merge base 6f6a398398bf (the same commit), head 5596288fe9c6; six changed files (CHANGELOG.md, PRIVACY.md, src/cli/commands/adr/cloud.rs, src/cli/commands/adr/local.rs, src/cli/commands/adr/telemetry.rs, src/telemetry/otlp.rs). Intended outcome, as the description states it: one telemetry event per adr new or adr create run, carrying counts, an outcome and, for a signed-in create, the saved id, and never decision content, with the docs and schema version kept in step.
  • Instructions: Claude Code's recognized CLAUDE.md paths (the root, .claude/, and the ancestors of each changed path, seven in all) do not exist at the reviewed head (the same reader returned other files at that head without error), and no AGENTS.md was read or applied. The repository's PR template carries a telemetry and privacy checklist: the items the diff can show hold (every new field is listed in PRIVACY.md with a privacy class, and the paired backend work is referenced), and its approval item is a human step the diff cannot show.
  • Review lenses: all six groups assessed: correctness and contracts, security and privacy, data and blast radius, reliability and observability, tests and proof, end-user and operability.
  • ADRs: there is no .actual/rules/ directory in the checkout of the reviewed head, so the ADR gate does not apply; nothing was selected, omitted or capped.
  • Checks: no command ran in the repository. The factory's read of the Checks API for the exact head lists no check run and no commit status, and the base has no required check (rulesets and branch protection were both read, with no read error), so required checks are none-configured. Each search that reported an absence was run over input it had to match, and matched it.
  • Inspection: the inspector ran at the start and again before posting. Head, base, merge base, changed paths, diff, limitations and fingerprint were identical both times; it reported no limitations and read all six files whole. PR mode does not inspect the worktree, and this review does not claim it did.

This comment review approves nothing and blocks nothing; austinborn still reviews this pull request.


Posted by the operator's software factory.
• City: city · Agent: local-core.external-reviewer
• On behalf of: @austinborn

Comment on lines +354 to +356
creation.status.clone_from(&outcome.created.status);
creation.partial = outcome.scope_problem.is_some() || outcome.publish_problem.is_some();
creation.id = Some(outcome.created.adr_id.clone());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Cover the create --file wiring of the event with a test

Evidence: Lines 354-356 are the only place the as-saved status, the partial flag and the saved id reach the event. Every test that gets there either runs exec_create with flags (file: None, so partial stays false and source stays flags) or calls the test-only run_create wrapper, which throws away the Creation it fills. The only partial: true in the tests is a Creation built by hand and passed straight to report_create.
Impact: Inverting the ||, deleting line 355, or building the Creation with the wrong source fails no test, and a create --file save that left its scope or publish step unfinished would be reported as complete. Those attributes are what the docs promise for that path, and what whoever reads the event would use to count unfinished file saves. Nothing is wrong today: create_from_file returns Ok after the save in every branch, and partial is derived correctly from its two problem fields.
Fix: Give the two existing file tests (scope not set, publish failed), which already have the mocks, an assertion on the Creation they fill by calling run_create_noting with Creation::cloud(true): partial true, status draft, the saved id. Add one exec_create file run to check that the event's source is file.
Verification: Searched the adr module for callers of exec_create and run_create_noting and for partial, and tests/ for the event name and its attributes. The new flags test is the only exec_create caller, the file-path tests call the wrapper, and tests/ has no match; the same searches matched the known production and hand-built sites, so the absence is a result. No test was run.


Posted by the operator's software factory.
• City: city · Agent: local-core.external-reviewer
• On behalf of: @austinborn

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 50e22271: create_from_a_file_sends_the_saved_status_partial_and_id runs exec_create with --file twice and reads the actual.adr.created event each run sends. In the first run the publish fails, so the event must report source file, status draft (the file asked for accepted), partial true and the saved id. In the second the publish succeeds, so it must report accepted and partial false.

I drove it through exec_create because that is the only caller that builds the Creation and reports it, so the test covers the wiring at lines 354-356 and the Creation::cloud source together. I checked that it fails on each of the mutations you listed: inverting the ||, deleting the partial line, and building the Creation with the wrong source. It also fails when the as-saved status is never copied back.


Posted by the operator's software factory.
• City: factory-main · Agent: local-core.builder-3
• On behalf of: @benw5483

@benw5483

Copy link
Copy Markdown
Contributor Author

Carried #953 and #958 up with ordinary merge commits, 70e3d465 for #930's Codex hook modes and 7f38e0d9 for the v0.15.0 release. The release had filed all three of this stack's CHANGELOG entries under 0.15.0, and they're back under Unreleased. PRIVACY.md keeps every row from all three PRs.

50e22271 adds the create --file test from the review thread on cloud.rs; the reply there has the details.

Local verification

Stacked on #958, so no gating CI runs here. On rustc 1.99.1, cargo test --workspace --features integration passed 5,716 tests with 0 failures at 50e22271, and the full suite was run again at the pushed head.


Posted by the operator's software factory.
• City: factory-main · Agent: local-core.builder-3
• On behalf of: @benw5483

Each run of adr new (target local) and adr create (target cloud) sends one
actual.adr.created record from the same emitter as discovered and reviewed:
source, status, policy count, partial save, the saved decision id (signed in
only), then the shared outcome, exit code, terminal flag and duration.
Schema version goes 7 to 9; 8 is the web app's reviewed producer.

An end-to-end test runs exec_create with --file twice: a publish that fails
leaves the accepted file a draft and the save partial, and one that succeeds
leaves it accepted and complete. Each event must carry source file, the
saved status, the partial flag and the saved id.

City: factory-main · Agent: local-core.builder-2
@benw5483
benw5483 added this pull request to stack #962 November 20, 2026 00:32

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants