Skip to content

feat(stack-stripe): --webhook-endpoint flag creates and persists whsec#4

Open
masonwyatt23 wants to merge 34 commits into
mainfrom
feat/stack-add-stripe-webhook
Open

feat(stack-stripe): --webhook-endpoint flag creates and persists whsec#4
masonwyatt23 wants to merge 34 commits into
mainfrom
feat/stack-add-stripe-webhook

Conversation

@masonwyatt23

Copy link
Copy Markdown
Member

Summary

  • stack add stripe --webhook-endpoint <url> [--events <e1,e2,...>] now calls POST /v1/webhook_endpoints, captures the whsec_… signing secret, and stores STRIPE_WEBHOOK_SECRET + STRIPE_WEBHOOK_ENDPOINT_ID in Phantom — fully agent-driveable with no Stripe dashboard copy-paste.
  • Default event list (when --events is omitted): customer.subscription.{created,updated,deleted,trial_will_end} + invoice.payment_failed.
  • --secret-key-from-vault skips the interactive sk_… paste and reuses the existing vault entry.
  • ProviderContext gains an optional hints field so providers can inspect CLI flags during login() (not only during provision()), enabling the vault-bypass path.
  • Plain stack add stripe (no webhook flag) is unchanged — regression-guarded by test (f).

Files changed

  • packages/core/src/providers/stripe.ts — rewritten from makeApiKeyProvider factory to a hand-written Provider with full webhook flow
  • packages/core/src/providers/_base.tsProviderContext.hints?: Record<string,unknown> added
  • packages/core/src/pipeline.ts — threads opts.hints into ctx so login() can read it
  • packages/cli/src/commands/add.ts--webhook-endpoint, --events, --secret-key-from-vault flags + buildHints() helper
  • packages/core/src/__tests__/stripe-webhook.test.ts — 8 new tests (all mock fetch, never hit real Stripe)
  • README.md — doc snippet showing all three invocation forms

Test plan

  • bun test in packages/core — 265 pass, 0 fail (includes 8 new stripe-webhook tests)
  • bun run typecheck at repo root — clean
  • Test (a): successful creation stores all 3 secrets in vault
  • Test (b): custom --events forwarded to Stripe POST body
  • Test (c): default events used when --events omitted
  • Test (d): --secret-key-from-vault throws STRIPE_AUTH_REQUIRED when vault empty
  • Test (e): Stripe 400 propagates as STRIPE_WEBHOOK_CREATE_FAILED
  • Test (f): plain stack add stripe flow (no webhook) unchanged

🤖 Generated with Claude Code

stack add stripe --webhook-endpoint <url> [--events <e1,e2>] now:
  - reuses or interactively pastes sk_live_… (or reads from vault via
    --secret-key-from-vault)
  - POSTs to /v1/webhook_endpoints using the Stripe secret key
  - captures the whsec_… signing secret and we_… endpoint ID
  - stores STRIPE_WEBHOOK_SECRET + STRIPE_WEBHOOK_ENDPOINT_ID in Phantom
    so a future rotate command can update the same endpoint

Default event list is the subscription-lifecycle set (5 events). Custom
events passed via --events override it.

ProviderContext gains an optional `hints` field so providers can read
CLI flags during login(), not only during provision().

8 new tests cover: successful creation, custom events, default events,
missing-vault error path, Stripe API error propagation, and regression
guard for the plain sk_… flow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@vercel

vercel Bot commented Apr 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
ashlr-stack Error Error Jun 29, 2026 8:17pm

Request Review

Provider helpers (_helpers/cloudflare/turso) resolve secret slots via @ashlr/config resolveSecret (phantom-first → env). Build green (67 pages), 360 tests pass. Platform-layer adoption.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- withTimeout() wraps each pipeline step (login/provision/materialize)
  in Promise.race against an AbortController-driven timer
- DEFAULT_STEP_TIMEOUT_MS = 30s; configurable via AddServiceOpts.timeoutMs
- timeoutMs: 0 disables the timeout (Infinity sentinel, skips setTimeout
  entirely to avoid 32-bit overflow)
- AbortSignal threaded into ProviderContext so providers can self-cancel
- Timeout rollback: secrets/MCP cleaned up, deprovision called, original
  PROVISION_TIMEOUT error re-thrown (not wrapped as ADD_SERVICE_ROLLED_BACK)
- 6 new tests: hanging login/provision/materialize, rollback with deprovision,
  timeout=0 disabled, happy path with default timeout

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- OrchestrationGroup type: declare providers with explicit deps
- Topological sort with cycle detection (Kahn's algorithm)
- provisionGroup: parallel where independent, sequential where dep-ordered
- Output threading: provisioned outputs available to dependents via hints.resolvedOutputs
- Reuses existing withTimeout/cancellation guards per step
- Additive: single-provider provisioning unchanged
- Tests: ordering, cycle detection, parallel execution, output threading (381 pass, 0 fail)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
masonwyatt23 and others added 2 commits June 29, 2026 01:31
- packages/core/src/provision-schema.ts (new): JSON Schema-based validator,
  dot-notation path resolver, Resource extractor, TS codegen, and schema
  completeness checker. Built-in schemas for supabase, neon, vercel,
  stripe, and github. Side-effect-free: runs validateProvisionResponse()
  before materialize() to catch malformed API responses early.

- packages/core/src/index.ts: export all public provision-schema symbols.

- packages/cli/src/commands/validate.ts (new): `stack validate-schema
  <provider>` CLI command. Supports --all (43-provider completeness sweep),
  --mock (live-shaped mock validation), --codegen (TS type emit), --json.

- packages/cli/src/index.ts: register validate-schema subcommand.

- packages/core/src/__tests__/provision-schema.test.ts (new): 59 tests
  covering validateSchema, resolvePath, all five built-in provider schemas,
  validateProvisionResponse (valid + error paths), validateSchemaCompleteness,
  generateTypeScript codegen, custom schema registration, and an integration
  matrix comparing codegen output against live API mocks.

All 440 tests pass (0 failures).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add explicit healthcheck() with ctx.signal + latency measurement to all
  32 providers that previously relied on makeApiKeyProvider's generic default
  (anthropic, openai, xai, deepseek, replicate, braintrust, railway, render,
  fly, digitalocean, gcp, hetzner, clerk, auth0, workos, datadog, grafana,
  posthog, mixpanel, plausible, resend, sendgrid, postmark, mailgun, linear,
  launchdarkly, modal, upstash, firebase, convex)
- Structural-only providers (gcp, firebase, convex, modal) get shape-check
  healthchecks with latencyMs; grafana adds live /api/health when GRAFANA_URL
  is present, falls back to token shape check otherwise
- All healthchecks: missing-secret → error, HTTP 4xx → error, abort signal
  propagated via ctx.signal on every network call
- Add packages/core/src/__tests__/healthcheck-all-providers.test.ts with 115
  tests covering ok/missing-secret/401/signal-cancel for all providers
- Add --coverage flag to `stack doctor` to report healthcheck coverage % and
  list providers without a healthcheck method

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
masonwyatt23 and others added 2 commits June 29, 2026 01:55
Add deprovision() to github, stripe, sentry, cloudflare, railway, fly,
firebase, upstash, aws, and gcp (10 providers that lacked it). The
existing supabase/neon/turso/vercel implementations are unchanged.

Provider semantics:
- Providers that create real upstream resources (stripe webhooks, sentry
  projects, github repos) do a GET-before-DELETE with idempotent 404
  handling and throw StackError with manual-cleanup instructions on
  permission errors.
- Providers that only attach to existing accounts (cloudflare, railway,
  fly, upstash, aws, gcp, firebase) validate credentials are still
  active then return a no-op — no upstream resource was created.

All 11 deprovision implementations:
1. Accept ctx.signal and throw *_DEPROVISION_ABORTED on cancellation.
2. Are idempotent — 404 on any check or delete is treated as success.
3. Throw StackError with a clear message + dashboard URL on failure.

Tests (15 new test files, pipeline-recovery extended):
- deprovision-{github,stripe,sentry,cloudflare,railway,fly,firebase,
  upstash,aws,gcp,supabase,neon,turso,vercel}.test.ts covering
  success, already-deleted (idempotent), permission-denied, and
  signal-abort scenarios.
- pipeline-recovery.test.ts: three new cases — deprovision called on
  materialize failure, teardown failure surfaced in error message, and
  multi-provider orchestration cleanup.

636 tests pass, 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add packages/core/src/dry-run.ts: dry-run engine with synthetic resource
  mocks, [REDACTED] secret materialization, MCP entry previews, schema
  validation integration, and a static cost registry for 20+ providers
- Add packages/core/src/__tests__/dry-run.test.ts: 55 tests covering all
  engine paths including security invariants (no real credentials in output)
- packages/core/src/pipeline.ts: addService() short-circuits to dryRunAddService()
  when opts.dryRun is true; adds dryRun and costEstimate opts; no Phantom
  required in dry-run path
- packages/cli/src/commands/add.ts: --dry-run now renders the full engine
  report (table + secret keys + MCP preview) instead of a static step list
- packages/cli/src/commands/apply.ts: adds --dry-run, --cost-estimate, and
  --json flags; dry-run path calls dryRunProviders() and renders a structured
  report or machine-readable JSON without touching Phantom/config
- packages/core/package.json: expose ./dry-run sub-path export
- packages/core/src/index.ts: re-export all dry-run types and functions

691 tests pass, 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
masonwyatt23 and others added 2 commits June 29, 2026 02:28
- Add 31 new test cases to healthcheck-all-providers.test.ts covering 9
  previously untested providers: aws, cloudflare, github, neon, sentry,
  stripe, supabase, turso, vercel
- Each provider gets: ok (200 + latencyMs), missing-secret (error +
  'missing'), auth-failure (401/403/404 → error), and where applicable
  no-resource_id (warn) and signal-cancellation (error) paths
- Fix supabase healthcheck to thread ctx.signal into fetchWithRetry so
  AbortSignal cancellation works correctly (was silently ignored)
- Test count: 691 → 722 (all pass); no existing tests weakened or deleted

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…linear

- render: DELETE /v1/services/{id}; 404=idempotent, errors log+no-throw
- convex: structural deploy-key validation; no upstream resource to delete
- posthog: DELETE /api/projects/{id}; 404=idempotent, errors log+no-throw
- clerk: validate JWKS endpoint still active; throws on 401/403/5xx
- auth0: verify OIDC discovery reachable; non-ok/network errors log+no-throw
- linear: validate GraphQL viewer query; throws on 401/403/5xx
- 6 test files (29 new tests); all 107 deprovision tests green, 751 total

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
masonwyatt23 and others added 2 commits June 29, 2026 02:41
- Wire validateProvisionResponse() into addService pipeline after
  provider.provision() returns (pipeline.ts line ~195). Throws
  StackError('PROVISION_SCHEMA_MISMATCH', ...) with detail on failure;
  attempts best-effort deprovision before throwing.

- Add JSON Schema + field mappings for all 34 remaining providers
  (turso, convex, railway, fly, cloudflare, render, firebase, upstash,
  openai, anthropic, xai, deepseek, replicate, braintrust, modal,
  posthog, sentry, linear, resend, sendgrid, mailgun, postmark, clerk,
  aws, auth0, datadog, digitalocean, gcp, grafana, hetzner,
  launchdarkly, mixpanel, plausible, workos). All 39 providers now have
  registered schemas.

- Add codegen script (scripts/gen-provider-types.ts) and run it to
  produce 39 typed TS interfaces under packages/core/generated/.

- Add 26 new test cases in provision-schema.test.ts covering:
  (1) happy path — schema passes, resource coerced correctly
  (2) missing required field — throws ProvisionSchemaValidationError
  (3) malformed response (null, array, wrong type) — throws with detail
  (4) all 39 providers have registered schemas
  (5) codegen determinism + correct output for new providers

Tests: 777 pass, 0 fail (was 751 before this change)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add packages/core/src/instrumentation.ts with Instrumentation class,
  MemoryCollector, FileCollector, and typed event shapes (StepEvent,
  RollbackEvent, PartialFailureEvent, OrchestrationStepEvent)
- Wire instrumentation.recordStep() after every withTimeout() call in
  pipeline.ts (login, provision, materialize, deprovision) with latency ms
- Emit rollback events (cleaned/failed items + recovery suggestion) on
  atomic rollback paths; emit partial_failure events when provider has no
  deprovision support
- Export all instrumentation types from packages/core/src/index.ts
- Add `stack add --trace <file>` flag: attaches FileCollector before
  addService, flushes JSON audit trail to file on success or failure
- Update doctor command to always display measured latency per provider
  (falls back to wall-clock if healthcheck does not report latencyMs)
- Add 17 tests in packages/core/src/__tests__/instrumentation.test.ts
  covering: step event shape/timestamps/latencies, rollback events with
  provider/resource_id/reason, partial-failure recovery suggestions,
  multi-provider orchestration audit trail, FileCollector flush

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
masonwyatt23 and others added 2 commits June 29, 2026 04:16
…-rate telemetry

Adds packages/core/src/probes/ module with 10 opt-in provider probes that
extend doctor baseline checks to measure quotas, rate-limits, and billing
burn-rate. Wires into `stack doctor --probes` via runProbes().

Probes implemented:
- GitHub: API rate-limit remaining (requests/hr ceiling, utilization %)
- OpenAI: token quota + daily spend USD (billing endpoint, rate-limit headers)
- Anthropic: token quota (anthropic-ratelimit-* headers)
- Stripe: failed chargebacks (open disputes) + rate-limit headers
- Vercel: build minutes + function invocations vs Hobby plan limits
- Supabase: db size bytes vs free-tier limit (management API)
- Neon: compute unit burn-rate across branches (free-tier 191.9h/month)
- Firebase: Realtime DB connection count vs 100-connection free limit
- Clerk: MAU utilization vs 10k free-tier limit
- Linear: issue sync GraphQL rate-limit (X-RateLimit-* headers)
- AWS: EC2 on-demand quota via ServiceQuotas API (SigV4-signed, no SDK dep)

Infrastructure:
- histogram.ts: 7-day rolling sample store at .stack/telemetry/health-probes.json
  (separate from privacy-sensitive telemetry); auto-prunes samples > 7 days
- computePercentiles(): p50/p95 latency from stored samples
- runProbes(): concurrent execution, histogram persistence, alertCount summary
- All probes: return status=skipped (not error) when credentials absent

Tests (58 assertions across 14 describe blocks):
- Quota parsing and alert threshold detection per provider
- warn at >= 80% utilization, error at >= 95%
- Histogram append, pruning, and percentile computation
- ProbeRunSummary alertCount, p50Ms/p95Ms enrichment
- Env-var isolation for skipped-credential tests

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…port

- Add generateOpenApiResource(providerName, schema) → emits
  #/components/schemas/ProvisionResponse_<Provider> fragment with
  x-stack-mapping extension; supports OpenAPI 3.1 type arrays + nullable

- Add generateTerraformResource(providerName, schema) → emits valid
  resource "<provider>_stack_provisioned" HCL block with required fields
  as '<required>' placeholders and optional fields commented out

- Add runCodegenForAllProviders(opts) → batch runner that writes OpenAPI
  fragments to packages/site/public/schemas/ and HCL docs to
  packages/site/public/terraform/ at build time; dryRun mode for tests

- Add packages/core/src/__tests__/codegen.test.ts with 133 tests covering:
  OpenAPI schema structure, Terraform HCL syntax validity (balanced braces,
  resource type convention, AUTO-GENERATED header), round-trip
  schema→JSON→parse integrity for all 39 providers, and dry-run runner
  path/count assertions

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
masonwyatt23 and others added 2 commits June 29, 2026 04:29
…cy ordering

- Add packages/core/src/orchestration/rollback-graph.ts with:
  - buildRollbackGraph(entries, failurePoint?, provisionOrder?) → RollbackGraph
  - computeRollbackPlan(graph) → RollbackPlan (reverse-topological wave schedule)
  - buildRollbackPlan(entries, provisionOrder, failurePoint?) convenience helper
  - executeRollbackPlan(plan, opts) → RollbackTranscript (wave-parallel execution)
  - Full type exports: RollbackGraph, RollbackPlan, RollbackWave, RollbackStepResult,
    RollbackTranscript, DeprovisionFn, ExecuteRollbackPlanOpts

- Update orchestration.ts:
  - Re-export all rollback types and functions
  - Add OrchestrationRollbackFn type for cross-provider coordination callbacks
  - Add onRollback to OrchestrationGroupDefaults and OrchestrationEntry.opts
  - runOrchestrationGroup: use Promise.allSettled per wave; on failure compute
    rollback plan scoped to successfully-provisioned providers and execute in
    reverse topological order; emit detailed ORCHESTRATION_PARTIAL_FAILURE error
    with full recovery transcript

- Update index.ts to re-export all new rollback symbols

- Add packages/core/src/__tests__/rollback-ordering.test.ts (38 tests):
  - buildRollbackGraph: scope, edge exclusion, failurePoint trimming
  - computeRollbackPlan: empty graph, linear, diamond, independent-parallel
  - buildRollbackPlan: stripe webhook chain, supabase/clerk with failurePoint
  - executeRollbackPlan: success, failure isolation, skip, abort signal,
    transcript fields, recovery suggestion text
  - 12-provider realistic scenario (Stripe→webhook/portal, Supabase→Clerk→auth,
    email, CDN, analytics): provision order, rollback order, full execution,
    failurePoint scoping, partial-failure state isolation, parallel waves
  - runOrchestrationGroup integration: ORCHESTRATION_PARTIAL_FAILURE code,
    onRollback callback ordering, successful group unaffected

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…replay CLI

- Add packages/core/src/errors/provision-errors.ts: 14 error codes
  (AUTH_EXPIRED, QUOTA_EXCEEDED, RATE_LIMITED, RESOURCE_CONFLICT,
  API_DEGRADED, INVALID_SCHEMA_RESPONSE, TIMEOUT, NETWORK_ERROR, etc.),
  classifyError(), buildHints(), captureProvisionError(), sanitizeBody(),
  buildReplayRecord/save/load, loadProvisionErrorReport, printProvisionError
- Update pipeline.ts: emit captureProvisionError + saveReplayRecord on
  login, provision, and materialize failure paths
- Add packages/cli/src/commands/replay.ts: stack replay <error-id> command
  re-runs failed provider with fresh auth for transient errors or manual
  remediation prompt for non-transient errors
- Wire replayCommand into CLI index.ts subCommands map
- Export all new types/functions from core/src/index.ts
- Add errors/provision-errors export path to core/package.json
- Add packages/core/src/__tests__/provision-errors.test.ts: 52 tests
  covering 15+ error scenarios, hint accuracy, sanitizeBody, report
  persistence, replay record round-trip idempotence

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add ApiKeyDeprovisionSpec interface + auto-wire deprovision() in makeApiKeyProvider
  (_api-key.ts): calls spec.deprovision.cleanup if defined, else logs a structured
  no-op with resourceId, provider name, reason, and docsUrl
- Add deprovision spec to all 19 API-key providers (anthropic, braintrust, datadog,
  deepseek, digitalocean, grafana, hetzner, launchdarkly, mailgun, mixpanel, modal,
  openai, plausible, postmark, replicate, resend, sendgrid, workos, xai)
- Datadog gets real cleanup: DELETE /api/v2/api_keys/:id using DD-APPLICATION-KEY
  when present; 404/204/200 are idempotent; missing app key or 'default' resourceId
  skips gracefully; network errors are logged as warn not thrown
- All other 18 providers: graceful no-op log with docs link for manual key revocation
- Add deprovision-api-key-suite.test.ts with 8 tests covering: (a) successful
  Datadog cleanup with 204 response, (b) all 18 no-op providers resolve cleanly,
  (c) Datadog 403 throws DATADOG_DEPROVISION_FAILED, (d) rollback transcript log
  fields, (e) partial failure across orchestration group; plus edge cases for
  missing app key, default resourceId, and 404 idempotency

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
masonwyatt23 and others added 2 commits June 29, 2026 05:21
- Add packages/core/src/quota-engine.ts with QuotaSnapshot interface,
  QuotaEngine class (p50/p95 latency, 7-day burn trends, OLS slope,
  alert detection at >=75% utilization or budget breach), buildForecast
  and probeResultToSnapshot convenience helpers
- Add packages/cli/src/commands/quota-forecast.ts — new 'stack quota-forecast'
  command running all 11 built-in probes in parallel, emits human-readable
  dashboard (utilization bar, spend forecast, provider breakdown, top
  spenders, at-risk list) or --json; accepts --budget <USD>
- Wire quota-forecast into CLI subcommands index
- Export QuotaEngine, buildForecast, probeResultToSnapshot, runProbes and
  related types from @ashlr/stack-core public index
- Add 39 tests in packages/core/src/__tests__/quota-engine.test.ts covering
  multi-provider aggregation, rolling-window computation, OLS slope
  direction, threshold detection (utilization + budget), buildForecast
  skipped-probe exclusion, and full QuotaForecast shape validation

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rage, CI gate

- Add ProbeRegistry class to packages/core/src/probes/index.ts mapping
  provider names → Probe instances with has/get/registeredNames/
  missingProviders/coverageStats methods
- Add generateProbeStub(providerName) emitting a TypeScript skeleton probe
  with TODO comments for quota endpoint URL + parsing logic
- Add writeProbeStub(providerName, outputDir) for file-based generation
- Export ProbeRegistry, defaultProbeRegistry, generateProbeStub, writeProbeStub
  from packages/core/src/index.ts
- Add packages/cli/src/commands/probes.ts with subcommands:
    stack probes generate-missing [--dry-run] [--output-dir ./generated]
    stack probes coverage
- Register `probes` in packages/cli/src/index.ts subCommands map
- Update stack doctor --coverage to also report probe coverage % alongside
  healthcheck adapter coverage (packages/cli/src/commands/doctor.ts)
- Add CI gate in .github/workflows/ci.yml: enforces probe coverage floor of
  11/39; fails when new providers are added without stubs
- Add packages/core/src/__tests__/probe-registry.test.ts (49 tests):
  construction, get/has lookups, missingProviders, coverageStats math,
  generateProbeStub output shape, writeProbeStub file writing, CI gate

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ation

Add comprehensive integration test suite covering four cross-provider scenarios:

1. Three-service dependency chain (supabase → clerk → stripe-webhook):
   - Validates provision order, rollback wave structure, reverse-topo execution
   - Pre-flight wave report shows which providers roll back in which waves
   - Deprovision failure in one wave does not block later waves
   - Full end-to-end via runOrchestrationGroup with onRollback callbacks

2. Partial-failure recovery — dangling resource tracking (doctor --reconcile):
   - requiresManualCleanup acts as the doctor --reconcile list
   - Multiple dangling resources all surfaced correctly
   - No secrets leak to Phantom vault when materialize() throws mid-chain
   - recoverySuggestion directs to stack doctor --fix for dangling resources

3. Provision timeout + abort cascading without secret leaks:
   - Hanging provision and materialize both produce PROVISION_TIMEOUT
   - AbortSignal prevents downstream rollback waves from starting
   - Phantom vault remains empty after timeout-triggered rollback

4. Quota consensus engine — 7-day spend-rate trend tracking:
   - Flat/accelerating/decelerating burn slopes verified via OLS
   - Multi-provider aggregate totals, monthly spend, topSpenders ordering
   - Utilization and burn alerts fire correctly for multi-provider stacks
   - sampleCount matches days of data, zero-burn excluded from topSpenders

Also exercises RollbackPlan pre-flight validation: wave assignments for
diamond, linear, and mixed dependency graphs verified before provisioning.

All 35 tests pass; no production code changes; fake Phantom harness used
for all secret-isolation assertions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implement 27 new ProbeResult adapters to achieve 100% coverage of the
39-provider catalog (up from 11/39). Each probe:
- Implements the Probe interface with ProbeResult fields (quotaUtilization,
  rateLimitCeiling, latencyMs, estimatedDailyBurnUSD where applicable)
- Makes a real API call to validate credentials and extract quota/rate-limit
  response headers
- Handles 401/403 auth errors, 429 rate-limit responses, and network failures
- Returns status: 'skipped' when credentials are absent from vault

New probes: braintrust, cloudflare, convex, datadog, deepseek, digitalocean,
fly, gcp, grafana, hetzner, launchdarkly, mailgun, mixpanel, modal, plausible,
posthog, postmark, railway, render, replicate, resend, sendgrid, sentry,
turso, upstash, workos, xai

Updated packages/core/src/probes/index.ts to register all 39 probes in
BUILTIN_PROBES. Added 120+ integration tests across probe-registry.test.ts
and probes.test.ts covering skip/ok/warn/error paths for each new provider.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
masonwyatt23 and others added 2 commits June 29, 2026 06:37
- Add ResourceConflictCheckConfig interface and ConflictCheckOpts to _base.ts
- Add optional Provider.checkConflict() method to base Provider interface
- Implement resource-conflict.ts with runConflictCheck(), buildConflictCheckMeta(),
  buildConflictCheckTelemetry(), generateUniqueName(), and defaultCiPrompt()
- Three outcomes: attach to existing (user confirmed / CI), auto-rename with
  unique suffix, or fail with explicit guidance
- checkConflict() implemented on: neon, supabase, vercel, turso, railway
  (each hits provider list/get endpoint before provision())
- Wire into pipeline.ts before provision(); conflict meta stored in
  [services.SERVICE_NAME.meta.conflictCheck] in .stack.local.toml
- Add checkConflicts, ciConflictStrategy, conflictPrompt opts to AddServiceOpts
- Unreachable/failed checks are non-blocking; dry-run skips checks
- Export all new public types from core index
- Add 67 tests covering: collision auto-recover (5+ providers), prompt
  acceptance/rejection, unreachable non-blocking, dry-run skip, telemetry
  shape, security invariants, provider interface conformance

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add SchemaValidator class with compile/cache/invalidate/clear API
- Add ProvisionResponseSchemaWithVersion interface (schemaVersion field)
- Add runValidateOnly() + formatValidateOnlyResults() for --validate-only CLI mode
- Add module-level schemaValidator singleton
- Extend Provider interface with schemaVersion + provisionResponseSchema fields
- Export all new symbols from packages/core/src/index.ts
- Add provision-schema-validation.test.ts: 76 tests covering
  (a) happy-path for 10 diverse providers
  (b) missing required field detection
  (c) type mismatch (string vs number, float vs integer)
  (d) nested object validation with deep path reporting
  (e) validator caching (compile, invalidate, clear, size, cachedProviders)
  (f) --validate-only mode (runValidateOnly, formatValidateOnlyResults)
  (g) ProvisionResponseSchemaWithVersion round-trip
  (h) formatValidateOnlyResults output shape

Zero breaking changes; all 161 provision-schema tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add ProviderMcpBroker (packages/core/src/provider-mcp-broker.ts) with:
  - MCP call definitions for 6 providers: Stripe, Vercel, GitHub, Anthropic, OpenAI, AWS
  - Per-call timeout (default 3 s) with automatic fallback to static registry
  - Session-scoped cache (default 60 s TTL) keyed by provider name
  - getCostEstimateWithFallback() integrating live + static in one call
  - Session singleton (getSessionBroker/setSessionBroker/resetSessionBroker)

- Update dry-run.ts to use dynamic pricing:
  - New livePricing + pricingBroker opts on DryRunProviderOpts
  - dryRunProviders shares a single broker across batch calls for cache efficiency
  - formatDryRunReport shows [live] vs [estimated] badges on cost lines

- Add --live-pricing flag to `stack recommend`:
  - Enriches each output hit with costBadge, costNotes, costMonthlyUsd
  - pricingSource field on JSON output (live/estimated/none)
  - Human output renders cost line with badge

- Add stack_pricing_lookup MCP tool to packages/mcp/src/server.ts:
  - Handled in-process via ProviderMcpBroker (no CLI spawn)
  - Accepts comma-separated provider list; defaults to all 6
  - Returns JSON with per-provider pricing, badge, and pricingSource

- Add packages/mcp/package.json dep: @ashlr/stack-core workspace:*

- Add 36 tests in provider-mcp-broker.test.ts:
  - Fallback to static on MCP error and timeout
  - Cache hit (second call does not invoke MCP)
  - Cache TTL expiry triggers fresh call
  - Cache invalidation (single + all)
  - Live data returned correctly for all 6 providers
  - getCostEstimateWithFallback: live overrides static; static used on failure
  - Session singleton lifecycle
  - dry-run.ts integration: livePricing true/false, [live]/[estimated] badges

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
masonwyatt23 and others added 2 commits June 29, 2026 07:21
- Import and register probe-auth0 in BUILTIN_PROBES (38→39, full catalog coverage)
- probe-auth0 implements M2M client_credentials token exchange + /api/v2/stats/daily quota check
- Tests cover: skipped (missing domain/client_id/secret), happy path (2-call flow), token endpoint 401, missing access_token, stats 403, x-ratelimit header parsing, rate-limit exhaustion (>=95%), abort signal propagation
- Probe registry CI gate now passes at 39/39 (100% coverage)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… stack doctor --audit

Add packages/core/src/rollback-audit.ts (~300 LOC):
- auditRollback(): reads provision replay log, cross-checks claimed
  cleaned/failed items against live Phantom vault, .mcp.json, .stack.toml
- Detects orphan_secret, orphan_mcp, orphan_config, stale_failed, ghost_clean
- Emits RollbackAuditReport with verified/orphans/stale/recommendations
- auditAllSessions(): multi-session sweep for doctor --audit
- formatAuditReport(): human-readable output helper
- Export via packages/core/package.json ./rollback-audit sub-path

Add packages/core/src/__tests__/rollback-audit.test.ts (31 tests):
- Successful rollback: all cleaned items absent → verified=true
- Partial rollback: stale failed items remain → verified=false
- Orphaned state: secrets/MCP in vault/filesystem not in rollback event
- Cascade failures: ghost_clean for unverifiable upstream resources
- Audit from replay log alone (no rollback event)
- formatAuditReport output assertions
- Unknown/missing session graceful handling
- Recommendation shape invariants

Wire packages/cli/src/commands/doctor.ts --audit flag:
- New --audit boolean arg routes to runAudit()
- runAudit() calls auditAllSessions(), formats human/JSON output
- Exits 1 when dangling state found

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…viders

Extend checkConflict() from 5 providers to cover 11 additional deploy/database/auth
providers: render, fly, cloudflare, firebase, aws, convex, linear, clerk, auth0,
workos, digitalocean.

Each implementation:
- Calls the provider's list/describe API to check name/resource existence
- Returns ResourceConflictCheckConfig with exists, existingResourceId,
  suggestedUniqueName, and action (attach/rename/ok/unreachable)
- Returns action:unreachable on API failure (never blocks provisioning)

Provider-specific strategies:
- render: lists services filtered by name via /v1/services
- fly: lists apps via Machines API /v1/apps
- cloudflare: lists Workers scripts under the account
- firebase: structural check against service-account JSON project_id
- aws: STS GetCallerIdentity account id comparison
- convex: deploy key structural parse (<env>:<team>:<project>|<token>)
- linear: GraphQL teams { nodes } list
- clerk: /v1/instance application_name comparison
- auth0: OIDC discovery check on domain match
- workos: /organizations list by name
- digitalocean: App Platform /v2/apps list by spec.name

Also adds displayConflictCheckRecommendation() to provision-loop.ts to surface
attach/rename/unreachable actions to the CLI user during stack add.

Tests: 75 tests in checkConflict.test.ts covering happy path (name not found → ok),
name exists with id, API unavailable graceful skip, and no desiredName → ok for
all 11 providers plus interface conformance suite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add packages/core/src/permission-validator.ts (~400 lines):
  PermissionSet schema, scope detectors (GitHub X-OAuth-Scopes header,
  AWS key prefix, Stripe key type, Anthropic /v1/models, GCP token),
  findViolations/findMissingRequired pattern matchers, validatePermissions()
  entry point, auditPermissions() batch function, --fix remediation guidance.

- Add validatePermissions() optional method to Provider interface in
  packages/core/src/providers/_base.ts.

- Add permission fields to ProbeResult in packages/core/src/probes/types.ts:
  permissionStatus, permissionDetail, grantedScopes, permissionViolations.

- Export new module from packages/core/src/index.ts and add
  ./permission-validator and ./providers/_helpers export paths to
  packages/core/package.json.

- Add packages/cli/src/commands/audit-permissions.ts: new CLI command
  `stack audit-permissions [--fix] [--provider <name>] [--json]`.

- Integrate into packages/cli/src/commands/doctor.ts via
  --audit-permissions flag (delegates to audit-permissions command).

- Register `audit-permissions` subcommand in packages/cli/src/index.ts.

- Add packages/core/src/__tests__/permission-validator.test.ts: 24 tests
  covering GitHub/AWS/Stripe/Anthropic scope detection, overprivileged
  detection, forbidden scope errors, batch audit, --fix remediation, and
  PermissionSet structural sanity checks. All 24 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add ResourceLifecycle registry (packages/core/src/resource-lifecycle.ts)
  tracking resource_id, region, tier, created_at, last_verified_at per service
- Persist lifecycle metadata in .stack.local.toml [lifecycle.*] section
- Drift detection: deleted / degraded (region+tier) / stale / ok / unknown
- reconcileOne() + reconcileAll() with injectable LiveProbeMap for live validation
- Auto-patch region/tier drift and bump last_verified_at when apply=true
- Add `stack reconcile` command (packages/cli/src/commands/reconcile.ts)
  with --apply, --service, --json, --stale-days flags
- Wire reconcile into CLI subCommands map
- Add `stack doctor --drift` flag surfacing stale lifecycle records
- Export all types from @ashlr/stack-core and add ./resource-lifecycle export path
- 62 tests covering CRUD, seeding, all drift kinds, reconcile scenarios,
  provider-side deletion, region/tier downgrades, multi-resource conflicts,
  stale threshold edge cases, persistence round-trips

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add packages/core/src/provision-readiness.ts with ProviderReadinessRule
  registry (parallel to provision-compliance.ts): hard (blocking) + soft
  (advisory) rules, runReadinessChecks(), enforceReadiness(),
  validateAllProviders(), generateReadinessJson() codegen,
  ProviderNotReadyError. 8 built-in provider rule sets: Vercel (billing,
  team-exists, region quota), Stripe (restricted account, API version
  compatibility), Supabase (org existence, quota exceeded), Neon (free-tier
  quota, region format), GitHub (org membership, rate limit), Anthropic
  (billing, key format, usage tier), OpenAI (key format, billing active,
  tier), AWS (quota, credentials, root warning).

- Integrate enforceReadiness() into pipeline.ts addService(): runs after
  auth/login but before provision(), so PROVIDER_NOT_READY errors surface
  with actionable remediation before any resource is created.

- Add packages/cli/src/commands/validate-readiness.ts: `stack validate`
  command checks all configured providers (or --all) without provisioning;
  supports --json, --codegen, single-provider targeting.

- Wire `validate` subcommand into packages/cli/src/index.ts.

- Add packages/core/src/__tests__/provision-readiness.test.ts: 99 tests
  covering registry, runReadinessChecks, enforceReadiness, validateAllProviders,
  codegen descriptors, ProviderNotReadyError, and all 8 built-in providers with
  real API scenarios (billing not enabled, restricted mode, quota exceeded, etc.).
  All 99 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…stry

scanProviderSchemaRegistry declared compliant[] but never pushed to it,
so the result always reported zero compliant providers. Push provider
name onto compliant[] on the schema-present success path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…name

- provider-graph: detectCycles no longer re-appends dep after path.slice,
  removing the duplicated start node in CIRCULAR_DEPENDENCY reports.
- permission-validator: Stripe live-key violation now uses real scope name
  "live_secret_key" instead of the literal glob "sk_live_*"; pattern and
  --fix remediation check updated to exact-match it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…iance modules

Heals a broken HEAD: committed code in index.ts, pipeline.ts, rollback-audit.ts
and cli/src/index.ts already imported symbols and modules that were left
uncommitted (ProvenanceCollector/checksumOutput/ProvisionProvenance in
instrumentation.ts, replay-log + ReplaySessionMeta exports in provision-errors.ts,
the provision-compliance module, graph-visualizer, audit-trail and resume commands).

Lands the production-source modifications plus the new source/support files and
their passing tests. Reduces tsc --noEmit errors from 127 to 77 and adds zero new
errors vs the prior HEAD baseline; full `bun run build` passes.

Excludes (left stashed for review): two broken WIP test files that add 19 type
errors (doctor-healthcheck-metrics.test.ts, deprovision-integration.test.ts) and
an unreferenced orphan module (orchestration/healthcheck-aggregator.ts).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant