Skip to content

Rewrite hubctl in Bun + TypeScript + Effect v4 (agent-first JSON envelope)#2

Open
jordangarrison wants to merge 86 commits into
mainfrom
rewrite/effect-v4
Open

Rewrite hubctl in Bun + TypeScript + Effect v4 (agent-first JSON envelope)#2
jordangarrison wants to merge 86 commits into
mainfrom
rewrite/effect-v4

Conversation

@jordangarrison

@jordangarrison jordangarrison commented Jun 24, 2026

Copy link
Copy Markdown
Owner

Summary

Reimplements hubctl from Ruby/Thor to Bun + TypeScript + Effect v4 as an agent-first CLI: every command emits a structured JSON envelope by default, with an optional --pretty human renderer over the same envelope. Full feature parity with the Ruby tool (minus the stubbed server command), distributed as a Bun-compiled binary and a Nix flake.

⚠️ BREAKING CHANGE — the Ruby implementation is removed and the --format table|json|list flag is dropped (the envelope subsumes it). See docs/adrs/000002-agent-first-json-envelope.md.

Architecture

Layered Effect app: thin cli/ command modules → centralized output/ envelope renderer → services/ domain services (Context.Service) → a single github/ Octokit wrapper → Bun platform. Typed errors flow in the E channel; services resolve via Layers. The envelope is always built; pretty mode is a renderer over it, so the two views can't drift.

Command surface (parity with the Ruby CLI)

version · auth · config <get|set|list|init|path> · repos <list|show|create|clone|archive|topics> · orgs <list|show|members|repos|teams|info> · users <show|whoami|list|invite|remove> · teams <list|show|create|members|add|remove> · enterprise <show|orgs|members|owners|billing|licenses|audit-log|sso|stats|security-analysis>

  • Root (no args) emits the full command tree as JSON for agent discovery.
  • next_actions (HATEOAS) on success; error.code/message + plain-language fix on failure.
  • Destructive ops require --yes in JSON mode (and prompt at a TTY).
  • NDJSON streaming for long operations (e.g. enterprise audit-log); ~50-item result truncation for context-window discipline.

How it was built & verified

  • TDD throughout — failing test → red → minimal impl → green → validate → commit, per task.
  • Multi-agent code review before cutover (per-group parity vs the Ruby source + correctness/security/test-quality, with adversarial verification): 10 Critical + 24 Important findings fixed (missing config/auth groups, orgs repos/teams/info, teams show, enterprise show, a teams list decode bug, clone exit-code handling, --org fallback, and more).
  • 32 deferred Minor findings tracked in docs/review-deferred-minor.md.

Robustness: decode hardening (no decode ever crashes the envelope)

Every GitHub payload was decoded with the throwing Schema.decodeUnknownSync, called inside Effect.map. A throw there is an uncaught defect — it escapes the typed E channel, so a real API response that didn't match the schema (a field absent on a list endpoint, a null where a string was assumed, a new/removed field) dumped a raw stack trace instead of the {ok,command,result,next_actions,error,fix} envelope. For an agent-first tool whose whole contract is "always emit a valid envelope," that's a contract violation. We hit four of these in orgs and hand-fixed them (95ae480); this makes the fix systemic.

  • Shared helper (src/schema/decode.ts): decode(schema, label) wraps the effectful Schema.decodeUnknownEffect into Effect<A, DecodeError> — a mismatch now fails in the typed E channel instead of throwing. DecodeError (a Data.TaggedError) carries code:"decode_error", a message naming the failing field/path, and a "this looks like a hubctl bug — please report it" fix.
  • Honest E channel, zero churn: DecodeError joined the GithubError union, so every Effect<…, GithubError> service signature gained it without edits. The existing emitOutput.fail path renders it as a standard ok:false envelope.
  • All decode sites converted across repos/orgs/users/teams/enterprise/auth (Effect.map(decode)Effect.flatMap(decode); data-first and inline-closure sites restructured to sequence the decode effectfully). The on-disk config file JSON parse is hardened the same way, so a corrupt config.json is a clean envelope, not a crash.
  • Schemas are already structurally lenientSchema.Struct ignores unknown keys in this Effect beta, so additive GitHub API changes never break decoding; only missing/null/wrong-type required fields do, and those now render cleanly.
  • Left as-is (intentional): the Schema.encodeSync sites in output/render.ts, output/service.ts, and the config write path — they encode hubctl's own post-decode, JSON-safe data, never external input.
  • TDD: each service got a test feeding a realistic-but-mismatched payload (one required field omitted/nulled — not an over-complete fixture, which is exactly what masked the original bugs) asserting a DecodeError in the E channel, plus a CLI-level regression asserting the full pipeline renders ok:false/decode_error. Verified end-to-end with a live-token smoke run across every command group — every output a single valid JSON envelope, success and error paths, zero stack traces.

Testing

  • bun run validate (format → lint → typecheck via tsgo → 269 tests → ast-grep): green
  • bun run test:e2e (compiled-binary smoke against a mock GitHub): green
  • nix build .#hubctl produces a working binary

Notes for reviewers

  • Nix packages a bun-wrapper over src/main.ts rather than the bun build --compile artifact: autoPatchelf rewrites the ELF and breaks Bun's detection of the bundle appended to the executable. The true standalone single-file binary still ships via bun run build:local / release artifacts.
  • Config moved ~/.hubctl.yml (YAML) → ~/.config/hubctl/config.json (JSON) — an intentional design decision.
  • users intentionally overlaps some org-membership routes with orgs (faithful to the Ruby users.rb).

Design for migrating hubctl from Ruby/Thor to Bun + TypeScript + Effect v4:
agent-first JSON envelope output with a --pretty human layer, Octokit wrapped
in an Effect service, full feature parity in a single cutover, Bun-compiled
binary distribution, and a floai-style validation/compilation toolchain
(tsgo, oxlint/oxfmt, @effect/language-service, ast-grep, vitest, runtime e2e).
Replace the Ruby dev environment with a Bun/TypeScript/Effect v4 toolchain:

- flake.nix: Bun + Node + git + (nixpkgs) ast-grep devShell; drop Ruby gem build
- package.json: bun scripts incl. validate chain (format/lint/typecheck/test/ast-grep)
- tsgo (@typescript/native-preview) + @effect/tsgo + @effect/language-service
  (all Effect diagnostics at error)
- oxlint (ultracite presets + @mpsuesser/oxlint-plugin-effect) and oxfmt
- vitest + @effect/vitest (threads pool); e2e config for compiled-binary smoke
- ast-grep structural-rule scaffold; commitlint; simple-git-hooks + nano-staged
- docs/effect-v4-api-notes.md: verified v4 CLI API (Flag/Argument/Command.runWith)

bun run validate is green. fallow and the layer-boundaries oxlint plugin are
deferred (see follow-ups).
fallow ships only a generic-glibc npm binary (NixOS can't run it) and isn't in
nixpkgs. Add an autoPatchelfHook derivation so it runs from the flake dev env
with no system dependency (no nix-ld). Remove the npm fallow dep so its broken
binary doesn't shadow the patched one on PATH. fallow stays a separate leg (not
in validate) until real source exists — on a bare scaffold it false-flags
tooling-only devDeps (@effect/vitest, nano-staged).
Fold in the 'flake + validation tooling first' feedback: Phase 0 is complete and
moved to the front. Replace the v3-shaped CLI snippets (Options/Args/Command.run)
with the verified v4 API (Flag/Argument/Command.runWith/BunServices.layer) per
docs/effect-v4-api-notes.md, and record the NixOS native-binary and worktree
git-hook deltas.
GET /orgs/{org}/teams (the list endpoint) does not return
members_count/repos_count — only the team detail endpoint does. The list
schema previously required them, so against real GitHub `teams list` threw
a decode defect (tests passed only because fixtures wrongly included the
fields). Make them Schema.optional and render '-' when absent, matching the
Ruby's `team[:members_count]` nil handling (lib/hubctl/teams.rb:18-31).
Add `hubctl teams show <team> --org <org>`, mirroring
lib/hubctl/teams.rb#show (id/name/slug/description/privacy/permission/
members_count/repos_count/created_at/updated_at/url). Reads the team detail
endpoint GET /orgs/{org}/teams/{team_slug}, which — unlike the list endpoint
— returns members_count/repos_count/timestamps, and surfaces a NotFoundError
when the slug is unknown. Wires `show` into the teams subcommand tree.
Shape the raw /stats/all payload into the Ruby's labeled report sections
(lib/hubctl/enterprise.rb#stats) instead of emitting it verbatim: present
sections only, each picking the Ruby's fixed field set, with the wire
`pulls` section surfaced as `pull_requests`. Absent numeric fields
default to 0.
Add `enterprise show`. Enterprise Cloud has no /enterprises/{enterprise}
detail endpoint, so (matching lib/hubctl/github_client.rb#enterprise) the
service resolves the account via GET /orgs/{org}, requires
plan.name == 'enterprise', and surfaces the populated detail subset,
failing with a ValidationError otherwise. Wired into the enterprise
command group and the root tree assertion.
Wire the billing CLI handlers (usage/actions) to honor the Ruby's
mode branch (lib/hubctl/enterprise.rb:376-389): JSON emits the structured
summary (or empty marker), pretty/table emits the flattened
category/metric/value rows via flattenBilling, and an empty usage payload
renders the "No billing usage data found" message. Previously the handlers
emitted the raw BillingResult in both modes so flattenBilling was never
reached by the CLI.
Mirror the Ruby BaseCommand#require_org! (lib/hubctl/base_command.rb:86-96):
make the users/teams --org flag optional and, when omitted, resolve via
Config.defaultOrg (GITHUB_ORG env, then config default_org). If still
unresolved, fail with an actionable ValidationError envelope. A shared
resolveOrg helper in src/cli/handle.ts threads the failure through emit.

The orgs and enterprise groups take their scope as positional arguments
(Ruby def show(org)/def show(enterprise)) and never call require_org!/
require_enterprise!, so they keep their required positional args.
The Bun + TypeScript + Effect v4 rewrite reaches full parity with the
Ruby/Thor tool. Removes lib/, spec/, Gemfile(.lock), gemset.nix,
hubctl.gemspec, Rakefile, .rubocop*, and shell.nix.
Adds packages.default (a bun-wrapper over src/main.ts with deps vendored
via a fixed-output derivation) and apps.default; verified `nix build`
produces a working hubctl. The standalone compiled binary still ships via
`bun run build:local` / release artifacts.
The mode flags were pre-scanned for mode resolution but passed through to a
command parser that does not declare them, so `hubctl --json version` tripped
an unrecognized-flag help screen. Strip the entry-point-only mode flags from
argv before parsing (--yes is kept; subcommands declare it).
Documents the agent-first JSON envelope contract, output modes, the full
command reference, Nix/Bun install, and the new vitest stack. Adds
ADR-000002 recording the agent-first output decision. Seeds the breaking
rewrite in CHANGELOG (release-please owns it going forward).
list/info (GET /user/orgs) return minimal org objects without per-org
counts or html_url; show (GET /orgs/{org}) never returns the user-only
`bio`; the teams list endpoint omits members_count/repos_count. The strict
schemas threw an uncaught SchemaError against real GitHub (fixtures had
masked it). Make those fields optional and render null / '-', matching the
Ruby's nil behavior. Tests now use realistic minimal payloads.
Wraps Schema.decodeUnknownEffect into Effect<A, DecodeError> so a payload
mismatch becomes a typed failure (rendered as an ok:false envelope) instead
of a thrown defect that dumps a stack trace. DecodeError carries
code:decode_error, a message naming the failing field, and a report-this fix.
Override the ConfigProvider with an empty env so the 'builds without a GitHub
token' case asserts auth:null regardless of an ambient GITHUB_TOKEN on the
developer's machine (it passed in CI, failed locally when a token was exported).
Every github-backed service method already declares Effect<…, GithubError>, so
widening the union lets a schema mismatch surface as a typed failure with zero
signature churn. No consumer does an exhaustive match over the union.
…opes

Replace every throwing Schema.decodeUnknownSync + Effect.map(decode) with the
effectful decode helper + Effect.flatMap across repos/orgs/users/teams/
enterprise/auth, so a payload that doesn't match the schema fails as a typed
DecodeError (clean envelope) instead of a stack-trace defect. Data-first sites
(auth status, orgs info) and inline object closures (repos topics, users
invite, teams add) restructured to sequence the decode effectfully. Each
service gains a TDD test feeding a realistic-but-mismatched payload (a required
field omitted or nulled) and asserting a DecodeError, not a throw.
A corrupt ~/.config/hubctl/config.json no longer throws an uncaught defect: it
fails with a typed DecodeError that the config command group renders as a clean
ok:false envelope (code:decode_error). get/set/list widen to PlatformError |
DecodeError; githubToken/defaultOrg keep their orElseSucceed recovery so a
corrupt file degrades to absent rather than crashing.
…envelope

End-to-end regression through the real command pipeline (runCli + FakeGithub):
a repo row missing the required full_name yields ok:false / error.code
decode_error / a report-this fix, never a stack trace.
Adds the first PR merge gate (previously only release-please ran, on main). Four
parallel lanes: a fast pure-Bun validate (format/lint/typecheck/test), the
authoritative `bun run validate` inside the Nix devShell (so ast-grep/fallow run
at flake-pinned versions), a nix build .#hubctl + flake check install gate, and a
native build/e2e matrix (ubuntu x64 + macos-15 arm64) that compiles the standalone
binary and smoke-runs it on each OS. Caching: bun install cache keyed on bun.lock
+ /nix/store via cache-nix-action keyed on flake.lock; concurrency cancels
superseded PR runs. Bun pinned to 1.3.0 to match the nixpkgs flake. Validated with
actionlint. Dependabot keeps the action pins fresh.
fallow audits changed files vs the PR base and needs the base ref (fetch-depth:0)
plus FALLOW_AUDIT_BASE; it is not part of `validate` and reports pre-existing
complexity on a large diff, so run it PR-only with continue-on-error rather than
blocking the merge gate.
Point Ruby users at the frozen ruby branch / v0.3.1 tag with install commands and
the fork point, and link the announcement issue (#3).
The GitHub Enterprise audit-log API returns the event time and id under the
prefixed keys @timestamp and _document_id, but AuditEntryRaw read bare
timestamp/document_id. Effect's open Struct silently ignored the real keys, so
every entry came back with timestamp:null and document_id:null (silent data
loss, not an error). Read the prefixed keys; the shaped output keeps the bare
timestamp/document_id names. The fixtures used the buggy keys and masked it, so
they're corrected to the real API shape (red before this fix).
A 403 carries x-accepted-oauth-scopes (what the endpoint needs) and
x-oauth-scopes (what the token has). Surface both in the fix so the ok:false
envelope tells the caller exactly which scope to add instead of a generic
'missing a required scope'. Falls back to the prior generic fix when the header
is absent; the rate-limit-on-403 path still wins first. e.g. enterprise
audit-log now reports 'accepts: admin:enterprise, read:audit_log. Your token
has: ...'.
audit-log called github.paginate, which auto-follows every Link page of the
entire enterprise retention window and collects them all before emitting — for a
real enterprise that never finishes, so the command hung. Fetch a single page via
requestRaw (per_page defaults to 30), parse the rel="next" cursor from the Link
header, and put it in next_actions as --after <cursor> so callers page
deliberately. Converts the command from the never-actually-incremental NDJSON
stream to one clean envelope. auditLog now returns { entries, nextAfter }.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant