Rewrite hubctl in Bun + TypeScript + Effect v4 (agent-first JSON envelope)#2
Open
jordangarrison wants to merge 86 commits into
Open
Rewrite hubctl in Bun + TypeScript + Effect v4 (agent-first JSON envelope)#2jordangarrison wants to merge 86 commits into
jordangarrison wants to merge 86 commits into
Conversation
Design for migrating hubctl from Ruby/Thor to Bun + TypeScript + Effect v4: agent-first JSON envelope output with a --pretty human layer, Octokit wrapped in an Effect service, full feature parity in a single cutover, Bun-compiled binary distribution, and a floai-style validation/compilation toolchain (tsgo, oxlint/oxfmt, @effect/language-service, ast-grep, vitest, runtime e2e).
Replace the Ruby dev environment with a Bun/TypeScript/Effect v4 toolchain: - flake.nix: Bun + Node + git + (nixpkgs) ast-grep devShell; drop Ruby gem build - package.json: bun scripts incl. validate chain (format/lint/typecheck/test/ast-grep) - tsgo (@typescript/native-preview) + @effect/tsgo + @effect/language-service (all Effect diagnostics at error) - oxlint (ultracite presets + @mpsuesser/oxlint-plugin-effect) and oxfmt - vitest + @effect/vitest (threads pool); e2e config for compiled-binary smoke - ast-grep structural-rule scaffold; commitlint; simple-git-hooks + nano-staged - docs/effect-v4-api-notes.md: verified v4 CLI API (Flag/Argument/Command.runWith) bun run validate is green. fallow and the layer-boundaries oxlint plugin are deferred (see follow-ups).
fallow ships only a generic-glibc npm binary (NixOS can't run it) and isn't in nixpkgs. Add an autoPatchelfHook derivation so it runs from the flake dev env with no system dependency (no nix-ld). Remove the npm fallow dep so its broken binary doesn't shadow the patched one on PATH. fallow stays a separate leg (not in validate) until real source exists — on a bare scaffold it false-flags tooling-only devDeps (@effect/vitest, nano-staged).
Fold in the 'flake + validation tooling first' feedback: Phase 0 is complete and moved to the front. Replace the v3-shaped CLI snippets (Options/Args/Command.run) with the verified v4 API (Flag/Argument/Command.runWith/BunServices.layer) per docs/effect-v4-api-notes.md, and record the NixOS native-binary and worktree git-hook deltas.
GET /orgs/{org}/teams (the list endpoint) does not return
members_count/repos_count — only the team detail endpoint does. The list
schema previously required them, so against real GitHub `teams list` threw
a decode defect (tests passed only because fixtures wrongly included the
fields). Make them Schema.optional and render '-' when absent, matching the
Ruby's `team[:members_count]` nil handling (lib/hubctl/teams.rb:18-31).
Add `hubctl teams show <team> --org <org>`, mirroring
lib/hubctl/teams.rb#show (id/name/slug/description/privacy/permission/
members_count/repos_count/created_at/updated_at/url). Reads the team detail
endpoint GET /orgs/{org}/teams/{team_slug}, which — unlike the list endpoint
— returns members_count/repos_count/timestamps, and surfaces a NotFoundError
when the slug is unknown. Wires `show` into the teams subcommand tree.
Shape the raw /stats/all payload into the Ruby's labeled report sections (lib/hubctl/enterprise.rb#stats) instead of emitting it verbatim: present sections only, each picking the Ruby's fixed field set, with the wire `pulls` section surfaced as `pull_requests`. Absent numeric fields default to 0.
Add `enterprise show`. Enterprise Cloud has no /enterprises/{enterprise}
detail endpoint, so (matching lib/hubctl/github_client.rb#enterprise) the
service resolves the account via GET /orgs/{org}, requires
plan.name == 'enterprise', and surfaces the populated detail subset,
failing with a ValidationError otherwise. Wired into the enterprise
command group and the root tree assertion.
Wire the billing CLI handlers (usage/actions) to honor the Ruby's mode branch (lib/hubctl/enterprise.rb:376-389): JSON emits the structured summary (or empty marker), pretty/table emits the flattened category/metric/value rows via flattenBilling, and an empty usage payload renders the "No billing usage data found" message. Previously the handlers emitted the raw BillingResult in both modes so flattenBilling was never reached by the CLI.
Mirror the Ruby BaseCommand#require_org! (lib/hubctl/base_command.rb:86-96): make the users/teams --org flag optional and, when omitted, resolve via Config.defaultOrg (GITHUB_ORG env, then config default_org). If still unresolved, fail with an actionable ValidationError envelope. A shared resolveOrg helper in src/cli/handle.ts threads the failure through emit. The orgs and enterprise groups take their scope as positional arguments (Ruby def show(org)/def show(enterprise)) and never call require_org!/ require_enterprise!, so they keep their required positional args.
The Bun + TypeScript + Effect v4 rewrite reaches full parity with the Ruby/Thor tool. Removes lib/, spec/, Gemfile(.lock), gemset.nix, hubctl.gemspec, Rakefile, .rubocop*, and shell.nix.
Adds packages.default (a bun-wrapper over src/main.ts with deps vendored via a fixed-output derivation) and apps.default; verified `nix build` produces a working hubctl. The standalone compiled binary still ships via `bun run build:local` / release artifacts.
The mode flags were pre-scanned for mode resolution but passed through to a command parser that does not declare them, so `hubctl --json version` tripped an unrecognized-flag help screen. Strip the entry-point-only mode flags from argv before parsing (--yes is kept; subcommands declare it).
Documents the agent-first JSON envelope contract, output modes, the full command reference, Nix/Bun install, and the new vitest stack. Adds ADR-000002 recording the agent-first output decision. Seeds the breaking rewrite in CHANGELOG (release-please owns it going forward).
list/info (GET /user/orgs) return minimal org objects without per-org
counts or html_url; show (GET /orgs/{org}) never returns the user-only
`bio`; the teams list endpoint omits members_count/repos_count. The strict
schemas threw an uncaught SchemaError against real GitHub (fixtures had
masked it). Make those fields optional and render null / '-', matching the
Ruby's nil behavior. Tests now use realistic minimal payloads.
Wraps Schema.decodeUnknownEffect into Effect<A, DecodeError> so a payload mismatch becomes a typed failure (rendered as an ok:false envelope) instead of a thrown defect that dumps a stack trace. DecodeError carries code:decode_error, a message naming the failing field, and a report-this fix.
Override the ConfigProvider with an empty env so the 'builds without a GitHub token' case asserts auth:null regardless of an ambient GITHUB_TOKEN on the developer's machine (it passed in CI, failed locally when a token was exported).
Every github-backed service method already declares Effect<…, GithubError>, so widening the union lets a schema mismatch surface as a typed failure with zero signature churn. No consumer does an exhaustive match over the union.
…opes Replace every throwing Schema.decodeUnknownSync + Effect.map(decode) with the effectful decode helper + Effect.flatMap across repos/orgs/users/teams/ enterprise/auth, so a payload that doesn't match the schema fails as a typed DecodeError (clean envelope) instead of a stack-trace defect. Data-first sites (auth status, orgs info) and inline object closures (repos topics, users invite, teams add) restructured to sequence the decode effectfully. Each service gains a TDD test feeding a realistic-but-mismatched payload (a required field omitted or nulled) and asserting a DecodeError, not a throw.
A corrupt ~/.config/hubctl/config.json no longer throws an uncaught defect: it fails with a typed DecodeError that the config command group renders as a clean ok:false envelope (code:decode_error). get/set/list widen to PlatformError | DecodeError; githubToken/defaultOrg keep their orElseSucceed recovery so a corrupt file degrades to absent rather than crashing.
…envelope End-to-end regression through the real command pipeline (runCli + FakeGithub): a repo row missing the required full_name yields ok:false / error.code decode_error / a report-this fix, never a stack trace.
Adds the first PR merge gate (previously only release-please ran, on main). Four parallel lanes: a fast pure-Bun validate (format/lint/typecheck/test), the authoritative `bun run validate` inside the Nix devShell (so ast-grep/fallow run at flake-pinned versions), a nix build .#hubctl + flake check install gate, and a native build/e2e matrix (ubuntu x64 + macos-15 arm64) that compiles the standalone binary and smoke-runs it on each OS. Caching: bun install cache keyed on bun.lock + /nix/store via cache-nix-action keyed on flake.lock; concurrency cancels superseded PR runs. Bun pinned to 1.3.0 to match the nixpkgs flake. Validated with actionlint. Dependabot keeps the action pins fresh.
fallow audits changed files vs the PR base and needs the base ref (fetch-depth:0) plus FALLOW_AUDIT_BASE; it is not part of `validate` and reports pre-existing complexity on a large diff, so run it PR-only with continue-on-error rather than blocking the merge gate.
Point Ruby users at the frozen ruby branch / v0.3.1 tag with install commands and the fork point, and link the announcement issue (#3).
The GitHub Enterprise audit-log API returns the event time and id under the prefixed keys @timestamp and _document_id, but AuditEntryRaw read bare timestamp/document_id. Effect's open Struct silently ignored the real keys, so every entry came back with timestamp:null and document_id:null (silent data loss, not an error). Read the prefixed keys; the shaped output keeps the bare timestamp/document_id names. The fixtures used the buggy keys and masked it, so they're corrected to the real API shape (red before this fix).
A 403 carries x-accepted-oauth-scopes (what the endpoint needs) and x-oauth-scopes (what the token has). Surface both in the fix so the ok:false envelope tells the caller exactly which scope to add instead of a generic 'missing a required scope'. Falls back to the prior generic fix when the header is absent; the rate-limit-on-403 path still wins first. e.g. enterprise audit-log now reports 'accepts: admin:enterprise, read:audit_log. Your token has: ...'.
audit-log called github.paginate, which auto-follows every Link page of the
entire enterprise retention window and collects them all before emitting — for a
real enterprise that never finishes, so the command hung. Fetch a single page via
requestRaw (per_page defaults to 30), parse the rel="next" cursor from the Link
header, and put it in next_actions as --after <cursor> so callers page
deliberately. Converts the command from the never-actually-incremental NDJSON
stream to one clean envelope. auditLog now returns { entries, nextAfter }.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reimplements
hubctlfrom Ruby/Thor to Bun + TypeScript + Effect v4 as an agent-first CLI: every command emits a structured JSON envelope by default, with an optional--prettyhuman renderer over the same envelope. Full feature parity with the Ruby tool (minus the stubbedservercommand), distributed as a Bun-compiled binary and a Nix flake.Architecture
Layered Effect app: thin
cli/command modules → centralizedoutput/envelope renderer →services/domain services (Context.Service) → a singlegithub/Octokit wrapper → Bun platform. Typed errors flow in theEchannel; services resolve via Layers. The envelope is always built; pretty mode is a renderer over it, so the two views can't drift.Command surface (parity with the Ruby CLI)
version·auth·config <get|set|list|init|path>·repos <list|show|create|clone|archive|topics>·orgs <list|show|members|repos|teams|info>·users <show|whoami|list|invite|remove>·teams <list|show|create|members|add|remove>·enterprise <show|orgs|members|owners|billing|licenses|audit-log|sso|stats|security-analysis>next_actions(HATEOAS) on success;error.code/message+ plain-languagefixon failure.--yesin JSON mode (and prompt at a TTY).enterprise audit-log); ~50-item result truncation for context-window discipline.How it was built & verified
validate→ commit, per task.config/authgroups,orgs repos/teams/info,teams show,enterprise show, ateams listdecode bug,cloneexit-code handling,--orgfallback, and more).docs/review-deferred-minor.md.Robustness: decode hardening (no decode ever crashes the envelope)
Every GitHub payload was decoded with the throwing
Schema.decodeUnknownSync, called insideEffect.map. A throw there is an uncaught defect — it escapes the typedEchannel, so a real API response that didn't match the schema (a field absent on a list endpoint, anullwhere a string was assumed, a new/removed field) dumped a raw stack trace instead of the{ok,command,result,next_actions,error,fix}envelope. For an agent-first tool whose whole contract is "always emit a valid envelope," that's a contract violation. We hit four of these inorgsand hand-fixed them (95ae480); this makes the fix systemic.src/schema/decode.ts):decode(schema, label)wraps the effectfulSchema.decodeUnknownEffectintoEffect<A, DecodeError>— a mismatch now fails in the typedEchannel instead of throwing.DecodeError(aData.TaggedError) carriescode:"decode_error", a message naming the failing field/path, and a "this looks like a hubctl bug — please report it"fix.Echannel, zero churn:DecodeErrorjoined theGithubErrorunion, so everyEffect<…, GithubError>service signature gained it without edits. The existingemit→Output.failpath renders it as a standardok:falseenvelope.repos/orgs/users/teams/enterprise/auth(Effect.map(decode)→Effect.flatMap(decode); data-first and inline-closure sites restructured to sequence the decode effectfully). The on-disk config file JSON parse is hardened the same way, so a corruptconfig.jsonis a clean envelope, not a crash.Schema.Structignores unknown keys in this Effect beta, so additive GitHub API changes never break decoding; only missing/null/wrong-type required fields do, and those now render cleanly.Schema.encodeSyncsites inoutput/render.ts,output/service.ts, and the config write path — they encode hubctl's own post-decode, JSON-safe data, never external input.DecodeErrorin theEchannel, plus a CLI-level regression asserting the full pipeline rendersok:false/decode_error. Verified end-to-end with a live-token smoke run across every command group — every output a single valid JSON envelope, success and error paths, zero stack traces.Testing
bun run validate(format → lint → typecheck via tsgo → 269 tests → ast-grep): greenbun run test:e2e(compiled-binary smoke against a mock GitHub): greennix build .#hubctlproduces a working binaryNotes for reviewers
bun-wrapper oversrc/main.tsrather than thebun build --compileartifact: autoPatchelf rewrites the ELF and breaks Bun's detection of the bundle appended to the executable. The true standalone single-file binary still ships viabun run build:local/ release artifacts.~/.hubctl.yml(YAML) →~/.config/hubctl/config.json(JSON) — an intentional design decision.usersintentionally overlaps some org-membership routes withorgs(faithful to the Rubyusers.rb).