Skip to content

fix(github): isolate bootstrap readiness from application delivery - #4472

Merged
devantler merged 8 commits into
mainfrom
codex/github-bootstrap-isolation-2626-round15
Oct 4, 2026
Merged

devantler merged 8 commits into
mainfrom
codex/github-bootstrap-isolation-2626-round15

Conversation

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Why

A GitHub repository waiting for manual setup can currently block every application deployment. Organisation housekeeping should remain visible without freezing unrelated delivery.

What

Let GitHub resources reconcile independently while retaining deployment errors and alerts for unsynced resources. Add a required check to prevent the delivery coupling from returning.

Fixes #2626

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Validation at 64bffd4fd528244eda2f78ae31e62618e7fa29ca:

  • Compared all 198 selected authorization objects against protected main af6cc58f458be40b0526d1588b31460805e72b07; the identity sets are equal in both directions. Exactly one object changes: the GitHub configuration Kustomization's wait setting changes from true to false.
  • Restoring that single boolean makes the canonical object equal to the approved baseline. No permission rules, service accounts, source identities, or health checks change.
  • Updated only the corresponding ledger line and object fingerprint. Authorization selection and rejection paths are unchanged. The frozen approval history remains unchanged.
  • The complete authorization regression suite passes. The real validation command passes with checksum-verified official kubectl v1.36.2 / Kustomize v5.8.1, after correctly refusing the newer host renderer.
  • Integrated protected main through the already merged lease regression fix without rewriting history. Both new commits have verified good signatures; the remote branch equals the named head.

CI is rerunning. This is validation evidence, not a readiness verdict or a claim that deployment has completed.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository YAML (base), Organization UI (inherited)
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 0d481e68-c9dd-42ad-a2d1-7b9934173668
📥 Commits

Reviewing files that changed from the base of the PR and between 64bffd4 and 8ce09f1.

📒 Files selected for processing (13)
  • .github/actions/deploy-prod/action.yml
  • .github/workflows/ci.yaml
  • docs/github-management.md
  • k8s/bases/apps/github-config/flux-kustomization.yaml
  • k8s/bases/infrastructure/cluster-policies/flux/enforce-flux-best-practices.yaml
  • scripts/guard-github-bootstrap-isolation.sh
  • scripts/tests/test-github-bootstrap-isolation.sh
  • scripts/validate-eks-ci-role-policy/approved-surface.txt
  • scripts/validate-eks-ci-role-policy/main.go
  • tests/flux-bootstrap-health-gating/kyverno-test.yaml
  • tests/flux-bootstrap-health-gating/missing-reliability/kyverno-test.yaml
  • tests/flux-bootstrap-health-gating/missing-reliability/resources.yaml
  • tests/flux-bootstrap-health-gating/resources.yaml

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository YAML (base), Organization UI (inherited)
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 51b74533-acb1-4b9a-af06-473556d3a781
📥 Commits

Reviewing files that changed from the base of the PR and between 5e37f34 and 64bffd4.

📒 Files selected for processing (8)
  • .github/actions/deploy-prod/action.yml
  • .github/workflows/ci.yaml
  • docs/github-management.md
  • k8s/bases/apps/github-config/flux-kustomization.yaml
  • scripts/guard-github-bootstrap-isolation.sh
  • scripts/tests/test-github-bootstrap-isolation.sh
  • scripts/validate-eks-ci-role-policy/approved-surface.txt
  • scripts/validate-eks-ci-role-policy/main.go

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Recent review details
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2026-08-10T13:01:12.782Z
Learnt from: devantler
Repo: devantler-tech/platform PR: 3057
File: .github/workflows/ci.yaml:622-659
Timestamp: 2026-08-10T13:01:12.782Z
Learning: Repository shell tests and scripts must remain compatible with macOS Bash 3.2. Do not use Bash 4+ features such as `mapfile`; use portable constructs, such as a `while IFS= read -r` loop, instead.

Applied to files:

  • scripts/guard-github-bootstrap-isolation.sh
  • scripts/tests/test-github-bootstrap-isolation.sh
🔇 Additional comments (8)
k8s/bases/apps/github-config/flux-kustomization.yaml (1)

17-20: LGTM!

docs/github-management.md (1)

47-55: LGTM!

Also applies to: 73-74

scripts/validate-eks-ci-role-policy/approved-surface.txt (1)

119-119: LGTM!

scripts/validate-eks-ci-role-policy/main.go (1)

275-275: LGTM!

scripts/guard-github-bootstrap-isolation.sh (1)

1-21: LGTM!

scripts/tests/test-github-bootstrap-isolation.sh (1)

1-41: LGTM!

.github/workflows/ci.yaml (1)

38-42: LGTM!

.github/actions/deploy-prod/action.yml (1)

191-197: LGTM!


📝 Walkthrough

Walkthrough

The GitHub Config Flux Kustomization now sets wait: false, while the parent apps Kustomization retains its readiness check. A Bash guard and regression test validate the GitHub Kustomization settings. CI runs the guard checks and the Crossplane sync alerter test. The production deployment action renders the apps manifests and runs the guard. Documentation and approved rendered fingerprints are updated.

Priority: ⬆️ High

Severity of issue fixed: High

Merge Risk: ⚪ Minimal · up to 64bff

GitHub repository bootstrap should no longer hold up unrelated application delivery, while the existing health check and validation remain in place. No identified issue blocks merging.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 64bff

The change isolates GitHub setup failures without intentionally widening access or removing application-health checks. The remaining uncertainty is whether independent reconciliation and alerting preserve failure visibility during degraded operation and recovery.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • observed — The documented GitHub App installation covers all organization repositories and includes organization administration and membership permissions. Namespace-scoped Kubernetes application therefore does not establish a per-repository external authorization boundary. The effective live installation was not available for verification, and this readiness change does not itself demonstrate expanded permissions.

Trust Boundaries and Controls

  • observed — The test constructs local fixtures with static mutations, and the guard parses manifest content rather than evaluating it as shell commands. CI checkout disables credential persistence and grants the changes job read permissions; production separately validates its own rendered manifest.

Resilience and Maintainability Implications

  • observed — Independent sync alerting contains handling for CRD inventory gaps, exporter or missing-series failures, and sustained Synced=False conditions. Its tests include rejected alert delivery. These controls counter the concern that removing Flux health waiting necessarily hides external reconciliation failure, but they do not prove live coverage of every degraded state.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Issue #2626 requires pending repository bootstrap to stop blocking application delivery, unsynced resources to remain visible, and a regression guard. The github-config Kustomization now sets `wait:…
Out of Scope Changes check ✅ Passed The changed deployment validation, CI test wiring, documentation, and rendered-surface fingerprint updates all support issue #2626 or validate the GitHub Kustomization change. No unrelated change is e…
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 3…
Title check ✅ Passed The title clearly summarizes the main change: isolating GitHub bootstrap readiness from application delivery.
Description check ✅ Passed The description explains the delivery-blocking problem and the change to let GitHub resources reconcile independently while retaining alerts and a regression guard.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Ready at 64bffd4fd528244eda2f78ae31e62618e7fa29ca: all 35 checks are settled without failures, GitHub reports CLEAN/MERGEABLE, and the complete thread census has zero unresolved findings. CodeRabbit's exact-head summary is structurally GREEN with no actionable or ancillary findings: #4472 (comment).

User evaluation exercised the actual production render and guard, confirmed the parent application-health gate stays enabled, and reproduced a pending GitHub managed resource through the existing Crossplane alert fixture. Enabling wait or adding explicit child health checks fails the new regression; missing, duplicate or malformed input remains UNKNOWN. Pending resources remain visible to monitoring. Authorization conservation and its real pinned-renderer check are recorded in #4472 (comment).

Promoting the exact reviewed head. Production convergence and the latest protected-base deployment remain merge-queue gates, followed by live readback.

@devantler
devantler marked this pull request as ready for review October 4, 2026 16:24
@devantler
devantler added this pull request to the merge queue Oct 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 4, 2026
@devantler
devantler added this pull request to the merge queue Oct 4, 2026
@devantler
devantler marked this pull request as draft October 4, 2026 17:07
@devantler
devantler removed this pull request from the merge queue due to a manual request Oct 4, 2026
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

The first protected deployment was rejected by the Flux reliability admission policy because the GitHub bootstrap resource now intentionally uses asynchronous reconciliation. The PR remains draft while that mismatch is repaired; it has not been blindly requeued.

The policy repair permits the asynchronous setting only for the exact platform-owned GitHub configuration identity and retains interval, timeout, retry, pruning, and source requirements. New admission fixtures fail against the old policy and pass after the repair. They also reject a matching name in another namespace, another name in the same namespace, a parent application health bypass, and disabled pruning.

All 385 Kyverno assertions pass, and the fixture-evaluation guard confirms every declared row was evaluated. The bootstrap guard, its negative and incomplete-census controls, the full authorization suite, and the actual rendered authorization check pass. Current-head CI and review remain required before another protected deployment attempt.

@devantler devantler left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by the Agentic Engineer

Self-review (fallback — CodeRabbit, Codex and Cursor Bugbot unavailable)

Reviewed commit: fff15dc

  • CodeRabbit: the account-wide included-review rate-limit refusal was observed at 2026-10-04T17:13:45Z on devantler-tech/agent-plugins#473 (comment) and remains inside its stated 28-minute reset window. The prior CodeRabbit green artifact covers only the superseded 64bffd4 head and is not reused.
  • Codex: the account-wide usage-limit refusal was observed at 2026-10-04T02:11:02Z on devantler-tech/ksail#7433 (comment) and requires account credits or settings recovery.
  • Cursor Bugbot: the user/team usage-limit refusal was observed at 2026-10-04T12:21:20Z on devantler-tech/ksail#7481 (comment) and requires usage or spend recovery.
  • Direct current-PR review objects, comments, threads, and checks were read before this fallback. The complete thread census is empty and no provider finding is being discarded.
  • The complete diff was reviewed for exact-identity policy scoping, retained Flux reliability controls, GitHub bootstrap isolation, Crossplane alert continuity, deployment guard ordering, authorization conservation, and fail-closed fixture coverage.
  • Exact-head validation passed all 385 Kyverno assertions, all 32 fixture-evaluation censuses, the bootstrap guard and its negative/incomplete controls, and the authorization suite with checksum-verified kubectl v1.36.2 / Kustomize v5.8.1. The installed v1.37.1 renderer was independently confirmed to fail closed.

Verdict: no P0/P1 findings

@devantler devantler left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by the Agentic Engineer

Exact-head local review at 8ce09f198b7911ece25aa11be094d6df298dd69b: no P0/P1 findings. Re-reviewed the refreshed diff after merging current main; the exact github-config/github-config identity exception retains interval, timeout, retry, prune and source constraints, lookalikes remain denied, and the parent applications layer remains health-gated. Exact-head verification passed the bootstrap guard, ShellCheck, all 385 Kyverno cases, and 308 EKS policy-validator tests.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@devantler devantler left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by the Agentic Engineer

Self-review (fallback, exact head)

Reviewed commit: 8ce09f198b7911ece25aa11be094d6df298dd69b

  • CodeRabbit: LIMITED — review-lane health observed a rate-limit at 2026-10-04T18:35:13Z, retry after 2026-10-04T19:25:10Z.
  • Codex: DOWN — account-wide usage limit remains active; maintainer-only recovery.
  • Cursor Bugbot: DOWN — account-wide usage limit observed at 2026-10-04T18:18:31Z; maintainer-only recovery.

Re-reviewed the refreshed diff after merging current main. The exact github-config/github-config exception retains interval, timeout, retry, prune, and source constraints; identity lookalikes remain denied; the parent applications layer remains health-gated. Exact-head evidence: all 25 hosted checks passed (8 expected skips), the bootstrap guard and ShellCheck passed, all 385 Kyverno cases passed, and all 308 EKS policy-validator tests passed.

Verdict: no P0/P1 findings

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Exact-head readiness at 8ce09f198b7911ece25aa11be094d6df298dd69b: current base 574b5436e8fcef2798a72fdf90676d4b26010a28, all 25 hosted checks passed (8 expected skips), zero unresolved threads, and the exact-head fallback review is green after current lane-health verification. Promoting for the protected merge queue and production-deploy gate.

@devantler
devantler marked this pull request as ready for review October 4, 2026 18:38
@devantler
devantler added this pull request to the merge queue Oct 4, 2026
Merged via the queue into main with commit a6ec25a Oct 4, 2026
33 checks passed
@devantler
devantler deleted the codex/github-bootstrap-isolation-2626-round15 branch October 4, 2026 19:00
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Actual protected merge verified: #4472 merged at 2026-10-04 18:59:58 UTC as a6ec25afca7aaa49c08f74c77d0b594f7577d9df; #2626 is closed by that merge. Winning merge-group CI run 37225264048 and production deployment job 111504924765 both succeeded, including the release checks.

Scoped live readback confirms the GitHub reconciliation is Ready with wait: false and its current generation observed. The parent applications reconciliation retains wait: true and is Ready, and the complete Flux inventory is Ready. The earlier admission rejection is repaired through the narrowly scoped policy change; unrelated reconciliation identities retain the required health gate.

No incident or alert was manually resolved, and no credential or access boundary was changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

Isolate net-new repo bootstrap CRs so a pending manual bootstrap can't freeze all app delivery

1 participant