Part of epic #150. Sibling task: #141 (ETag caching for the REST endpoints that remain after this migration).
Scope
This issue is narrowly focused on one call site: ExtendedList() in internal/github/repos.go. It replaces the REST N+1 pattern (1 + N calls for an org with N repos) with a single paginated GraphQL query.
Other GraphQL opportunities (org members, team members) are explicitly not part of this issue — REST cost there is already low and is addressed by ETag caching in #141.
A detailed design has been drafted in docs/graphql-migration-plan.md.
Problem
ExtendedList() performs the following on every reconcile cycle:
GET /orgs/{org}/repos → 1 call (paginated, 100 per page)
for each repo:
GET /repos/{org}/{repo}/teams → N calls (one per repo)
For an org with 300 repositories this is 301+ REST calls per reconcile. Each call counts against the primary rate limit and returns 200 OK even when nothing changed.
Proposed Solution
Replace the REST N+1 pattern with a single paginated GraphQL query that fetches all repositories and their team permissions at once.
GraphQL query
query OrgRepositoriesWithTeams($org: String!, $repoCursor: String, $teamCursor: String) {
organization(login: $org) {
repositories(first: 100, after: $repoCursor, orderBy: {field: NAME, direction: ASC}) {
pageInfo { hasNextPage endCursor }
nodes {
name
isPrivate
teams(first: 100, after: $teamCursor) {
pageInfo { hasNextPage endCursor }
edges {
permission # ADMIN | MAINTAIN | WRITE | TRIAGE | READ
node { slug }
}
}
}
}
}
}
Permission enum mapping
| GraphQL enum |
REST string |
GithubTeamPermission |
ADMIN |
admin |
admin |
MAINTAIN |
maintain |
maintain |
WRITE |
push |
push |
TRIAGE |
triage |
triage |
READ |
pull |
pull |
WRITE → push is the critical mapping difference.
Implementation Plan
- Add
graphqlClient *githubv4.Client to DefaultRepositoryProvider (uses github.com/shurcooL/githubv4, already in go.mod).
- Construct the GraphQL client in
NewRepositoryProvider reusing the same authenticated HTTP client used by go-github (same pattern as users.go:149).
- New file
internal/github/repos_graphql.go containing query struct definitions and pagination logic.
- New method
ExtendedListGraphQL(ctx) on RepositoryProvider returning the same []GithubRepository shape as today.
- Add a permission converter
graphqlPermissionToTeamPermission.
- Add GraphQL-specific rate-limit error detection in
internal/controller/utils.go (the shurcooL/githubv4 library returns graphql.Errors rather than HTTP 403).
- Wire
GithubOrganizationReconciler.Reconcile() to call ExtendedListGraphQL instead of ExtendedList (optionally behind a feature flag during rollout).
Files to Change
| File |
Change |
internal/github/repos.go |
Add graphqlClient field; update NewRepositoryProvider; add ExtendedListGraphQL; add permission converter |
internal/github/repos_graphql.go |
New file: GraphQL query struct + pagination loop |
internal/controller/githuborganization_controller.go |
Replace ExtendedList call |
internal/controller/utils.go |
GraphQL rate-limit error detection |
internal/github/repos_test.go |
Unit tests for converter + mock-based tests |
What Does NOT Change
repoChangeCalculator — operates on []GithubRepository; unaffected by data source.
GithubRepository / GithubTeamWithPermission types — no schema changes.
- Mutation calls (
RepositoryTeamAdd, RepositoryTeamRemove) — REST is correct for writes; GraphQL mutations for team-repo permissions are not available in the current GitHub GraphQL schema.
GithubTeamRepository (exception CRDs) — fetched from Kubernetes.
Out of Scope (handled elsewhere)
Expected API Call Reduction
| Scenario |
REST calls today |
GraphQL calls after |
| 100 repos, ≤100 teams each |
1 + 100 = 101 |
1–2 |
| 300 repos, ≤100 teams each |
3 + 300 = 303 |
3–4 |
| 500 repos, ≤100 teams each |
5 + 500 = 505 |
5–6 |
GitHub GraphQL is point-based (5,000 points/hour for App installations); a single read-only query costs 1 point regardless of node count, so the reduction in primary rate-limit pressure is dramatic.
Related
Scope
This issue is narrowly focused on one call site:
ExtendedList()ininternal/github/repos.go. It replaces the REST N+1 pattern (1 + N calls for an org with N repos) with a single paginated GraphQL query.Other GraphQL opportunities (org members, team members) are explicitly not part of this issue — REST cost there is already low and is addressed by ETag caching in #141.
A detailed design has been drafted in
docs/graphql-migration-plan.md.Problem
ExtendedList()performs the following on every reconcile cycle:For an org with 300 repositories this is 301+ REST calls per reconcile. Each call counts against the primary rate limit and returns
200 OKeven when nothing changed.Proposed Solution
Replace the REST N+1 pattern with a single paginated GraphQL query that fetches all repositories and their team permissions at once.
GraphQL query
Permission enum mapping
GithubTeamPermissionADMINadminadminMAINTAINmaintainmaintainWRITEpushpushTRIAGEtriagetriageREADpullpullWRITE→pushis the critical mapping difference.Implementation Plan
graphqlClient *githubv4.ClienttoDefaultRepositoryProvider(usesgithub.com/shurcooL/githubv4, already ingo.mod).NewRepositoryProviderreusing the same authenticated HTTP client used bygo-github(same pattern asusers.go:149).internal/github/repos_graphql.gocontaining query struct definitions and pagination logic.ExtendedListGraphQL(ctx)onRepositoryProviderreturning the same[]GithubRepositoryshape as today.graphqlPermissionToTeamPermission.internal/controller/utils.go(theshurcooL/githubv4library returnsgraphql.Errorsrather than HTTP 403).GithubOrganizationReconciler.Reconcile()to callExtendedListGraphQLinstead ofExtendedList(optionally behind a feature flag during rollout).Files to Change
internal/github/repos.gographqlClientfield; updateNewRepositoryProvider; addExtendedListGraphQL; add permission converterinternal/github/repos_graphql.gointernal/controller/githuborganization_controller.goExtendedListcallinternal/controller/utils.gointernal/github/repos_test.goWhat Does NOT Change
repoChangeCalculator— operates on[]GithubRepository; unaffected by data source.GithubRepository/GithubTeamWithPermissiontypes — no schema changes.RepositoryTeamAdd,RepositoryTeamRemove) — REST is correct for writes; GraphQL mutations for team-repo permissions are not available in the current GitHub GraphQL schema.GithubTeamRepository(exception CRDs) — fetched from Kubernetes.Out of Scope (handled elsewhere)
Expected API Call Reduction
GitHub GraphQL is point-based (5,000 points/hour for App installations); a single read-only query costs 1 point regardless of node count, so the reduction in primary rate-limit pressure is dramatic.
Related
docs/graphql-migration-plan.md