feat: add retry transport and switch to silent-by-default logging

Author: pcanilho

.github/workflows/ci.yml

@@ -114,7 +114,7 @@ jobs:
         run: go vet ./...
 
   live-drift:
-    name: Live ETag drift probe
+    name: Live drift and retry probe
     runs-on: ubuntu-latest
     timeout-minutes: 5
     needs: test
@@ -130,7 +130,7 @@ jobs:
         with:
           go-version-file: go.mod
           cache: true
-      - name: go test -tags=live
+      - name: go test -tags=live (etag + retry)
         env:
           GITHUB_TOKEN: ${{ github.token }}
-        run: go test -tags=live -run TestETag_Live ./etag/...
+        run: go test -tags=live -run 'TestETag_Live|TestRetry_Live' ./etag/... ./retry/...

CHANGELOG.md

@@ -5,6 +5,115 @@ All notable changes to **go-github-kit** are documented in this file.
 The format is based on [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.1.0] - 2026-04-26
+
+Adds a `retry` middleware for transient failures (5xx, network errors,
+transport-level deadline exceeded), filling the gap between the rate-limit
+layer (which owns 429s) and the underlying transport. Composes with the
+existing stack without new runtime dependencies.
+
+### Added
+
+#### Retry middleware (`retry/`)
+
+- New `retry.NewTransport(base, opts...)` chainable `http.RoundTripper`.
+  Defaults: 3 attempts (1 initial + 2 retries), 200ms..2s decorrelated jitter
+  (per AWS guidance), idempotent methods only (GET/HEAD/OPTIONS/PUT/DELETE),
+  default predicate retries {500, 502, 503, 504} or `*net.OpError`/`io.EOF`/
+  `io.ErrUnexpectedEOF`/`context.DeadlineExceeded`.
+- Options: `WithMaxAttempts(n)` (clamped to [1, 100]), `WithBackoff(min, max)`
+  (clamped to [1ms, 1h] with min<=max), `WithRetryOn(predicate)` (replaces
+  the default predicate; takes ownership of method-safety),
+  `WithLogger(*slog.Logger)`.
+- Top-level `ghkit.WithRetry(opts ...retry.Option)` slots the layer between
+  RateLimit and oauth2 in the chain - 429s never reach retry, and retried
+  requests get the latest token via oauth2's per-call `Source.Token()`.
+- 429 hard-exclusion lives outside the predicate so a user-supplied
+  `WithRetryOn` cannot accidentally fight `ratelimit`.
+- `Retry-After` honored (delta-seconds and HTTP-date formats); when it
+  exceeds the operator's `maxDelay` the call returns
+  `(prior_resp, retry.ErrRetryAfterExceedsMax)` and the caller owns
+  drain+close on the response.
+- Caller-context cancellation is terminal: `req.Context().Err() != nil`
+  always stops retries before any predicate is consulted, and during
+  backoff sleep via `time.NewTimer` + `Stop` (no leaked timers on long
+  `Retry-After` values).
+- Body-bearing retries require `req.GetBody`; missing GetBody on a retry
+  attempt yields `errors.Join(retry.ErrBodyNotRewindable, prior_err)` so
+  callers see both causes.
+- Exported predicate primitives - `retry.IsIdempotent(method)`,
+  `retry.IsRetryable5xx(code)`, `retry.IsTransientNetErr(err)` - so callers
+  composing their own `WithRetryOn` don't have to reimplement the defaults.
+- Panic recovery around user predicates: a panicking `WithRetryOn` is
+  treated as "do not retry" and emits a `retry_predicate_panic` event
+  rather than crashing the transport.
+- Sanitised structured logging via `slog.Logger` with explicit per-event
+  levels: `retry_sleep`, `retry_decision` (Debug; silent-success first
+  attempts skipped), `retry_abort`, `retry_body_unrewindable`,
+  `retry_exhausted`, `retry_predicate_panic` (Warn). `last_err_type` walks
+  joined/wrapped error chains so operators see meaningful types instead of
+  `*errors.joinError`.
+- DoS protection: prior-response drain capped at 128 KiB before close,
+  bounding the time we hold a connection on hostile/oversize bodies.
+
+#### Documentation
+
+- README transport-stack diagram updated to show retry between RateLimit
+  and oauth2.
+- New retry recipe with default and tuned-policy examples (including
+  `Idempotency-Key`-based POST opt-in).
+- New "Things worth knowing" note on retry/throttle interaction (each retry
+  attempt consumes a throttle token).
+- Package `doc.go` for `retry/` and updated top-level `doc.go` with the new
+  chain layout.
+
+#### Tooling and CI
+
+- Live integration test (`retry/live_check_test.go`, build-tag `live`,
+  function `TestRetry_Live`) that exercises retry against `api.github.com`.
+- CI `live-drift` job renamed to `Live drift and retry probe` and extended
+  to run both `TestETag_Live` and `TestRetry_Live`.
+
+### Changed
+
+- **Silent by default.** `ghkit`, `etag`, `ratelimit`, and `retry` no longer
+  default-initialise a `slog.Default()` logger. Without an explicit
+  `WithLogger(...)` call, the library emits no log records. This is a
+  behaviour change from 1.0.0; users who relied on default stderr output
+  must now opt in via `ghkit.WithLogger(slog.Default())` (or any logger).
+- Per-sub-package `WithLogger` options inside `WithRetry`, `WithETagCache`,
+  and `WithRateLimit` now correctly override the top-level `WithLogger`
+  instead of being silently shadowed by it. Previously the chain assembly
+  appended ghkit's logger after user-supplied sub-package options, so
+  user values lost; the prepend pattern lets user values win.
+- `etag.WithLogger(nil)` and `ratelimit.WithLogger(nil)` now mean
+  "explicitly silent" instead of being a no-op. Combined with silent-by-
+  default, this lets callers compose `WithLogger(real)` then
+  `WithLogger(nil)` to silence on a per-construction basis.
+- `retry.IsTransientNetErr` now short-circuits known-permanent failures to
+  `false`: DNS NXDOMAIN (`*net.DNSError.IsNotFound`), `syscall.ECONNREFUSED`
+  (TCP RST on connect to a closed port), and x509 cert-validation errors
+  (`x509.UnknownAuthorityError`, `*x509.HostnameError`,
+  `x509.CertificateInvalidError`). Misconfigured URLs and expired certs now
+  fail fast instead of burning the retry budget. Other DNS errors (server
+  failure, timeout) remain transient.
+- `ghkit.WithRateLimit` and `ghkit.WithRateLimitDisabled` are now mutually
+  exclusive. Combining them returns `ErrConflictingRateLimit` at
+  construction. Previously the kit logged a warning and silently dropped
+  the registered callbacks; with silent-by-default that warning would have
+  been invisible. The hard error matches the existing
+  `ErrConflictingAuth` precedent for `WithToken` + `WithTokenSource`.
+
+### Added (continued)
+
+- New runnable example at `examples/retry-on-flaky/` demonstrating
+  `WithRetry` with a tuned backoff and a custom predicate that opts POST in
+  via `Idempotency-Key`.
+
+### Dependencies
+
+No changes. Same as 1.0.0.
+
 ## [1.0.0] - 2026-04-25
 
 Initial public release.
@@ -132,4 +241,5 @@ and rotating PATs alike.
 - `golang.org/x/oauth2` v0.36.0
 - `golang.org/x/time` v0.15.0
 
+[1.1.0]: https://github.com/pcanilho/go-github-kit/releases/tag/v1.1.0
 [1.0.0]: https://github.com/pcanilho/go-github-kit/releases/tag/v1.0.0

README.md

@@ -51,21 +51,22 @@ func main() {
 
 `ghkit.New` is generic over the returned type; passing `github.NewClient` lets type inference pick up `*github.Client`. ghkit itself has zero dependency on `go-github`. It isn't in `go.mod`, isn't imported, and won't end up in your compiled binary unless you pull it in yourself. Pass whichever go-github major (or any other `func(*http.Client) T` factory) you want.
 
-For runnable starter programs, see [`examples/`](examples/): `static-pat`, `installation-token`, `backfill`, and `github-enterprise` are each a complete `main()` you can copy-paste.
+For runnable starter programs, see [`examples/`](examples/): `static-pat`, `installation-token`, `backfill`, `github-enterprise`, and `retry-on-flaky` are each a complete `main()` you can copy-paste.
 
 ## How?
 
 ```
 http.Client
  Throttle              (x/time/rate proactive)       [WithRequestsPerSecond]
   RateLimit            (go-github-ratelimit v2)      [default ON]
-   oauth2.Transport    (clones req, sets Auth)       [WithToken/WithTokenSource]
-    ETag               (hashes auth'd clone)         [WithETagCache]
-     Base              (*http.Transport,
+   Retry               (5xx + transient net errors)  [WithRetry]
+    oauth2.Transport   (clones req, sets Auth)       [WithToken/WithTokenSource]
+     ETag              (hashes auth'd clone)         [WithETagCache]
+      Base             (*http.Transport,
                         DisableCompression=true)     [WithBaseTransport]
 ```
 
-Each layer is optional. The stack is opt-in: `ghkit.HTTPClient(...)` only includes the layers you asked for. The order is load-bearing, though. ETag sits below the oauth2 layer so it hashes the request with the current Authorization header. Rate limiting sits above so it sees every outgoing call including ETag-triggered conditional GETs. The proactive throttle sits outermost so it caps issued RPS regardless of cache replays.
+Each layer is optional. The stack is opt-in: `ghkit.HTTPClient(...)` only includes the layers you asked for. The order is load-bearing, though. ETag sits below the oauth2 layer so it hashes the request with the current Authorization header. Rate limiting sits above so it sees every outgoing call including ETag-triggered conditional GETs. Retry sits below RateLimit so 429s are deferred to the rate-limit layer; sits above oauth2 so retried requests get the latest token via oauth2's per-call `Source.Token()`. The proactive throttle sits outermost so it caps issued RPS regardless of cache replays.
 
 The rate-limit layer's named options (`WithPrimaryLimitDetected`, `WithSecondaryLimitDetected`, `WithTotalSleepLimit`, `WithLogger`) cover the common callbacks. For upstream features ghkit does not curate, `ratelimit.WithUpstreamOptions(opts ...any)` forwards raw options to `gofri/go-github-ratelimit/v2`.
 
@@ -183,6 +184,44 @@ gh, err := ghkit.New(func(hc *http.Client) *github.Client {
 `WithEnterpriseURLs` requires both URLs to end with a trailing slash and returns an error otherwise. `UserAgent` can also be set at the transport level via `ghkit.WithUserAgent("my-app/1.0")`, which applies to every outbound request regardless of which SDK you wrap around `HTTPClient()`.
 </details>
 
+<details>
+<summary><b>Retry on transient failures (5xx, network errors)</b></summary>
+
+```go
+gh, err := ghkit.New(github.NewClient,
+    ghkit.WithToken(os.Getenv("GITHUB_TOKEN")),
+    ghkit.WithRetry(), // 3 attempts, 200ms..2s decorrelated jitter, idempotent methods only
+)
+```
+
+Tuned policy with POST opt-in via `Idempotency-Key`:
+
+```go
+import "github.com/pcanilho/go-github-kit/retry"
+
+gh, err := ghkit.New(github.NewClient,
+    ghkit.WithToken(token),
+    ghkit.WithRetry(
+        retry.WithMaxAttempts(5),
+        retry.WithBackoff(500*time.Millisecond, 10*time.Second),
+        retry.WithRetryOn(func(req *http.Request, resp *http.Response, err error) bool {
+            // Retry POST/PATCH when the caller asserted idempotency.
+            if req.Header.Get("Idempotency-Key") != "" {
+                if err != nil { return retry.IsTransientNetErr(err) }
+                return resp != nil && retry.IsRetryable5xx(resp.StatusCode)
+            }
+            // Otherwise the default behaviour.
+            if !retry.IsIdempotent(req.Method) { return false }
+            if err != nil { return retry.IsTransientNetErr(err) }
+            return resp != nil && retry.IsRetryable5xx(resp.StatusCode)
+        }),
+    ),
+)
+```
+
+429 is hard-excluded regardless of the predicate so `ratelimit` (the layer above) owns it. `Retry-After` is honored when present; if it exceeds `maxDelay` the call returns `(resp, retry.ErrRetryAfterExceedsMax)` and the caller owns drain+close on `resp`.
+</details>
+
 <details>
 <summary><b>Use only the etag sub-package in a hand-built stack</b></summary>
 
@@ -240,6 +279,8 @@ Gzip has to be disabled on the underlying transport, otherwise the hash domain d
 
 `etag.WithKeyScope` is required the moment you share a cache across identities, static PAT or JIT alike. Two callers hitting the same URL under different auth without a scope would race their bodies into the same key, and the library refuses to guess which one wins. Use the installation ID, a per-app scope, or any opaque string.
 
+Each retry attempt is a real HTTP call from the throttle layer's perspective. `WithRetry(retry.WithMaxAttempts(5))` combined with `WithRequestsPerSecond(2, 1)` means a worst-case failing request can briefly use 5x your nominal RPS budget. Size accordingly or leave `maxAttempts` at the default (3).
+
 ## Using a different go-github version
 
 The kit has no compile-time pin on `go-github`. Its main `go.mod` does not require `github.com/google/go-github`, so you choose the major. Two equally valid shapes:

doc.go

@@ -1,20 +1,26 @@
-// Package ghkit bundles ETag caching, rate limiting, and a proactive token
-// bucket behind a single options-pattern API. New is generic over the
-// returned client type, so ghkit has no compile-time dependency on any
-// specific GitHub SDK; pass any func(*http.Client) T factory at the call
-// site (canonically github.com/google/go-github's NewClient).
+// Package ghkit bundles ETag caching, rate limiting, retry on transient
+// failures, and a proactive token bucket behind a single options-pattern
+// API. New is generic over the returned client type, so ghkit has no
+// compile-time dependency on any specific GitHub SDK; pass any
+// func(*http.Client) T factory at the call site (canonically
+// github.com/google/go-github's NewClient).
 //
 // Transport stack (outer -> inner, each layer optional):
 //
 //	http.Client
 //	 UserAgent             (overwrites User-Agent)       [WithUserAgent]
 //	  Throttle             (x/time/rate proactive)       [WithRequestsPerSecond]
 //	   RateLimit           (go-github-ratelimit v2)      [default ON]
-//	    oauth2.Transport   (clones req, sets Auth)       [WithToken/WithTokenSource]
-//	     ETag              (hashes auth'd clone)         [WithETagCache]
-//	      Base             (*http.Transport,
+//	    Retry              (5xx + transient net errors)  [WithRetry]
+//	     oauth2.Transport  (clones req, sets Auth)       [WithToken/WithTokenSource]
+//	      ETag             (hashes auth'd clone)         [WithETagCache]
+//	       Base            (*http.Transport,
 //	                        DisableCompression=true)     [WithBaseTransport]
 //
+// Retry sits below RateLimit so 429s are deferred to the rate-limit layer;
+// sits above oauth2 so retried requests get the latest token via oauth2's
+// per-call Source.Token().
+//
 // The ETag precompute algorithm is the reason to use this kit. GitHub's
 // server-side ETag hash includes the Authorization header, so a passive
 // store-and-forward cache falls over under rotating auth (GitHub App

etag/options.go

@@ -1,6 +1,9 @@
 package etag
 
-import "log/slog"
+import (
+	"cmp"
+	"log/slog"
+)
 
 // Option configures a Transport. The interface form (rather than a bare
 // `func(*config)`) lets us evolve the API without a breaking change: new
@@ -43,9 +46,9 @@ func newConfig(opts []Option) *config {
 	for _, o := range opts {
 		o.apply(c)
 	}
-	if c.logger == nil {
-		c.logger = slog.Default()
-	}
+	// Silent by default: a nil logger becomes a discard logger so call sites
+	// can write unconditionally without nil-checks.
+	c.logger = cmp.Or(c.logger, slog.New(slog.DiscardHandler))
 	return c
 }
 
@@ -98,13 +101,10 @@ func WithMaxCacheBytes(n int64) Option {
 }
 
 // WithLogger supplies the slog.Logger the transport emits events to.
-// Default: slog.Default().
+// Pass nil (or omit the option) to silence the package; a nil logger is
+// replaced with slog.New(slog.DiscardHandler) at construction.
 func WithLogger(l *slog.Logger) Option {
-	return optionFunc(func(cfg *config) {
-		if l != nil {
-			cfg.logger = l
-		}
-	})
+	return optionFunc(func(cfg *config) { cfg.logger = l })
 }
 
 // WithDriftDetected registers a callback fired on each drift state

examples/README.md

@@ -26,5 +26,6 @@ When **copying an example into your own project**: drop the `replace` directive
 | `installation-token/`     | A GitHub App installation token via `oauth2.TokenSource`, with a shared ETag cache scoped per installation. |
 | `backfill/`               | Backfill / batch jobs: ETag cache + a client-side requests-per-second cap.    |
 | `github-enterprise/`      | Targeting GitHub Enterprise Server via `WithEnterpriseURLs` and a custom user agent. |
+| `retry-on-flaky/`         | `WithRetry` with a tuned backoff and a custom predicate that opts POST in via `Idempotency-Key`. |
 
 Each example reads its credentials from environment variables (e.g. `GITHUB_TOKEN`, `GITHUB_ENTERPRISE_TOKEN`) and calls `gh.Repositories.Get(...)` as a smoke test. They will fail at the API call without valid credentials. That's expected; they're starting templates, not standalone tools.

examples/retry-on-flaky/main.go

@@ -0,0 +1,51 @@
+// Retry on transient failures: 5xx, EOF, dial errors. Default policy is
+// idempotent-only; this example also shows opting POST in via
+// Idempotency-Key.
+package main
+
+import (
+	"context"
+	"fmt"
+	"log"
+	"net/http"
+	"os"
+	"time"
+
+	"github.com/google/go-github/v85/github"
+	ghkit "github.com/pcanilho/go-github-kit"
+	"github.com/pcanilho/go-github-kit/retry"
+)
+
+func main() {
+	gh, err := ghkit.New(github.NewClient,
+		ghkit.WithToken(os.Getenv("GITHUB_TOKEN")),
+		ghkit.WithRetry(
+			retry.WithMaxAttempts(5),
+			retry.WithBackoff(500*time.Millisecond, 10*time.Second),
+			retry.WithRetryOn(func(req *http.Request, resp *http.Response, err error) bool {
+				if req.Header.Get("Idempotency-Key") != "" {
+					if err != nil {
+						return retry.IsTransientNetErr(err)
+					}
+					return resp != nil && retry.IsRetryable5xx(resp.StatusCode)
+				}
+				if !retry.IsIdempotent(req.Method) {
+					return false
+				}
+				if err != nil {
+					return retry.IsTransientNetErr(err)
+				}
+				return resp != nil && retry.IsRetryable5xx(resp.StatusCode)
+			}),
+		),
+	)
+	if err != nil {
+		log.Fatalf("ghkit.New: %v", err)
+	}
+
+	repo, _, err := gh.Repositories.Get(context.Background(), "google", "go-github")
+	if err != nil {
+		log.Fatalf("Repositories.Get: %v", err)
+	}
+	fmt.Println(repo.GetFullName())
+}