okhttp-gson ApiClient cannot deserialize responses larger than 2GB

[klin1344]

Describe the bug
The okhttp-gson ApiClient cannot handle responses larger than 2GB.

Client Version
22.0.1-legacy is what we're currently using, but this issue occurs in all versions

Kubernetes Version
1.30.9

Java Version
Java 21

To Reproduce

1. Set up a SharedIndexInformer for a resource in a cluster where the total size of all resources > 2GB.
2. The initial list() in the ReflectorRunnable will fail with:

class io.kubernetes.client.openapi.models.V1Pod#Reflector loop failed unexpectedly

java.lang.IllegalArgumentException: byteCount: 2293832235
  at okio.Buffer.readString(Buffer.kt:306)
  at okio.Buffer.readString(Buffer.kt:302)
  at okio.RealBufferedSource.readString(RealBufferedSource.kt:96)
  at okhttp3.ResponseBody.string(ResponseBody.kt:187)
  at io.kubernetes.client.openapi.ApiClient.deserialize(ApiClient.java:758)
  at io.kubernetes.client.openapi.ApiClient.handleResponse(ApiClient.java:978)
  at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:905)
  at io.kubernetes.client.informer.SharedInformerFactory$1.list(SharedInformerFactory.java:271)
  at io.kubernetes.client.informer.cache.ReflectorRunnable.run(ReflectorRunnable.java:91)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
  at java.lang.Thread.run(Thread.java:1583)

Expected behavior
The ApiClient can deserialize responses greater than 2GB.

Additional context
This issue occurs because the response body is converted to a String here in the ApiClient, which hits this validation in okhttp:

require(byteCount >= 0 && byteCount <= Integer.MAX_VALUE) { "byteCount: $byteCount" }

because of Java's String max length limit of Integer.MAX_VALUE which is 2^31 - 1.

[yue9944882]

https://github.com/kubernetes-client/java/blob/master/util/src/main/java/io/kubernetes/client/informer/cache/ReflectorRunnable.java#L90-L92

The proper fix is to add pagination to the list call in the reflector..

[klin1344]

https://github.com/kubernetes-client/java/blob/master/util/src/main/java/io/kubernetes/client/informer/cache/ReflectorRunnable.java#L90-L92

The proper fix is to add pagination to the list call in the reflector..

@yue9944882 Yes, we have already implemented a ListerWatcher that does pagination as a fix for our use case. However, similar to the official golang client pagination implementation, the ResourceVersion can expire mid-pagination, and the fallback mechanism is to do a full list.

Thus, pagination will help, but it is still necessary to support deserialization of a large response in case of a fallback.

[brendandburns]

OpenAPITools/openapi-generator#21115 has merged, we'll pick this up in the next release.

Also @klin1344 if you wanted to commit the pagination support to our ListerWatcher that would be great.

[klin1344]

OpenAPITools/openapi-generator#21115 has merged, we'll pick this up in the next release.

Also @klin1344 if you wanted to commit the pagination support to our ListerWatcher that would be great.

@brendandburns Sure, the paginated ListerWatcher implementation is simpler than the one in the golang client right now, but I will still find some time to try to upstream it to this project.

[klin1344]

PR to bump the openapi-generator version here kubernetes-client/gen#280 so this gets picked up in the next release

[brendandburns]

Cool, do you want us to regenerate a patch release for this? Or are you willing to wait for the next Kubernetes version?

[klin1344]

@brendandburns Yes, if we could generate a patch release that would be great.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[klin1344]

This has been fixed with 24.0.0 which includes OpenAPITools/openapi-generator#21115 and kubernetes-client/gen#280 .