Fix logs viewer going empty (aiodocker log-connection leak) - #105
Merged
Merged
Conversation
service_logs and manager_logs_stream read container logs through a single
aiodocker client cached for the manager's whole lifetime (_get_aio_client /
_AIO_CLIENT). Each follow=true stream that the browser abandoned (closing the
logs page) left its connection unreleased ("Unclosed connector"); over time the
client's connection pool exhausted and every log request hung -> the dashboard
logs page rendered empty (200 with an empty body). A manager restart cleared it
temporarily. Reproduced on the Docker-29 GCP hosts, but the leak is
version-agnostic.
Use a fresh aiodocker client per log request and close it in `finally`, so the
connection is always released -- including when the async generator is cancelled
on client disconnect. Removes the degrading cached singleton entirely.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
The dashboard logs page renders empty for a running service (200 response, empty body), while
docker logs <container>on the host shows plenty of lines. Seen on the Docker-29 GCP hosts; a manager restart temporarily fixes it.Root cause
service_logsandmanager_logs_streamread logs through a single aiodocker client cached for the manager's entire lifetime (_get_aio_client()→ class-level_AIO_CLIENT). Everyfollow=truestream the browser abandons (closing the logs page) leaves its connection unreleased — theUnclosed connectorwarnings — so over time the client's aiohttp connection pool exhausts and new log requests hang, which surfaces as an empty logs page.Verified: the
service_logsgenerator returns instantly in a fresh process/client but hangs in the day-old running manager; direct uvicorn hit times out; a restart restores it.Fix
Use a fresh
aiodocker.Docker()per log request, closed infinally(runs even when the async generator is cancelled on client disconnect), and drop the cached singleton. The connection is always released, so nothing accumulates.Log requests are infrequent, so a client per request is negligible overhead; and a per-request client also avoids the stale-loop pitfalls of a long-lived cached one.
Notes
🤖 Generated with Claude Code