Skip to content

Docker host tools backend: accurate disk usage, host summary + free-space API, -1 sentinels fail closed - #647

Merged
thinkbig1979 merged 7 commits into
mainfrom
run/hostbe2
Oct 14, 2026
Merged

thinkbig1979 merged 7 commits into
mainfrom
run/hostbe2

Conversation

@thinkbig1979

Copy link
Copy Markdown
Owner

Summary

Docker host tools, backend half (spec .agent-os/specs/2026-10-14-docker-host-tools), plus the -1 sentinel fix. The Host page UI is NOT in this PR.

  • cixa: Docker disk usage summed from LayersSize, so shared image layers are no longer counted once per image; unknown volume sizes are flagged.
  • qczn: image list shows real container counts; Delete disabled for in-use images.
  • p8sm: GET host summary and cleanup-candidates endpoints (HostSpaceService, fails closed).
  • 8hdg: cleanup policy categories, alert threshold, nextRunAt; migration 24.
  • yyat: POST /resources/host/free-space, under the exclusive operation lock, rechecks candidates under the lock, never forces, reports bytes actually freed; every removal route invalidates the host summary cache (AST guard).
  • gkgr: Volumes and Networks lists fail closed when Docker reports -1 or an inspect fails ("size unknown"/"unknown", Delete blocked); dead GetImageDiskUsage removed.

Test plan

  • Each bead reviewed with its own orchestrator mutant (all red)
  • Combined on main: go build + vet; go test handlers, services, database, cmd/server ok; tsc -b 0; knip:production 0; dashboard tab tests 75 green
  • CI

🤖 Generated with Claude Code

thinkbig1979 and others added 6 commits October 14, 2026 12:36
… volume sizes (agent-os-cixa)

GetDiskUsage summed img.Size per image, counting shared layers once per
image (dev laptop: 3.79 GB against LayersSize 3.21 GB; production 21.24 GB
against docker system df's 17.52 GB). It also added Docker's -1 "not
available" volume sizes and dereferenced a nil UsageData.

The arithmetic now lives in the pure summarizeDiskUsage, which uses
LayersSize for images, skips unknown volume sizes and reports them through
the new volumesSizeKnown field. GetDiskUsage delegates to it. Wire types
regenerated with tygo; two typed DashboardStats fixtures and the empty-host
dashboard JSON pin gain the field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… for in-use images (agent-os-qczn)

Docker's /images/json always reports Containers as -1 and the Go client
cannot ask for the count, so every image showed 0 containers. ListImages
now takes counts from one image-only DiskUsage call keyed by ID (the
ListVolumes pattern), through the pure toDockerImages. A failed read, or
an image missing from the snapshot, sets containersKnown false instead of
reporting 0.

With real counts, ImagesTab would have force-removed every in-use image.
Delete is now disabled for an in-use image with a "Used by N container(s)"
tooltip, an unknown count reads "unknown", and the delete path never sends
force. The test that pinned the force-remove behaviour is replaced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(agent-os-p8sm)

HostSpaceService reads one DiskUsage, the network list (one inspect per
network) and the stacks table, and derives both the per-category summary
and the removable items from that one snapshot. Summary is served from a
30 s snapshot behind a context-aware one-slot semaphore, so concurrent
pollers share one Docker read; Candidates always reads fresh. Invalidate()
drops the snapshot and a fetch in flight across it is not stored.

Candidate rules (spec "Reclaimable rules"): images with Containers == 0 in
the DiskUsage list (-1 is unknown, not unused); build cache neither in use
nor shared; containers exited/created/dead with no compose project label
at all; volumes with RefCount 0 not owned by a stack; networks with no
attached container, not built in (bridge, host, none, ingress,
docker_gwbridge) and not owned by a stack. A failed network inspect is
not a candidate. Stack-owned = the compose project label names any stack
row, running or not. disk.reclaimable sums the dialog's default
categories; reclaimableAll adds volumes.

GET /resources/host/summary and GET /resources/host/candidates answer 503
when Docker is unavailable and 500 on any other read failure, never a
zeroed body. The service is installed by SetHostSpaceService only when
Docker is present. Wiring Invalidate() into the prune and delete routes
is left to agent-os-yyat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ation 24 (agent-os-8hdg)

D98-A/C. The Docker cleanup policy gains one switch per category
(dangling images, unused tagged images, build cache, stopped containers,
volumes, networks) and an alert threshold, stored as seven settings keys.
An absent row gives today's behaviour (dangling images + build cache on,
5 GiB threshold); an unreadable row refuses, as the existing keys do.

- PUT /resources/cleanup/policy accepts a partial categories object and
  alertThresholdBytes (0..1 TiB), validates everything before writing, and
  writes the changed keys in one SetSettings call. An enabled policy with
  every category off is a 400. A categories/enabled change over an
  unreadable policy is refused (the all-off rule cannot be checked).
- GET/PUT report nextRunAt (RFC3339) only while enabled and armed. The
  scheduler records armedAt/interval in startLocked and clears them in
  stopLocked; NextRunAt is the next interval multiple after now.
- Execute honours the dangling-images and build-cache switches (orchestrator
  decision: skip what is off, record what ran). The other four categories
  are stored but not executed; that is agent-os-lr7x. A manual run reads the
  categories from the policy and refuses if it is unreadable.
- Migration 24 adds categories (default 'dangling_images,build_cache'),
  containers/volumes/networks_deleted and other_bytes_reclaimed to
  docker_cleanup_runs. The four new reclaim figures sit in a new
  pendingReclaimFigures table pinned at 0 until agent-os-lr7x.
- The new keys are classified as not sensitive and spelled with their
  constants at every read and write, so the settings-key guard sees them.
- Frontend: regenerated wire types; DockerCleanupRun.status gains 'skipped'
  (reserved, no writer yet); DockerCleanupPolicy gains categories,
  alertThresholdBytes and nextRunAt; api.ts request type extended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…echeck and measured bytes freed (agent-os-yyat)

POST /resources/host/free-space removes only the listed IDs that are still
candidates when re-read under the exclusive stack turn. Requested IDs that
are now in use, stack-owned or gone are skipped with a reason. Every removal
is unforced (images without PruneChildren), in the order containers, images,
networks, volumes, build cache (BuildCachePrune All=false). spaceReclaimed is
DiskUsage total before minus after, floored at 0, omitted when unmeasurable.
The run uses an operation-owned 15 min context; audit ActionPrune with
resource host_free_space; broadcast resource_changed/host_free_space.

hostSpaceFreer joins exclusiveGatedInterfaces, so FreeSpace without
acquireExclusiveLock fails TestContainerLockGuard_EveryContainerPruneTakes
TheExclusiveTurnFirst. Every handler that removes Docker objects (9 routes in
resource_mutations.go, runCleanup, freeSpace) now defers
invalidateHostSpace(), kept there by the new AST guard
TestHostSpaceGuard_EveryRemovalRouteInvalidates.

Request/result wire types in services/wire_types.go; generated-services.ts
regenerated with tygo. Host routes added to the no-Docker route list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…wn sizes and counts (agent-os-gkgr)

ListVolumes copied Docker's -1 "not computed" Size and RefCount through as
known values, so a volume rendered a size of -1 bytes and read as not in use,
and a failed DiskUsage read made every volume look unused. volumeUsageOf now
resolves the sentinel: DockerVolume gains inUseKnown, and sizeKnown is false
for -1. ListNetworks fell back to NetworkList's empty Containers (0) on a
failed inspect, so the network read as unused; it now passes -1 and sends
containersKnown false.

The Volumes tab says "size unknown" and the Networks tab "unknown", both sort
unknown after every known value, and Delete is disabled with a reason when
in-use state or attachments are unknown.

Deletes GetImageDiskUsage (0 callers, and it summed per-image Size, which
counts shared layers once per image).

The cleanup preview reclaim figure is left to agent-os-lr7x (orch decision).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

React Doctor found no new issues. 🎉

Reviewed by React Doctor for commit 13a14b1.

…t (ws-read-deadline gate)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thinkbig1979
thinkbig1979 merged commit bad28d9 into main Oct 14, 2026
15 checks passed
@thinkbig1979
thinkbig1979 deleted the run/hostbe2 branch October 14, 2026 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant