Repository navigation
feat: run the testcontainers and e2e suites in containers - #7306
Merged
Merged
Conversation
Code Review CompleteThe automated review ran but did not post an updated summary — this usually means no new issues were found since the previous review. If you've pushed changes and want a fresh pass, comment |
otavio
force-pushed
the
feat/docker-test-runner
branch
2 times, most recently
from
October 2, 2026 23:14
76b0184 to
beb3c03
Compare
tests/, server/api/store/pg and the pg-backed service tests need the Docker socket, so they ran on the host, with a host Go, Node and buildx setup that differs per machine and breaks on NixOS. exec server cannot run them either: it mounts part of the repo under /go/src, so the bind mounts in the suites' compose files resolve to nothing on the daemon, and the image builds miss agent/, gateway/ and the root module. The test service (profile tools) reuses the server development image and mounts the whole workspace at its host path, so those bind mounts resolve. Each other setting answers a failure: - the socket comes from SHELLHUB_DEV_DOCKER_SOCKET, the path the wrapper resolves for the server under rootless Docker, and testcontainers gets it as TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE, as the server does. The runner keeps the image's root user: development runs on rootless Docker, which maps that root to the calling user, so files it writes in the checkout stay the user's. On a rootful daemon they are root's, as the rest of the dev stack's already are. - network_mode host: the harness reaches the stack at http://localhost:<port>, and TESTCONTAINERS_HOST_OVERRIDE=localhost points testcontainers there too; inside a container it otherwise guesses the bridge gateway. - entrypoint []: the development entrypoint generates keys and starts air instead of the command. - the shellhub-tools-cache volume keeps the Go and npm caches, shared by every instance. - TESTCONTAINERS_RYUK_DISABLED: CI's unit test step sets it for every module (qa.yml), and the integration suite's TestMain sets it too. - docker and docker-buildx come from docker:29.8.1-cli, the image the e2e runner takes its CLI from, so both move together. Without buildx the compose library inside the test binary falls back to the classic builder and fails with "the --mount option requires BuildKit". - GOFLAGS=-buildvcs=false: a T3 worktree's .git points outside the mounted workspace. - SHELLHUB_LICENSE_FILE is mounted at its own path, interpolated from the same env files the stack reads, so the enterprise suites find it.
e2e.sh needed a host Go for the stack and a host Node and browser for Playwright. On NixOS the browser Playwright downloads exits 127 (libglib-2.0.so.0 missing). It now runs tests/cmd/stack in the test runner and Playwright in the e2e image: the official Playwright image at the @playwright/test version package-lock.json resolved, so the browser revision always matches, plus the Docker CLI and Compose plugin the specs' seeding helpers call, reaching the daemon through the same socket the test runner mounts. The wrapper exports that version, so a bare `compose run e2e` builds the same image as e2e.sh. The CLI comes from docker:29.8.1-cli, as in the server development image. The Dockerfile sits in ui/, where Dependabot's docker updates look, and its own .dockerignore keeps the build context empty instead of sending node_modules. The test and e2e services live in docker-compose.dev.yml, so e2e.sh stops with a pointer to SHELLHUB_ENV=development when the wrapper does not load them, and passes any other wrapper error through as it is. CI sets E2E_RUNNER=host and keeps its runner toolchain: it already installs Go, Node and the browser, and building the server development image there would only add time.
seed.ts reached the stack's containers as shellhub-e2e-$E2E_STACK_NAME, a name stack up never exported. A Playwright run started outside e2e.sh test, or in a container that did not inherit the caller's variable, seeded against shellhub-e2e-default and failed with "service server is not running". stack up now exports the compose project itself as E2E_COMPOSE_PROJECT, and seed.ts requires it, so the shellhub-e2e- prefix lives only in tests/cmd/stack.
otavio
force-pushed
the
feat/docker-test-runner
branch
from
October 2, 2026 23:26
beb3c03 to
b7762e9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The testcontainers suites and the console e2e suite ran only on the host, so every machine needed its own Go, Node, buildx and browser. On NixOS the browser Playwright downloads cannot start.
exec servercannot run these suites either. It mounts part of the repo under/go/src, so the bind mounts in the suites' compose files resolve to nothing on the daemon.stack upexports the compose project asE2E_COMPOSE_PROJECT, andseed.tsrequires it, so a Playwright run against a named stack seeds that stack and notshellhub-e2e-default. Both runners follow the rootless support on master. They mountSHELLHUB_DEV_DOCKER_SOCKETlike the server does.e2e.shstops with a pointer toSHELLHUB_ENV=developmentwhen the services are not loaded.CI is unchanged.
ui-e2e.ymlsetsE2E_RUNNER=hostand keeps the runner's toolchain.Evidence
Before: these suites had no container path. On NixOS,
npm run e2e:testfailed with Chromium exiting 127 onlibglib-2.0.so.0.After: each suite passes in the containers on a rootless daemon, through the wrapper with fix: check the ports the stack publishes in the rootless guard #7313, and the runs leave every file in the checkout owned by the user. The table's runs also passed earlier on a rootful daemon:
server/api/store/pgtests/:TestLoginWithoutANamespace,TestDeviceListRefusesAStatusThePostgresEnumCannotHoldcloud/pkg/store/pgwithGOWORK=offe2e.sh test auth.spec.tsE2E_STACK_NAME=review e2e.sh test auth.spec.ts -g "removed member|signs in"e2e.shwith noSHELLHUB_ENV=developmentbats tests/compose/Merge Danger
Door: two-way
Blast Radius: dev tooling
dockerCLI now comes fromdocker:29.8.1-cliinstead of Alpine's package, and it gains buildx. Run./bin/docker-compose build server.bin/docker-composeexportsSHELLHUB_WORKSPACEandE2E_PLAYWRIGHT_VERSION, and Compose interpolates them for every service. Thetestande2eservices sit in thetoolsprofile, soupnever starts them.e2e.sh test auth.spec.ts(8 specs) pass through the wrapper with the runners as root, and the checkout keeps your ownership.network_mode: hostneeds Docker Desktop's host networking on macOS.testservice mounts from the env files, and the Stripe keys, whiche2e.shpasses through.