Skip to content

feat: run the testcontainers and e2e suites in containers - #7306

Merged
otavio merged 3 commits into
masterfrom
feat/docker-test-runner
Oct 2, 2026
Merged

otavio merged 3 commits into
masterfrom
feat/docker-test-runner

Conversation

@otavio

@otavio otavio commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

Summary

The testcontainers suites and the console e2e suite ran only on the host, so every machine needed its own Go, Node, buildx and browser. On NixOS the browser Playwright downloads cannot start. exec server cannot run these suites either. It mounts part of the repo under /go/src, so the bind mounts in the suites' compose files resolve to nothing on the daemon.

./bin/docker-compose run --rm -T -w <dir> test go test ./...
  test   server development image, docker and buildx from docker:29.8.1-cli
         workspace mounted at its host path, the daemon's socket, host network
         the image's root, which rootless Docker maps to you
         Go and npm caches in the shellhub-tools-cache volume

ui/scripts/e2e.sh test [--edition=<edition>] [playwright args]
  stack up     in test
  playwright   in e2e: the official image at the version package-lock.json resolved,
               plus docker and the compose plugin from docker:29.8.1-cli

stack up exports the compose project as E2E_COMPOSE_PROJECT, and seed.ts requires it, so a Playwright run against a named stack seeds that stack and not shellhub-e2e-default. Both runners follow the rootless support on master. They mount SHELLHUB_DEV_DOCKER_SOCKET like the server does. e2e.sh stops with a pointer to SHELLHUB_ENV=development when the services are not loaded.

CI is unchanged. ui-e2e.yml sets E2E_RUNNER=host and keeps the runner's toolchain.

Evidence

  • Before: these suites had no container path. On NixOS, npm run e2e:test failed with Chromium exiting 127 on libglib-2.0.so.0.
    After: each suite passes in the containers on a rootless daemon, through the wrapper with fix: check the ports the stack publishes in the rootless guard #7313, and the runs leave every file in the checkout owned by the user. The table's runs also passed earlier on a rootful daemon:

    Run Result
    server/api/store/pg ok
    tests/: TestLoginWithoutANamespace, TestDeviceListRefusesAStatusThePostgresEnumCannotHold ok, images built with BuildKit
    cloud/pkg/store/pg with GOWORK=off ok
    e2e.sh test auth.spec.ts 8 passed
    E2E_STACK_NAME=review e2e.sh test auth.spec.ts -g "removed member|signs in" 2 passed; the seeding helper reaches the named stack
    e2e.sh with no SHELLHUB_ENV=development stops and names the setting
    bats tests/compose/ all ok; the wrapper only adds two exports

Merge Danger

Door: two-way

Blast Radius: dev tooling

  • The server development image changes. Its docker CLI now comes from docker:29.8.1-cli instead of Alpine's package, and it gains buildx. Run ./bin/docker-compose build server.
  • bin/docker-compose exports SHELLHUB_WORKSPACE and E2E_PLAYWRIGHT_VERSION, and Compose interpolates them for every service. The test and e2e services sit in the tools profile, so up never starts them.
  • Development targets rootless Docker. On a rootful daemon the runners still work, but files they write in the checkout belong to root, as the rest of the dev stack's already do.
  • Under rootless Docker, the wrapper's port guard from feat: run the dev stack under rootless Docker #7309 aborts every command in a worktree instance, these runners included. The cause is port 80, which an instance does not publish. fix: check the ports the stack publishes in the rootless guard #7313 fixes the guard. With it, on a rootless daemon, an integration test and e2e.sh test auth.spec.ts (8 specs) pass through the wrapper with the runners as root, and the checkout keeps your ownership.
  • network_mode: host needs Docker Desktop's host networking on macOS.
  • The enterprise and cloud e2e editions have not run in the containers yet. They need the license file, which the test service mounts from the env files, and the Stripe keys, which e2e.sh passes through.

@otavio
otavio requested review from a team as code owners October 2, 2026 17:31
@github-actions

github-actions Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Code Review Complete

The automated review ran but did not post an updated summary — this usually means no new issues were found since the previous review. If you've pushed changes and want a fresh pass, comment /review.

View job

@otavio
otavio force-pushed the feat/docker-test-runner branch 2 times, most recently from 76b0184 to beb3c03 Compare October 2, 2026 23:14
otavio added 3 commits October 2, 2026 20:20
tests/, server/api/store/pg and the pg-backed service tests need the Docker socket, so they ran on
the host, with a host Go, Node and buildx setup that differs per machine and breaks on NixOS.
exec server cannot run them either: it mounts part of the repo under /go/src, so the bind mounts
in the suites' compose files resolve to nothing on the daemon, and the image builds miss agent/,
gateway/ and the root module.

The test service (profile tools) reuses the server development image and mounts the whole
workspace at its host path, so those bind mounts resolve. Each other setting answers a failure:

- the socket comes from SHELLHUB_DEV_DOCKER_SOCKET, the path the wrapper resolves for the server
  under rootless Docker, and testcontainers gets it as TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE, as
  the server does. The runner keeps the image's root user: development runs on rootless Docker,
  which maps that root to the calling user, so files it writes in the checkout stay the user's.
  On a rootful daemon they are root's, as the rest of the dev stack's already are.
- network_mode host: the harness reaches the stack at http://localhost:<port>, and
  TESTCONTAINERS_HOST_OVERRIDE=localhost points testcontainers there too; inside a container it
  otherwise guesses the bridge gateway.
- entrypoint []: the development entrypoint generates keys and starts air instead of the command.
- the shellhub-tools-cache volume keeps the Go and npm caches, shared by every instance.
- TESTCONTAINERS_RYUK_DISABLED: CI's unit test step sets it for every module (qa.yml), and the
  integration suite's TestMain sets it too.
- docker and docker-buildx come from docker:29.8.1-cli, the image the e2e runner takes its CLI
  from, so both move together. Without buildx the compose library inside the test binary falls
  back to the classic builder and fails with "the --mount option requires BuildKit".
- GOFLAGS=-buildvcs=false: a T3 worktree's .git points outside the mounted workspace.
- SHELLHUB_LICENSE_FILE is mounted at its own path, interpolated from the same env files the
  stack reads, so the enterprise suites find it.
e2e.sh needed a host Go for the stack and a host Node and browser for Playwright. On NixOS the
browser Playwright downloads exits 127 (libglib-2.0.so.0 missing).

It now runs tests/cmd/stack in the test runner and Playwright in the e2e image: the official
Playwright image at the @playwright/test version package-lock.json resolved, so the browser
revision always matches, plus the Docker CLI and Compose plugin the specs' seeding helpers call,
reaching the daemon through the same socket the test runner mounts. The wrapper exports that
version, so a bare `compose run e2e` builds the same image as e2e.sh. The CLI comes from
docker:29.8.1-cli, as in the server development image. The Dockerfile sits in ui/, where
Dependabot's docker updates look, and its own .dockerignore keeps the build context empty instead
of sending node_modules.

The test and e2e services live in docker-compose.dev.yml, so e2e.sh stops with a pointer to
SHELLHUB_ENV=development when the wrapper does not load them, and passes any other wrapper error
through as it is.

CI sets E2E_RUNNER=host and keeps its runner toolchain: it already installs Go, Node and the
browser, and building the server development image there would only add time.
seed.ts reached the stack's containers as shellhub-e2e-$E2E_STACK_NAME, a name stack up never
exported. A Playwright run started outside e2e.sh test, or in a container that did not inherit the
caller's variable, seeded against shellhub-e2e-default and failed with "service server is not
running". stack up now exports the compose project itself as E2E_COMPOSE_PROJECT, and seed.ts
requires it, so the shellhub-e2e- prefix lives only in tests/cmd/stack.
@otavio
otavio force-pushed the feat/docker-test-runner branch from beb3c03 to b7762e9 Compare October 2, 2026 23:26
@otavio
otavio merged commit 9687b12 into master Oct 2, 2026
43 checks passed
@otavio
otavio deleted the feat/docker-test-runner branch October 2, 2026 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant