Skip to content

MCP fixes from staging e2e, clip hard-link fix - #1921

Merged
luke-lombardi merged 8 commits into
mainfrom
ll/mcp-install-host
Sep 24, 2026
Merged

luke-lombardi merged 8 commits into
mainfrom
ll/mcp-install-host

Conversation

@luke-lombardi

@luke-lombardi luke-lombardi commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

From the staging end-to-end run:

  • mcp install writes the HTTP host the gateway reports (was the gRPC host on hosted deployments).
  • MCP delete_app stops deployments and containers (ran the REST DeleteApp handler in-process); it only soft-deleted the app row.
  • Deleting a database removes its app record; the dashboard kept a stopped ghost app per deleted database.
  • clip bumped to master for the hard-link-to-symlink fix (Hard links copy the whole target node, symlinks included clip#52); nixpacks-built images had unreadable symlinks.
  • runsc restore test: fake reports a pid only after the fake restore ran (flaky on slow runners).

The container capability change that was here is now its own PR.

luke-lombardi and others added 2 commits September 23, 2026 15:48
cfg.http_url guessed the REST host from the gRPC host, which is wrong on
hosted deployments (gateway.stage.beam.cloud vs app.stage.beam.cloud);
ServiceClient.http already resolves it from ExportWorkspaceConfig.

Co-authored-by: Cursor <cursoragent@cursor.com>
It only soft-deleted the app record, leaving every version active. It now
runs the REST DeleteApp handler in-process, which stops deployments and
containers first.

Co-authored-by: Cursor <cursoragent@cursor.com>
luke-lombardi and others added 4 commits September 23, 2026 16:20
The deployments and secrets went but the app stayed, so the dashboard kept
a stopped ghost app for every deleted database.

Co-authored-by: Cursor <cursoragent@cursor.com>
…indexing

- CAP_NET_BIND_SERVICE and CAP_SETPCAP join the base capability set, matching
  Docker's defaults. nginx and every other image that listens on 80/443
  failed with EACCES.
- clip: a hard link replays as a copy of its target node, so a hard link to a
  symlink stays a symlink. nix's optimised store hard-links symlinks into
  /nix/store/.links, which left nixpacks-built images with unreadable
  libz.so.1 and friends.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Restore returns on the first state with a pid; the fake reported one
immediately, so on a slow runner the assertion read the bundle before the
fake restore had copied it.

Co-authored-by: Cursor <cursoragent@cursor.com>
luke-lombardi and others added 2 commits September 23, 2026 20:16
Only CAP_NET_BIND_SERVICE is needed for images that listen on 80/443.
CAP_SETPCAP came along as 'match Docker's defaults' but nothing here needs
it, and it is the one capability in that set that exists to manipulate
capabilities; keep the base set at the minimum the feature requires. The
docker-in-docker runtime keeps its own, larger set.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@luke-lombardi luke-lombardi changed the title fix: mcp install host MCP fixes from staging e2e, clip hard-link fix Sep 24, 2026
@luke-lombardi
luke-lombardi merged commit a7318ea into main Sep 24, 2026
4 checks passed
@luke-lombardi
luke-lombardi deleted the ll/mcp-install-host branch September 24, 2026 00:25
luke-lombardi added a commit that referenced this pull request Sep 24, 2026
## What

The image service accepts a **git source**
(`BuildImageRequest.git_source` / `VerifyImageBuildRequest.git_source`):
`{repo_url, ref, token, working_dir, dockerfile_path, start_command,
build_command}`.

- **Gateway** pins the ref to a commit with `git ls-remote` up front, so
a bad repo or ref fails at verify time with a precise message
(`repository … not found or not accessible`, `ref "x" not found in …`).
The image id hashes `{clip version, repo, commit, working dir,
dockerfile path, start/build command}`, never the token or runtime env,
so the same commit is the same image and a new push is a new one.
- **Build worker** clones the commit (GIT_ASKPASS for the token, nothing
in argv or `.git/config`) onto the disk-backed spool dir and builds, in
Railway's resolution order:
1. `dockerfile_path` (request or `railway.json`/`railway.toml`
`build.dockerfilePath`), else
  2. the repository's `Dockerfile`, else
3. a Dockerfile **rendered in-tree** (`pkg/worker/git_dockerfile.go`)
from the detected stack: Python (`requirements.txt`/`pyproject.toml`;
Django/FastAPI/Flask server detection), Node (lockfile → npm/yarn/pnpm,
`build`/`start` scripts, `engines.node`), Go (multi-stage on the `go`
directive's version), static (`index.html` on nginx). Official images
only; start/build commands from the request, railway config or Procfile
win; gunicorn gets `--bind 0.0.0.0:$PORT`. Anything else: `repository
has no Dockerfile and no recognised stack; add a Dockerfile`.

The checkout is the buildah context, so the repository's own multi-stage
Dockerfiles work unchanged.
- `docker/Dockerfile.worker`: adds `git`. Nothing else.
- `ContainerRequest.BuildOptions.GitSource` travels via proto;
`IsBuildRequest` recognises it; `PrivateWorkerRequest` strips the token.
- Bumps clip to the hard-link fix (`1c772eb`, already on #1921).

This is what lets internal-api drop its sandbox builder
(beam-cloud/internal-api#309): the deploy agent calls `/images/build`
with a `gitSource`, then `/stubs` + `/stubs/deploy`.

## Verified locally (k3d)

Through internal-api's deploy path against the local gateway, then HTTP
to the deployed pod:

| repo | path | build | result |
|---|---|---|---|
| traefik/whoami | repo Dockerfile (multi-stage) | 72s | image built |
| railwayapp-templates/fastapi | rendered `python:3.11-slim`,
`railway.json` startCommand (hypercorn) | 22s | **200** |
| railwayapp-templates/expressjs | rendered `node:20-slim`, yarn + `tsc`
build | 32s | **200** |
| railwayapp-templates/flask | rendered `python:3.11-slim`, gunicorn
detected from `Flask(` | 12s | **200** |
| beam-cloud/does-not-exist | — | — | `repository … not found or not
accessible` at verify |
| expressjs @ `nope` | — | — | `ref "nope" not found in …` at verify |
| flask, `working_dir: x` | — | — | `working directory "x" not found in
repository` |

Tests: image-id keying, request normalisation, Dockerfile resolution
order (explicit → railway config → repo → rendered, path escape, TOML
quoting), rendered Dockerfiles for each stack, a real `git` checkout by
commit with branch fallback and error cleanup, proto round trip incl.
token stripping.

Not in this PR: `CAP_NET_BIND_SERVICE` for services binding :80 is
#1922.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jackf-beamcloud <333347262+jackf-beamcloud@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants