Skip to content

fix(tlsnotary): bake wstcp into the image + publish the proxy port range#949

Open
Shitikyan wants to merge 5 commits into
stabilisationfrom
fix/tlsn-wstcp-proxy-reachability
Open

fix(tlsnotary): bake wstcp into the image + publish the proxy port range#949
Shitikyan wants to merge 5 commits into
stabilisationfrom
fix/tlsn-wstcp-proxy-reachability

Conversation

@Shitikyan

@Shitikyan Shitikyan commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Problem

TLSNotary verifications (e.g. Discord) fail with nginx 502 / CloseEvent {code: 1006} on the prover. The notary itself is healthy (/tlsnotary/health → 200, port 7047) — "connected to verifier" succeeds — but the next hop, wss://<node>/tlsn/<port>/, 502s.

Root cause is two layers, both confirmed live on node3:

  1. wstcp can't spawn. The TLSNotary flow spawns a wstcp websocket→TCP proxy on demand (proxyManager.ts). The slim runtime image has no Rust toolchain, so ensureWstcp()'s cargo install wstcp fallback fails silently — nothing ever binds the proxy port. Confirmed: docker exec demos-node → no wstcp, no cargo.

  2. The proxy port isn't reachable from the host. Even with wstcp, the proxy binds a dynamic port in 55000-57000 inside the container, but docker-compose.yml publishes none of that range. nginx forwards /tlsn/<port>/127.0.0.1:<port> → dead upstream → 502. Manually binding a single port (55688) is a band-aid that breaks on the next session's different port — which is why this kept recurring.

Fix

  1. Dockerfile — bake wstcp into the image via a dedicated FROM rust:1-slim AS wstcp stage and copy the binary to the exact path proxyManager.ts probes ($HOME/.cargo/bin/wstcp, HOME=/app). No toolchain in the runtime image; the proxy spawns instantly. (Validated: the stage builds, wstcp 0.2.1 runs.)
  2. constants.tsPORT_MIN/PORT_MAX are now overridable via TLSNOTARY_PROXY_PORT_MIN/MAX (defaults unchanged) so the allocation range can be narrowed to a host-publishable window.
  3. docker-compose.yml — publish a narrow, localhost-bound proxy window (127.0.0.1:55000-55063, env-driven). The allocation range and the published range read the same vars so they can't drift, and binding to 127.0.0.1 keeps the ports reachable by the host reverse proxy only, not the public internet.

Deploy on node3

  • Rebuild the node image (now contains wstcp) and recreate the demos-node container so the new ports: mapping applies. (Immediate unblock without a rebuild: docker cp a prebuilt wstcp into /app/.cargo/bin/wstcp and add the port-range mapping.)
  • If node3 uses a custom compose, mirror the TLSNOTARY_PROXY_PORT_MIN/MAX env + the 127.0.0.1:<min>-<max> port mapping there.

Refs: docs/runbooks/wstcp-reachability-check.md (this makes the containerized default actually reachable instead of "internal only").

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added environment-driven TLSNotary proxy port range configuration (with sensible defaults) for verification traffic.
    • Updated container setup so the proxy helper is available in runtime builds without requiring additional tooling.
    • Published the configured proxy port range on the local host only (loopback), improving compatibility with the local reverse proxy while keeping exposure limited to localhost.
  • Refactor
    • Centralized proxy port window configuration so all port allocation logic uses the same shared settings.

TLSNotary verifications failed with nginx 502 / CloseEvent 1006 because
the wstcp websocket→TCP proxy could never be reached, on two layers:

1. The runtime image ships no Rust toolchain, so ensureWstcp()'s
   on-demand `cargo install wstcp` fallback silently failed and the proxy
   never spawned. Bake the binary in via a dedicated `rust` build stage
   and copy it to the exact path proxyManager.ts probes
   ($HOME/.cargo/bin/wstcp).

2. The proxy binds a dynamic port in 55000-57000 inside the container,
   but docker-compose published none of them — so nginx forwarding
   /tlsn/<port>/ to 127.0.0.1:<port> hit a dead upstream. Publish a
   narrow, localhost-bound, env-configurable window; the allocation range
   (PORT_MIN/MAX, now env-overridable) and the published range read the
   same vars so they cannot drift.

Manually binding one host port was a band-aid that broke on the next
session's different port. This fixes both layers durably.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@qodo-code-review

Copy link
Copy Markdown
Contributor

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@Shitikyan, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 36 minutes and 41 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e8e852d5-0a88-4ec2-b90f-a0b48358c80b

📥 Commits

Reviewing files that changed from the base of the PR and between b56b20c and 81b8fe9.

📒 Files selected for processing (2)
  • Dockerfile
  • docker-compose.devnet.yml

Walkthrough

The Docker image now bakes in wstcp, and the TLSNotary proxy port range is configurable through environment variables. The node service in docker-compose now uses matching defaults and publishes that port range on localhost.

Changes

TLSNotary wstcp runtime and port wiring

Layer / File(s) Summary
Proxy port contract
src/features/tlsnotary/constants.ts, docker-compose.yml
PORT_CONFIG reads TLSNotary proxy bounds from environment variables, and the node service sets and publishes matching port-range defaults on localhost.
Bake wstcp into the runtime image
Dockerfile
A Rust build stage installs wstcp, and the runtime stage copies the binary into /app/.cargo/bin/wstcp.
Share port config imports
src/features/tlsnotary/portAllocator.ts, src/features/tlsnotary/proxyManager.ts
PORT_CONFIG is imported from ./constants in the TLSNotary port allocator and proxy manager modules.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 I hopped through layers, neat and bright,
and packed my wstcp just right.
The ports line up, the carrots glow,
localhost tunnels start to flow.
🥕

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the core changes: baking wstcp into the image and publishing the TLSNotary proxy port range.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/tlsn-wstcp-proxy-reachability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jun 26, 2026

Copy link
Copy Markdown

Greptile Summary

This PR updates TLSNotary proxy packaging and port exposure. The main changes are:

  • Adds a dedicated Docker build stage that installs pinned wstcp and copies it into the runtime image.
  • Makes the TLSNotary proxy allocation window configurable through TLSNOTARY_PROXY_PORT_MIN and TLSNOTARY_PROXY_PORT_MAX.
  • Publishes matching localhost-bound proxy port ranges in the main and devnet Docker Compose files.
  • Updates portAllocator.ts and proxyManager.ts to use the shared TLSNotary PORT_CONFIG.

Confidence Score: 5/5

The change is narrowly scoped to TLSNotary proxy packaging, configuration, and Compose port exposure.

No correctness issues were identified in the changed Docker, Compose, or TLSNotary proxy allocation paths.

T-Rex T-Rex Logs

What T-Rex did

  • The wstcp image build and test logs were inspected across base 01ef0a0 and head 81b8fe9, revealing docker is not found (exit code 127) and preventing runtime execution proofs.
  • The proxy port-range configuration was reviewed for base 01ef0a0 and head 81b8fe9, confirming default 55000-57000 and override 62000-62002, with allocations [62000, 62001, 62002, null], and noting that Docker Compose rendering is blocked due to docker not being installed, so validation relied on a grep fallback.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (4): Last reviewed commit: "fix(tlsnotary): publish the wstcp proxy ..." | Re-trigger Greptile

Comment thread src/features/tlsnotary/constants.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
Dockerfile (1)

150-151: 📐 Maintainability & Code Quality | 🔵 Trivial

Pin the wstcp version for reproducible builds.

cargo install wstcp fetches the latest release (currently 0.2.1) at build time, making builds non-reproducible. The wstcp crate is available on crates.io with versions ranging from 0.1.0 to 0.2.1. To prevent unexpected behavior changes or build failures from upstream updates, pin the specific version and use --locked.

♻️ Suggested update
 FROM rust:1-slim AS wstcp
-RUN cargo install wstcp --root /wstcp
+RUN cargo install wstcp --version 0.2.1 --locked --root /wstcp
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Dockerfile` around lines 150 - 151, The wstcp install step is unpinned and
can pull a changing latest release, so update the Dockerfile’s wstcp build stage
to install a specific crates.io version of wstcp and use --locked for
reproducible builds. Keep the change focused on the FROM rust:1-slim AS wstcp
stage and the cargo install wstcp command so the image always builds against the
same crate release.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/features/tlsnotary/constants.ts`:
- Around line 60-61: `PORT_CONFIG` is duplicated and the allocator is still
using the hardcoded range instead of the env-driven values. Update
`portAllocator.ts` to remove its local `PORT_CONFIG` and import the shared one
from `./constants`, then make `proxyManager.ts` also use `PORT_CONFIG` from
`./constants` so there is a single source of truth for the TLSNotary port range.

---

Nitpick comments:
In `@Dockerfile`:
- Around line 150-151: The wstcp install step is unpinned and can pull a
changing latest release, so update the Dockerfile’s wstcp build stage to install
a specific crates.io version of wstcp and use --locked for reproducible builds.
Keep the change focused on the FROM rust:1-slim AS wstcp stage and the cargo
install wstcp command so the image always builds against the same crate release.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 93605c23-fd3e-4cf7-ac95-581870d043e0

📥 Commits

Reviewing files that changed from the base of the PR and between 01ef0a0 and f6c3acc.

📒 Files selected for processing (3)
  • Dockerfile
  • docker-compose.yml
  • src/features/tlsnotary/constants.ts

Comment thread src/features/tlsnotary/constants.ts
Shitikyan and others added 4 commits June 26, 2026 15:09
…ocation

portAllocator.ts kept its own hardcoded PORT_CONFIG (55000-57000) and
proxyManager imported PORT_CONFIG from it, so the env-overridable values in
constants.ts were never used for allocation: the allocator would walk past
the published window (e.g. 55064) and hand nginx an unreachable port,
re-creating the 502 after the first 64 allocations. Point both consumers at
constants.ts (the env-driven copy) and drop the duplicate.

Addresses CodeRabbit/Greptile review on #949.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`cargo install wstcp` pulled the latest release at build time. Pin 0.2.1
and pass --locked so the image always builds the same wstcp and an
upstream release can't change behaviour or break the build.

Addresses CodeRabbit review on #949.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`--locked` forces wstcp 0.2.1's bundled Cargo.lock, whose pinned deps no
longer compile on the current rust:1-slim toolchain (cargo exit 101).
Keep the version pin for reproducibility but let cargo resolve compatible
dependency versions. Validated: the stage builds and yields wstcp 0.2.1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The devnet override replaces the node `ports:` list (`!override`), which
dropped the proxy-range publish from the base compose — so devnet
verifications would still 502. Add a devnet-specific window (55100-55163,
+100 offset from mainnet so the two stacks don't collide on host ports),
with the allocation env and the published range driven by the same values.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant