catraca

Single-host TCP dispatcher in Rust.

Accepts TCP connections via io_uring (AcceptMulti) and hands the file descriptor off to local workers via SCM_RIGHTS over a Unix Domain Socket. Round-robin, no byte copying, no HTTP parsing.

maracatu.org · Contributing · Code of Conduct · Security

catraca (pt-BR) — turnstile. Controls who passes and where they go.

What it is

• L4 connection dispatcher, not a reverse proxy. Once a connection is handed off, the worker owns the socket end-to-end.
• Single host only: workers must live on the same machine to receive a file descriptor over Unix Domain Sockets.
• Zero-copy handoff: the LB never reads request bytes. The kernel hands the socket to the worker via SCM_RIGHTS.
• Tiny: ~300 lines, two dependencies (libc, io-uring), no async runtime, no allocator pressure on the hot path.

What it isn't

• Not L7. Cannot route by path, header, or hostname.
• Not cross-host. Workers must be on the same machine as catraca.
• No TLS, no HTTP, no health checks, no retries, no metrics. By design.
• Not a drop-in replacement for NGINX / HAProxy / Envoy. Different category.

When to use it

• Workers on the same host that you want to feed via round-robin with minimum latency overhead.
• You're writing the workers yourself and can implement the SCM_RIGHTS recv contract.
• Resource-constrained environments (containers with shared CPU/memory budgets, benchmark / competition setups).

If any of those don't fit, reach for NGINX, HAProxy, Envoy, or Caddy.

Requirements

• Linux 5.19+ (uses IORING_OP_ACCEPT multishot)
• io_uring available to the process (not blocked by seccomp)

Quickstart

Build:

cargo build --release

Run:

PORT=9999 \
UPSTREAMS=/run/api1.sock,/run/api2.sock \
./target/release/catraca

For each upstream /run/api1.sock, catraca connects to a control socket at /run/api1.sock.ctrl (i.e., the upstream path with .ctrl appended) and pushes accepted file descriptors there.

Environment variables

Var	Default	Description
PORT	9999	TCP port to listen on (binds 0.0.0.0)
BACKLOG	4096	listen() backlog
UPSTREAMS	—	Comma-separated UDS paths (without .ctrl suffix)

UPSTREAMS is required. Each entry is a worker's UDS base path; catraca appends .ctrl to derive the control socket address.

Worker contract

Each worker must:

1. Bind a Unix Domain Socket of type SOCK_STREAM at <path>.ctrl and listen() on it.
2. accept() the catraca control connection.
3. On that control connection, repeatedly recvmsg() with a control buffer sized for CMSG_SPACE(sizeof(int)). Each successful recv yields one client TCP file descriptor via SCM_RIGHTS.
4. Take ownership of that fd and serve the request.

A reference C-shaped pseudocode:

char dummy;
struct iovec iov = { &dummy, 1 };
char cmsg_buf[CMSG_SPACE(sizeof(int))];
struct msghdr msg = {0};
msg.msg_iov = &iov;
msg.msg_iovlen = 1;
msg.msg_control = cmsg_buf;
msg.msg_controllen = sizeof(cmsg_buf);

ssize_t n = recvmsg(ctrl_fd, &msg, 0);
struct cmsghdr *c = CMSG_FIRSTHDR(&msg);
int client_fd;
memcpy(&client_fd, CMSG_DATA(c), sizeof(int));
// client_fd is yours

Design notes

• One thread, one io_uring instance. Single SQE for AcceptMulti covers all incoming connections.
• The handoff sendmsg() is a synchronous syscall on the accept thread. This is intentional: the cost of submitting the sendmsg via io_uring is larger than the syscall itself for a 1-byte payload + 1 cmsg.
• Round-robin index is a wrapping usize with no synchronization (single thread).
• SO_REUSEPORT is set on the listener so multiple catraca instances can share the port if you want a NUMA-aware fanout (not currently exposed).

Status

Alpha. API may change. Used in production for a benchmark workload; not yet battle-tested in adversarial conditions. Issues and PRs welcome — see CONTRIBUTING.md.

License

MIT.