Skip to content

Stanza service: CPU-only torch, pinned dependencies - #786

Merged
mircealungu merged 1 commit into
masterfrom
stanza-cpu-torch
Sep 28, 2026
Merged

mircealungu merged 1 commit into
masterfrom
stanza-cpu-torch

Conversation

@mircealungu

Copy link
Copy Markdown
Member

Tonight's Stanza rebuild installed PyPI's default torch, which brings about 2.5 GB of CUDA libraries (nvidia-cudnn, cublas, nccl, cusparse, ...). The server has no GPU, so they never load. They only made the image's main layer 3.09 GB and took up most of a 3-minute build.

  • torch from PyTorch's CPU index: torch==2.14.0+cpu, the same version as tonight's build without CUDA.
  • pinned to what that build installed and ran with: flask 3.1.3, gunicorn 26.2.0, psutil 7.2.2, numpy 2.5.3, protobuf 7.36.2. Before, each rebuild could silently pick up new major versions.

Checked without building (no Docker here): pip install --dry-run --platform manylinux_2_28_x86_64 --python-version 3.12 --only-binary=:all: resolves to 27 packages instead of 46, with no nvidia, CUDA or triton packages.

Runtime: tokenization speed should be unchanged, since both builds run the same CPU code. Container start and image builds get faster; memory may drop slightly.

Takes effect on the next Stanza rebuild. Do it at a quiet moment, because a Stanza restart briefly breaks uncached tokenization. Recreate stanza_crawl when no crawler_* container is running:

ssh zeeguu@$ZORGIP 'cd ~/ops/running/api && git -C ~/src/api pull --ff-only && docker compose build stanza && docker compose up -d --no-deps stanza'
ssh zeeguu@$ZORGIP 'docker ps --filter name=crawler_ --format "{{.Names}}"'   # wait until empty
ssh zeeguu@$ZORGIP 'cd ~/ops/running/api && docker compose up -d --no-deps stanza_crawl'

After the restart, /health should report pipelines_loaded: 30.

🤖 Generated with Claude Code

The Stanza image installed PyPI's default torch, which brings ~2.5GB of
CUDA libraries (nvidia-cudnn, cublas, nccl, ...). The server has no GPU,
so they never load; they only made the image's main layer 3.09GB and
most of a 3-minute rebuild. Install torch from PyTorch's CPU index.

Also pin what the Sep 2026 rebuild installed (torch 2.14, numpy 2.5,
protobuf 7, gunicorn 26, flask 3.1, psutil 7.2): unpinned, each rebuild
could silently pick up new majors. Resolved for manylinux x86_64 /
cp312 with pip --dry-run: 27 packages instead of 46, none CUDA.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

ArchLens - No architecturally relevant changes to the existing views

@mircealungu
mircealungu merged commit d33c071 into master Sep 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant