feat: make server deps optional behind server feature flag#400
Open
emanueleDiVizio wants to merge 1 commit into
Open
feat: make server deps optional behind server feature flag#400emanueleDiVizio wants to merge 1 commit into
server feature flag#400emanueleDiVizio wants to merge 1 commit into
Conversation
… flag This allows candle-vllm to be used as a library for embedding/inference without pulling in the HTTP server stack (axum, hyper, tower-http, utoipa, rustchatui). The `server` feature is enabled by default for backwards compatibility.
emanueleDiVizio
force-pushed
the
feat/optional-server-deps
branch
from
384950e to
c3ef853
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
axum,hyper,tower-http,utoipa, andrustchatuioptional dependencies behind a newserverfeature flagserverfeature is enabled by default, so this is fully backwards-compatibleChatResponder,IntoResponseimpl,Streamimpl forStreamer, etc.) behind#[cfg(feature = "server")]Motivation
This allows candle-vllm to be used as a library dependency for embedding/inference workloads without pulling in the HTTP server stack. Useful when candle-vllm is embedded in another application that provides its own API layer.
Test plan
cargo build(default features, includes server) — should compile as beforecargo build --no-default-features— should compile without server depscargo build --no-default-features --features cuda— CUDA without server