Skip to content

[Feature] 大模型推理适配非LLM模型及其兼容api接口 #24930

Description

@DSYZayn

What would you like to be added:
当前仅能支持LLM类型,缺少Embedding和Reranker等类型的模型兼容性。

Image

此外,许多大模型部署依赖特定的后端参数,可参考在gpustack的模型库中,点击每个模型卡片后相应的参数。

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions