Problem Statement
We need GuideLLM to integrate with the Eval Hub system.
Proposed Solution
We can start from scratch or embrace / update the prototype adapter module at https://github.com/eval-hub/eval-hub-contrib/tree/main/adapters/guidellm, ensuring a production-ready and supportable implementation.
This additionally needs to be built through the official AIPCC process.
Alternatives Considered
#90 proposed building lm-eval into GuideLLM. At this time, we prefer the Eval Hub integration.
Usage Examples
Additional Context
No response
Problem Statement
We need GuideLLM to integrate with the Eval Hub system.
Proposed Solution
We can start from scratch or embrace / update the prototype adapter module at https://github.com/eval-hub/eval-hub-contrib/tree/main/adapters/guidellm, ensuring a production-ready and supportable implementation.
This additionally needs to be built through the official AIPCC process.
Alternatives Considered
#90 proposed building
lm-evalinto GuideLLM. At this time, we prefer the Eval Hub integration.Usage Examples
Additional Context
No response