[BE] feat: add Requesty as an LLM provider#7445
Conversation
Add Requesty (OpenAI-compatible LLM router, https://router.requesty.ai/v1) as a first-class provider, mirroring the existing OpenRouter provider. Backend (apps/opik-backend): - New infrastructure/llm/requesty package: RequestyModule, RequestyLlmServiceProvider, RequestyModelName, RequestyErrorMessage - Add REQUESTY to the LlmProvider enum - Register RequestyModule in OpikApplication Guice modules - Route models via the "requesty/" prefix in LlmProviderFactoryImpl - Add requestyUrl to LlmProviderClientConfig and config(-test).yml (default https://router.requesty.ai/v1, env LLM_PROVIDER_REQUESTY_URL) - Add requestyProviderEnabled toggle to ServiceTogglesConfig and config(-test).yml - Include RequestyErrorMessage in the LangChain error-mapping chain Frontend/docs: - Add Requesty.py integration script - Add Requesty integration docs pages (docs and docs-v2) and nav entries - Add Requesty card to the integrations overview pages Requesty is OpenAI-compatible and uses the same provider/model naming as OpenRouter, so it reuses the generic OpenAI-compatible client path with a fixed base URL. Verified the base URL, Bearer auth and provider/model naming against the live endpoint. Signed-off-by: Thibault Jaigu <thibault.jaigu@gmail.com>
github-actions
Bot
added
🟠 size/L
documentation
![baz-reviewer[bot]](https://avatars.githubusercontent.com/in/933528?s=60&v=4){kind=small}
|
Thanks for the contribution! When you have a chance, could you take a look at the automated review comments from Baz? For each comment, please either address it with a code change or reply with a brief explanation if you believe the current implementation is the right approach. Thanks! |
- Add REQUESTY branch to StructuredOutputStrategy so Requesty models with structured output support use the tool-calling strategy instead of falling back to the instruction strategy - Enable tool calling for REQUESTY in AgenticScoringService - Guard null/blank fields in RequestyErrorMessage.toErrorMessage since Bean Validation annotations are not enforced on the Jackson deserialization path (prevents NPE from autounboxing a null code)
|
Thanks for the ping! I've gone through all the Baz review comments: Fixed in 870e56b:
Replied with rationale (3 low-severity dedup findings):
The backend compiles cleanly with the changes. Let me know if anything else is needed! |
| case OPEN_AI, ANTHROPIC, GEMINI, OPEN_ROUTER, REQUESTY, VERTEX_AI, BEDROCK -> true; | ||
| case OLLAMA, CUSTOM_LLM, OPIK_FREE -> false; |
There was a problem hiding this comment.
Requesty capability regressions
REQUESTY is now treated as tool-calling capable in the capability matrix and StructuredOutputStrategy.supportsStructuredOutput(), but this diff doesn't add a regression test for the new switch arms, so they stay uncovered. Should we add a small parameterized test that checks REQUESTY against the existing OpenRouter/OpenAI-style providers, including an unsupported Requesty model returning false for structured output?
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
apps/opik-backend/src/main/java/com/comet/opik/api/resources/v1/events/AgenticScoringService.java
around lines 418-423, the supportsToolCalling switch now returns true for REQUESTY (line
421-422). Add a small parameterized regression test in the existing test suite for
AgenticScoringService (or the closest matching test class) that asserts REQUESTY is
treated as tool-calling capable like the existing OpenAI/OpenRouter-style providers, and
also includes an assertion that for an unsupported Requesty model name structured output
is false. In
apps/opik-backend/src/main/java/com/comet/opik/domain/llm/structuredoutput/StructuredOutputStrategy.java
around lines 45-46, ensure the test covers REQUESTY’s structured-output capability
behavior (including both supported and unsupported model names, with unsupported
returning false) so the new switch arms remain covered and won’t regress.
| case REQUESTY -> RequestyModelName.byValue(modelName) | ||
| .map(StructuredOutputSupported::isStructuredOutputSupported).orElse(false); |
There was a problem hiding this comment.
Prefixed Requesty models mis-detect structured output
StructuredOutputStrategy.getStrategy() passes the raw requesty/... model name into RequestyModelName.byValue(), so prefixed Requesty models miss the enum lookup and isStructuredOutputSupported falls back to InstructionStrategy — should we normalize the model name before matching?
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents
Before applying, verify this suggestion against the current code. In
apps/opik-backend/src/main/java/com/comet/opik/domain/llm/structuredoutput/StructuredOutputStrategy.java
around lines 45-46 inside StructuredOutputStrategy.getStrategy(), fix the REQUESTY
branch which currently calls RequestyModelName.byValue(modelName) with the unnormalized
modelName that still includes the “requesty/” prefix. Refactor so that before
looking up the RequestyModelName, you strip the leading “requesty/” (or otherwise
normalize to the format byValue expects), or switch to the appropriate RequestyModelName
resolver that accepts the prefixed/qualified form. Add a regression test ensuring that
when the upstream model string is “requesty/<provider/model>”,
isStructuredOutputSupported returns true rather than silently falling back to
InstructionStrategy.
andrescrz
left a comment
andrescrz
left a comment
There was a problem hiding this comment.
Opik reviewer (mined from your team's review history)
9 findings — 1 high · 5 medium · 3 low. Suppressed by team conventions: see suppressed.md.
React 👍/👎 on each comment — your feedback helps tune what it flags.
| return LlmProvider.OPEN_ROUTER; | ||
| } | ||
|
|
||
| if (model.startsWith("requesty/") |
There was a problem hiding this comment.
[high] Requesty model names collide with OpenRouter; OpenRouter is matched first so most Requesty models never route to REQUESTY
The new REQUESTY resolution branch is placed AFTER the OpenRouter check (lines 135-138), and the two providers share the identical bare provider/model namespace. Verified: 21 of 24 RequestyModelName values are byte-for-byte identical to entries already in OpenRouterModelName (openai/gpt-4o, openai/gpt-4o-mini, openai/gpt-4.1, openai/o1, openai/o3, anthropic/claude-3.5-sonnet, anthropic/claude-3.7-sonnet, anthropic/claude-opus-4, anthropic/claude-sonnet-4, deepseek/deepseek-chat, google/gemini-2.5-flash, google/gemini-2.5-pro, meta-llama/llama-3.1-8b-instruct, mistralai/mistral-large, ...). Because isModelBelongToProvider(model, OpenRouterModelName.class, ...) runs first, every overlapping bare model resolves to LlmProvider.OPEN_ROUTER and never reaches the Requesty branch — making the isModelBelongToProvider(..., RequestyModelName.class, ...) half of the new condition dead for 21/24 models. Only 3 Requesty-unique bare values (anthropic/claude-sonnet-4-5, deepseek/deepseek-reasoner, google/gemini-2.0-flash) and the requesty/ prefix reliably reach REQUESTY.
Downstream impact (getProviderApiKey filters keys by strict provider equality): a user who configured only a Requesty key and selected e.g. openai/gpt-4o gets BadRequestException: API key not configured for LLM. provider='openrouter'; a user with both keys silently hits OpenRouter (wrong endpoint, wrong billing). StructuredOutputStrategy.getStrategy is likewise routed to the OPEN_ROUTER case for these models, bypassing the curated RequestyModelName structured-output set. Confirmed by existing LlmProviderFactoryTest (lines 139-140), which asserts every OpenRouterModelName value resolves to OPEN_ROUTER.
💡 Because the two OpenAI-compatible routers share an identical model namespace, bare-model-name matching cannot disambiguate them. Route Requesty solely by an explicit discriminator (e.g. the requesty/ prefix — and strip it before use, see the related prefix finding) and drop the ambiguous isModelBelongToProvider(..., RequestyModelName.class, ...) fallback, which can never win against OpenRouter for overlapping names. Add a Requesty routing test mirroring the existing OpenRouter one.
rule: backend-correctness-wire-every-new-field-enum-value-or-constructor-parameter-thr
| return LlmProvider.OPEN_ROUTER; | ||
| } | ||
|
|
||
| if (model.startsWith("requesty/") |
There was a problem hiding this comment.
[medium] requesty/ prefix is never stripped, so it leaks downstream and breaks structured-output detection
Since the enum branch is shadowed by OpenRouter (see previous finding), the only reliable way to reach REQUESTY is the requesty/ prefix. But that prefix is never removed anywhere in the pipeline (no stripping here, and getResolvedModelInfo at line 255 returns the model verbatim; there is no prefix handling in RequestyLlmServiceProvider). Consequences: (1) StructuredOutputStrategy.getStrategy(REQUESTY, model) calls RequestyModelName.byValue(model) with the still-prefixed string (requesty/openai/gpt-4o), which never matches a value (values are bare, e.g. openai/gpt-4o), so byValue returns empty and structured output is always treated as unsupported for Requesty — it silently falls back to InstructionStrategy. (2) The prefixed model name is forwarded to the OpenAI-compatible Requesty client, but per RequestyModelName's own doc the router expects the bare provider/model form, so the outbound model id is likely wrong. Contrast with how CustomLlmModelNameChecker.substring(...) strips its custom-llm/ prefix.
💡 Strip the requesty/ prefix once at resolution (or before the byValue lookup and before the client call) so the bare provider/model value is what flows to RequestyModelName.byValue(), the structured-output check, getResolvedModelInfo, and the Requesty API.
general finding
| return LlmProvider.OPEN_ROUTER; | ||
| } | ||
|
|
||
| if (model.startsWith("requesty/") |
There was a problem hiding this comment.
[medium] New REQUESTY resolution branch has no test coverage
LlmProviderFactoryTest parametrizes every OpenRouterModelName value (lines 139-140) and has a dedicated openrouter/ route-slug test, but no equivalent coverage exists for Requesty (grep for REQUESTY in src/test finds only config-test.yml). A Requesty test would have immediately surfaced the collision in finding #1 — asserting a bare Requesty model resolves to REQUESTY fails today because it resolves to OPEN_ROUTER.
💡 Add parametrized coverage for RequestyModelName values and for the requesty/ route-slug path (mirroring the OpenRouter tests), asserting the resolved LlmProvider and the downstream model string. These tests should encode the intended routing precedence so the namespace collision is locked down.
rule: backend-testing-cover-every-branch-of-new-changed-backend-behavior-with-dedi
| return LlmProvider.OPEN_ROUTER; | ||
| } | ||
|
|
||
| if (model.startsWith("requesty/") |
There was a problem hiding this comment.
[medium] requesty/ prefix branch is unreachable for enum models and, when taken, sends an invalid model string to Requesty
The RequestyModelName enum values are bare (e.g. "openai/gpt-4o"), so model.startsWith("requesty/") never matches any curated Requesty model; real Requesty models only match via the enum branch. The prefix branch therefore only fires when a caller manually passes 'requesty/...'. But the model string is never stripped of that prefix before use: OpenAIClientGenerator (lines 113, 168) sets .modelName(modelParameters.name()) verbatim, so a 'requesty/openai/gpt-4o' selection is sent to Requesty's OpenAI-compatible API as 'requesty/openai/gpt-4o', which Requesty does not recognize. This is symmetric with the pre-existing openrouter/ branch (also never stripped), but it means the new prefix routing cannot produce a working call.
💡 If a requesty/ prefix is intended as the disambiguator, strip it before the model reaches OpenAIClientGenerator.newOpenAiClient/newOpenAiChatLanguageModel (as would be needed for the fix to finding 1), and align the enum values so curated models actually carry the prefix; otherwise drop the dead startsWith("requesty/") branch.
general finding
| return errorMessage; | ||
| } | ||
|
|
||
| errorMessage = getErrorMessage(throwable, log, RequestyErrorMessage.class); |
There was a problem hiding this comment.
[medium] New RequestyErrorMessage branch is unreachable — OpenRouterErrorMessage (identical schema, tried first) shadows it
The Requesty branch is inserted AFTER the OpenRouter attempt (line 247 vs 253), but OpenRouterErrorMessage.OpenRouterError and RequestyErrorMessage.RequestyError deserialize the exact same shape — {"error":{"message":String,"code":Integer}}, both @JsonIgnoreProperties(ignoreUnknown=true). Bean-Validation annotations (@NotBlank/@NotNull) do NOT run during Jackson deserialization (JsonUtils.MAPPER registers only JavaTimeModule), so both records bind the identical payload. parseError returns empty only when error()==null or on a parse exception — identical for both types — so any Requesty payload OpenRouter could parse is consumed at line 249 and line 253 is never reached. The new branch is dead code.
Worse: for a Requesty error with null/absent code, OpenRouterErrorMessage.toErrorMessage() runs new ErrorMessage(error.code(), error.message()), unboxing the null Integer → NullPointerException (parseError only catches UncheckedIOException, so the NPE propagates out of getErrorObject). RequestyErrorMessage's explicit null-code guard (returns 500) and blank-message guard — the exact cases this change was written to handle — never execute. Runtime path confirmed: RequestyLlmServiceProvider.getService returns LlmProviderOpenAi, whose getError calls LlmProviderLangChainMapper.INSTANCE.getErrorObject.
💡 Give Requesty a discriminator so its branch is reachable (e.g. only attempt RequestyErrorMessage when the request was routed to REQUESTY, or reorder so the more-defensive parser runs first), OR harden OpenRouterErrorMessage to tolerate a null code. As written the new class adds no behavioral effect and leaves the null-code NPE it was meant to fix in place.
rule: backend-api-contract-preserve-api-contract-and-query-correctness-ship-fully-wired
| */ | ||
| @Slf4j | ||
| @RequiredArgsConstructor | ||
| public enum RequestyModelName implements StructuredOutputSupported { |
There was a problem hiding this comment.
[medium] New Requesty provider enum has no routing/structured-output test coverage
This enum adds observable behavior (provider routing via LlmProviderFactoryImpl.getLlmProviderFromEnums and structured-output selection via StructuredOutputStrategy.getStrategy for LlmProvider.REQUESTY), but no test exercises it. LlmProviderFactoryTest.testGetService's @MethodSource enumerates every sibling provider enum (OpenaiModelName, AnthropicModelName, GeminiModelName, OpenRouterModelName, VertexAIModelName) yet omits RequestyModelName, so neither byValue matching, the REQUESTY routing branch, nor the isStructuredOutputSupported true/false split is verified. A drift between the enum values and the router (e.g. the 'requesty/' prefix handling or a mistyped model string) would ship undetected.
💡 Add RequestyModelName to the testGetService MethodSource (mapping each value to LlmProvider.REQUESTY / its expected service class), and add a case covering both isStructuredOutputSupported outcomes (a supported model like openai/gpt-4o and an unsupported one like anthropic/claude-3.5-haiku or deepseek/deepseek-reasoner).
rule: backend-testing-cover-every-branch-of-new-changed-backend-behavior-with-dedi
| return errorMessage; | ||
| } | ||
|
|
||
| errorMessage = getErrorMessage(throwable, log, RequestyErrorMessage.class); |
There was a problem hiding this comment.
[low] New Requesty error-parsing branch has no test coverage
getErrorObject() and the new Requesty branch have no tests in apps/opik-backend/src/test (no reference to getErrorObject, RequestyErrorMessage, or OpenRouterErrorMessage). A dedicated test — parsing a representative Requesty error payload (including the null/absent-code case) and asserting the resulting ErrorMessage code/message — would immediately expose the unreachable-branch/NPE issue above and lock in the intended precedence.
💡 Add a parameterized test over provider error payloads (OpenRouter, Requesty with numeric code, Requesty with null code, OpenAi) asserting the mapped ErrorMessage status and message, so branch precedence and the null-code path are verified.
rule: backend-testing-cover-every-branch-of-new-changed-backend-behavior-with-dedi
| return new ErrorMessage(Response.Status.INTERNAL_SERVER_ERROR.getStatusCode(), error.message()); | ||
| } | ||
|
|
||
| return new ErrorMessage(error.code(), error.message()); |
There was a problem hiding this comment.
[low] Requesty error code used raw as HTTP status without range mapping
error.code() is passed straight through as the ErrorMessage status code. Downstream, ChatCompletionService.failHandlingLLMProviderError() computes familyOf(code) and then constructs either ClientErrorException(msg, code) (requires a 4xx) or ServerErrorException(msg, code) (requires a 5xx). If Requesty ever returns a code that is not a valid HTTP status (e.g. 0, a 2xx/3xx, or an OpenAI-style non-HTTP error code), familyOf() returns a non-CLIENT family and ServerErrorException(msg, code) throws IllegalArgumentException, masking the real provider error with an unrelated exception. Sibling wrappers OpenAiErrorMessage and CustomLlmErrorMessage deliberately map the provider code through OpenAiCompatStatusCodes (and clamp unknowns to 400) precisely for this reason; this class (like OpenRouterErrorMessage) uses the code raw. Reachable only if Requesty emits a non-HTTP integer code, so severity is low / defensive-hardening.
💡 Validate/clamp the code to a legal HTTP status before constructing ErrorMessage (e.g. fall back to 500 or 400 when the code is outside 400-599), or route it through OpenAiCompatStatusCodes like the OpenAI/CustomLlm wrappers do.
rule: backend-error-handling-map-errors-to-the-correct-http-status-code-never-uniformly-m
| public record RequestyError(String message, Integer code) { | ||
| } | ||
|
|
||
| public ErrorMessage toErrorMessage() { |
There was a problem hiding this comment.
[low] New toErrorMessage() branches have no dedicated test
toErrorMessage() introduces three distinct branches (blank message -> null, non-null message but null code -> 500, message + code -> new ErrorMessage(code, message)) with an explicit design choice to guard fields manually because Bean Validation does not run on deserialized provider responses. The sibling OpenAiErrorMessage has a dedicated OpenAiErrorMessageTest, but no equivalent RequestyErrorMessageTest was added, so none of these branches are covered.
💡 Add a RequestyErrorMessageTest (or extend the LLM error-parsing tests) covering: blank/null message -> Optional.empty, null code -> 500, and valid code -> that code, ideally as a @ParameterizedTest.
rule: backend-testing-cover-every-branch-of-new-changed-backend-behavior-with-dedi
2 participants
Summary
Adds Requesty — an OpenAI-compatible LLM router — as an LLM provider, mirroring the existing OpenRouter provider. Requesty exposes 400+ models through one endpoint, addressed as
provider/model.Changes
Backend (
apps/opik-backend, mirroringinfrastructure/llm/openrouter/)infrastructure/llm/requesty/:RequestyModule.java,RequestyLlmServiceProvider.java,RequestyModelName.java,RequestyErrorMessage.javaLlmProviderenum:REQUESTY("requesty")OpikApplication.java: registeredRequestyModulein the Guice module listLlmProviderFactoryImpl.java:requesty/-prefix routing (prefix-only, so it does not intercept OpenRouter/OpenAI provider/model strings)LlmProviderClientConfig.java,ServiceTogglesConfig.java,config.yml,config-test.yml:requestyUrl(defaulthttps://router.requesty.ai/v1, envLLM_PROVIDER_REQUESTY_URL) + enable toggleLlmProviderLangChainMapper.java:RequestyErrorMessageadded to the error-mapping chainFrontend / docs
integration-scripts/Requesty.pymirroringOpenRouter.pydocs+docs-v2) + nav entries + overview cardsTesting
https://router.requesty.ai/v1(Bearer auth,provider/modelnaming,HTTP-Referer/X-Titleheaders) returned successfully — exercising exactly the OpenAI-compatible pathRequestyModulewires. Please runmvn testin CI to confirm the module build.Note: the frontend integrations-grid card (
constants/integrations.ts) is intentionally not wired because it needs a Requesty logo PNG asset I don't have — happy to add it if you can point me at the brand asset. Everything else (backend provider, routing, config, docs) is complete.I work at Requesty. This mirrors the existing OpenRouter provider as closely as possible. Happy to adjust or close it if it's not a fit.