Skip to content

feat: add ModelRunner as a chat provider - #98

Open
hakankaan wants to merge 2 commits into
LettuceAI:mainfrom
hakankaan:feat/modelrunner-provider
Open

hakankaan wants to merge 2 commits into
LettuceAI:mainfrom
hakankaan:feat/modelrunner-provider

Conversation

@hakankaan

Copy link
Copy Markdown

Upfront disclosure: I work on ModelRunner, so this PR adds my own company as a provider. Close it without ceremony if that isn't something you take from outside.

On your AI usage policy — this was written with Claude Code, and I've read every line of it, traced the code paths it touches, and run your bun run build against it. It's ~75 added lines across 11 files with no new abstraction, so it's small enough to maintain and small enough to review honestly. If any part of it reads as generated-and-not-understood, say so and I'll explain the reasoning or drop it.

What this adds

ModelRunner speaks the OpenAI chat format, so it needs no new adapter. OpenAIAdapter already composes {base}/chat/completions from a base URL ending in /v1, and the trait's default list_models_endpoint / parse_models_list handle the discovery list unchanged.

Provider id modelrunner
Base URL https://queue.modelrunner.run/v1
Adapter openai::OpenAIAdapter
Auth Authorization: Bearer <key>
Key page https://modelrunner.ai/settings/api-keys

Everything else is the wiring one provider needs — the config tuple, the onboarding default base URL in both reducers, the aggregator description bucket, the icon, the parameter-support profile, and the key-location guide. All 11 files are additive; no existing line changes behaviour.

Choices worth flagging

Three of these are deliberate omissions, which are easier to mistake for oversights than the additions are:

  • Not added to supports_model_list in models/verify.rs. That list makes the save path check the model against GET /models and refuse on a miss. ModelRunner's discovery list is public models only, while a key can also call private ones — so adding it there would block a user from saving a model they legitimately own, with "Model not found on provider". The default (skip the check, assume it exists) is correct for this provider, not an oversight.
  • The dispatch arm is written out rather than left to _ => OpenAIAdapter. The fallback would already do the right thing, but your Backend Guidelines ask for explicit behaviour over hidden fallback logic, and lettuce-host sets the precedent of naming it anyway.
  • reasoningSupport: "none". The profile mirrors your Mistral entry: the surface forwards the standard OpenAI sampling parameters (temperature, top_p, max_tokens, frequency_penalty, presence_penalty) to the model and exposes no reasoning-effort toggle of its own. Claiming "effort" or "dynamic" would put a control in the UI the backend can't honour.
  • Guide copy is English only. translateWithLocale() falls back to the English tree for a missing key, so all 21 locales render it correctly today. The other 20 translations should come from speakers of those languages, not from me — I'd rather leave a gap your process can fill than seed 20 machine translations into the locale files.
  • Grouped with the aggregator description (nanogpt / featherless / openrouter) in both ProviderSetup.tsx and ProviderCard.tsx, since it's one key over many models rather than a single lab's own API.
  • Placed after anannas in config.rs, among the hosted providers rather than at the top of the list.

One thing I noticed and did not touch, since your CONTRIBUTING asks to avoid unrelated refactors: the default base URL for a provider is now spelled in three places — config.rs, providerReducer.ts and onboardingReducer.ts — and the two reducers are byte-identical switches. Happy to open a separate PR folding them into one if you'd find that useful.

Testing

  • bun install --frozen-lockfile && bun run build (i.e. tsc && vite build) — exit 0, which is your PR Check "Frontend Check" job. tsc --noEmit covers all nine TypeScript edits, including the TranslationKey union derived from en.ts.
  • The icon import resolves and bundles: dist/assets/modelrunner-YrAT_Kko.png.
  • GET https://queue.modelrunner.run/v1/models answers 401 where every neighbouring GET path answers 404 (/v1/nonsense, /v1/models/extra) — the route exists and authenticates, which is what build_verify_url's {base}/models default branch and list_models_endpoint both target.
  • The Rust side is two lines: a (&str, &str, &str) tuple in the existing vec! and a match arm identical in shape to the lettuce-host one directly below it. I don't have a Rust toolchain on this machine, so cargo check was not run locally — your Linux/macOS/Windows/Android check jobs will cover it.

🤝 Partnership & contact

This PR comes from the ModelRunner team. Two things beyond the code, in case they're useful:

  • Free API credits for this project. We run an Open Source Program that grants credits to maintainers of open-source projects — the tiers are on the page, and this project sits comfortably inside them. That offer stands whether or not you merge this PR; it isn't conditional on the integration.
  • Happy to revise anything here to match your conventions — naming, structure, scope, or splitting it up. Leave a comment and I'll push a change.

Reach us anytime:

🤖 Generated with Claude Code

ModelRunner serves an OpenAI-compatible chat surface, so it needs no new
adapter: OpenAIAdapter already composes `{base}/chat/completions` from a base
URL ending in `/v1`, and the trait's default `list_models_endpoint` /
`parse_models_list` handle its discovery list unchanged. The dispatch arm is
written out rather than left to the `_` fallback, matching how lettuce-host
is registered.

Everything else is the surrounding wiring one provider needs: the config
tuple, the onboarding default base URL in both reducers, the aggregator
description bucket, the icon, the parameter-support profile, and the
key-location guide.

The parameter profile mirrors the Mistral one — the surface forwards the
standard OpenAI sampling parameters to the model and exposes no reasoning
toggle of its own, so reasoningSupport is "none" rather than a claim the
backend cannot honour.

Deliberately NOT added to `supports_model_list` in models/verify.rs: the
discovery list is public models only, while a key can also call private ones,
and `exists: false` there blocks the save outright. The default (skip the
check) is the correct behaviour for this provider rather than an oversight.

Guide copy is English only. translateWithLocale() falls back to the English
tree for a missing key, so every locale renders it; the other 20 should come
from speakers of those languages rather than from me.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Husky110

Husky110 commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Hmm - Looking that this, two things catch my eye:
1.) Please also provide the language strings for the other languages.
2.) I like your offering about free credits, however: Looking at you website I was unable to locate where your company is located. There are e-mailaddresses, but if shit hits the fan, where should I file my complaint about a databreach or a privacy-concern?
I would kindly ask you for this information. :)

@hakankaan

Copy link
Copy Markdown
Author

Both fair.

1. Languages — you're right, English-only made it the odd one out. I'll push the remaining 20 here, keeping "ModelRunner" untranslated.

Two pre-existing ones I hit while matching the register — happy to fix in this PR or leave for a separate one: tr.ts translates the brand, so s1 reads "tüysüz.ai'yi ziyaret edin" and points at a site that doesn't exist; and ru.ts s2 is truncated mid-sentence.

2. Where we are — ModelRunner, Inc., 1111B S Governors Ave # 95898, Dover, DE 19904, United States. The Terms put governing law in California, venue San Francisco.

You were right that it wasn't findable, and that's now fixed: the address is on https://modelrunner.ai/privacy and https://modelrunner.ai/terms. Privacy contact is support@modelrunner.ai.

Each entry follows its own file rather than a generic translation: title and
s1 keep that locale's existing featherless wording with the brand and domain
swapped, and s3/s4 are lifted verbatim from the same block, so register stays
consistent per file (du/Sie, tu/vous, formal/informal) instead of being
re-decided by me.

s2 is the only genuinely new sentence. "Settings" and "API Keys" stay English
in every locale because that settings page is English-only — they are the
words the reader will see on screen.

Three neighbours were not usable as a source and are written out instead:
pt and vi carry the whole guides section in English, and tr's featherless
entry translates the brand ("tüysüz.ai"), which points at a site that does not
exist. Those pre-existing entries are left untouched here.

These are drafts by a non-native speaker; corrections welcome.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@MegalithOfficial

Copy link
Copy Markdown
Contributor

Thanks a lot for taking the time to put this together, and especially for being upfront about your connection to ModelRunner.

For now, we're going to pass on adding ModelRunner as a built-in provider. We're trying to keep the officially supported provider list fairly limited, and we generally only add new providers when there's a clear need from our user base or a strong reason for us to maintain a dedicated integration.

This isn't related to the quality of the PR. You clearly put a lot of effort into keeping the implementation small and consistent with the existing provider setup, and we appreciate that.

Since ModelRunner is OpenAI-compatible, users who want to use it can still add it through LettuceAI's custom provider support without needing a built-in integration.

Thanks again for the contribution and for the thoughtful PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants