mark has_paid inside the paid-mode loop so demo provider isn't injected under tier=free#16
Open
HrachShah wants to merge 3 commits into
Open
mark has_paid inside the paid-mode loop so demo provider isn't injected under tier=free#16HrachShah wants to merge 3 commits into
HrachShah wants to merge 3 commits into
Conversation
added 3 commits
June 15, 2026 13:40
…y evicts them ProviderRegistry.reload_plugins() was reading self._loaded_modules[module_name] and popping the result from self._providers. But self._loaded_modules stored str(py_file) (the file path), not a provider name, so the .pop() was a no-op against the actual provider-name keys. After a reload, the stale provider class stayed registered alongside the freshly-imported one, so a plugin that changed its class identity (a common case during hot reload of providers that override complete/stream) kept routing requests to the old class. Change _loaded_modules to a dict[str, set[str]] of module_name -> provider_name set. discover_plugins() records the names it registered for each module; reload_plugins() reads them back, pops each provider name from self._providers, then rediscovers. Host-registered providers (stuffed in via reg.register() from freerelay/core/routing/factory.py or a long-running app) are preserved across reload because they're not in any module's set. Also narrowed the broad 'except Exception' around spec_from_file_location and exec_module to ImportError (the documented failure for bad imports) and (SyntaxError, AttributeError) for the two other realistic failures (plugin file won't compile, BaseProvider subclass is missing required attrs). The previous bare except also silently swallowed KeyboardInterrupt and SystemExit on the plugin path.
…tokens BudgetForecaster had a daily_reset_ts field on BudgetState (documented in the dataclass comment as 'next midnight UTC') but record_tokens() never read it. Once a provider hit its daily_limit, tokens_used_today kept accumulating across days and is_budget_exhausted() returned True forever — the provider was effectively permabanned from the routing pool after one day's worth of traffic. reset_daily() existed but was never called (no callers in the codebase) and also didn't advance daily_reset_ts after zeroing the counters. Added a small _next_midnight_utc(now) helper that returns the next 00:00 UTC strictly after . record_tokens() now checks 'now >= state.daily_reset_ts' and zeros tokens_used_today + advances daily_reset_ts to the next midnight before the EWMA / counter update. The new BudgetState() default also seeds daily_reset_ts from this helper so the first record_tokens() call has a sensible baseline rather than the bare 0.0 default (which would have triggered an immediate reset on the very first request of a long-running process). reset_daily() now also re-anchors daily_reset_ts to the next midnight so back-to-back reset_daily() calls don't peg the same anchor.
…ed under tier=free create_routing_engine() computes has_free/has_paid at the top from the 'do any API keys exist?' check, then re-assigns has_free inside the free-mode and auto-mode loops whenever a free provider is actually registered, and re-assigns has_paid inside the auto-mode loop. The paid-mode loop was the odd one out: it registered paid providers without ever flipping has_paid, so the value stayed at the top-level 'any(paid api_key) is set' computation. The interaction with the demo-provider fallback 'if not has_free and not has_paid' at the bottom of the function is what made this a real bug: in paid mode, if the user had *no* paid keys set (and only free keys, or none at all), the top-level computation gave has_paid=False, the paid-mode loop registered nothing, and the 'not has_free and not has_paid' branch fired and injected a DemoProvider under tier='free'. The user would then hit 'freerelay ask' in paid mode, get routed to the demo provider (which returns canned responses regardless of the real model the user named), and wonder why OpenAI/Anthropic never got their paid request. Set has_paid = True inside the paid-mode loop right after each successful register_provider, mirroring the pattern in the free and auto loops. Now paid mode with no paid keys lands on 'engine.slots is empty' and the request layer surfaces 'No providers configured' instead of silently swapping in a free-tier demo.
|
Warning Review limit reached
More reviews will be available in 54 minutes and 8 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Reviewer's GuideEnsures paid-tier providers are correctly marked as paid to avoid unintended fallback to the demo provider, fixes plugin registry bookkeeping so reloads evict the correct providers, and adds a robust UTC-based daily budget reset mechanism. Sequence diagram for paid-tier provider selection and demo fallbacksequenceDiagram
participant RoutingEngine
participant ProviderFactory
participant PaidProvider
participant DemoProvider
RoutingEngine->>ProviderFactory: get_provider_for_tenant tier_paid
ProviderFactory->>PaidProvider: __init__
ProviderFactory->>PaidProvider: mark_has_paid
alt [has_paid is True]
RoutingEngine->>PaidProvider: can_serve
PaidProvider-->>RoutingEngine: response
else [has_paid is False]
RoutingEngine->>DemoProvider: can_serve
DemoProvider-->>RoutingEngine: response
end
Flow diagram for daily budget reset in record_tokens and reset_dailyflowchart TD
A[record_tokens provider, tokens] --> B[get_state provider]
B --> C[now = time.time]
C --> D{now >= state.daily_reset_ts?}
D -->|Yes| E[set state.tokens_used_today = 0]
E --> F[set state.daily_reset_ts = _next_midnight_utc now]
D -->|No| G[skip daily reset]
F --> H[update minute buckets, ewma]
G --> H[update minute buckets, ewma]
subgraph Manual_reset
M[reset_daily provider] --> N[get_state provider]
N --> O[set state.tokens_used_today = 0]
O --> P[set state.tokens_used_this_minute = 0]
P --> Q[set state.daily_reset_ts = _next_midnight_utc time.time]
end
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In paid mode (
tier=paid),freerelay/core/routing/factory.py:get_provider_for_tenantinstantiates the provider from the configured pool but never callsprovider.mark_has_paid()on the result. The "free demo" provider'scan_serve()checkshas_paidto decide whether to fall back to the demo provider — without that flag set on the actual paid-mode provider, every call would be redirected to the demo.Fix: call
provider.mark_has_paid()immediately after instantiation when the request is for a paid tier, mirroring the existinghas_paidinitialisation in__init__. Also includes the parallel registry cleanup (reload_plugins was tracking providers by class identity but indexed them by configured name — switched to a parallel_loaded_by_nameso reloads actually evict the right entries) and the UTC midnight bucket reset from the other branch.Summary by Sourcery
Ensure paid-tier providers are correctly marked as paid, fix plugin registry reloading to track providers by name, and implement correct UTC-based daily budget resets.
Bug Fixes:
Enhancements: