Repository navigation
fix(solana): never replay a billed account-rail request on 5xx - #81
Merged
Merged
Conversation
With an API key there is no unsigned probe: the first request carries the key and is the billed one. The raw paid helpers (_request_with_payment_raw, _get_with_payment_raw), the chat helper and the stream probe still replayed it on a 502/503 (the stream probe up to three times on any 5xx), so a call the gateway had already accepted could be charged twice. Affected search, Exa, pm, defi, dex, modal, rpc, price, portrait/realface enroll, chat and streaming on SolanaLLMClient, and the same helpers on the Base LLMClient, sync and async. One shared guard, apikey.raise_for_api_key_5xx, now runs before every one of those retries. On the account rail any 5xx raises APIError at once, keeping its status and Retry-After, saying the request may already have been accepted and billed and that retrying may bill it again. The wallet rail is unchanged: its unsigned probe costs nothing and is still retried.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
On the account rail (client built with a
brk_...API key) there is no unsigned 402 probe: the first request carries the key and is the billed request. Several paid helpers still replayed that first request on a 502/503 (and the streaming probe up to three more times on any 500/502/503/504), so a call the gateway had already accepted and charged could be billed again.Release 1.18.0 already moved Solana image/video/music/speech/sound-effects off this path. Everything else was still on it.
Endpoints affected
SolanaLLMClient/AsyncSolanaLLMClient_request_with_payment_raw:search,exa/exa_search/exa_contents/exa_find_similar/exa_answer,pm_query,dex(POST),modal(+modal_sandbox_*),rpc/rpc_batch,portrait_enroll,realface_enroll_get_with_payment_raw:pm(+pm_*wrappers),defi,dex(GET),price,price_history,list_symbols_request_with_payment(_request_once):chat/chat_completion_stream_with_paymentprobe:chat_completion_stream_request_image_with_paymentprobe already skipped the replay. It now raises the same billing-aware message.Base
LLMClient/AsyncLLMClient(same pattern)_request_with_payment_raw:search, Exa (sync),pm_query,dex(POST),modal,image_edit_get_with_payment_raw:pm,defi,dex(GET)_request_with_payment: chat._stream_with_paymentprobe:chat_completion_streamsearch.py,rpc.py,price.py,surf.py,image.py,music.py,speech.py,voice.py,phone.py,portrait.py,realface.pyandvideo.pyhave no 5xx replay, so they need no change.Fix
There is one shared guard,
apikey.raise_for_api_key_5xx(response, api_key), next toraise_for_api_key_402. It runs right after the first request at every site that auto-retries, and before the retry. On the account rail, any 5xx raisesAPIErrorat once. The error keeps the status code andRetry-After. Its message says the request may already have been accepted and billed, that it was not retried, and that retrying may bill it again. It points to the usage dashboard. On the wallet rail the guard does nothing, so the free unsigned probe retry is unchanged.The status code is unchanged, so
_should_fallbackbehaves as before. A fallback chain still moves to the next model. It never replays the same request.Wallet rail: signed replay
The signed wallet POST is also replayed once on a 502/503, with the same
PAYMENT-SIGNATURE. This PR leaves it as it was. The same signed authorization can settle at most once: the same Solana transaction has the same signature, and the EVM EIP-3009 nonce is single-use. On top of that, the gateway claims the nonce. One adjacent risk is noted here and not changed:If the first signed POST settled before the 5xx, the replay can come back as
PAYMENT_REPLAY._is_safe_resign_errortreats that as a pre-broadcast failure, and the outer loop then re-runs the request with a fresh signature. That path deserves a separate look.Tests
tests/unit/test_solana_media.py: account-railsearch,exa_search,exa,pm(GET),pm_query,chatandchat_completion_stream× {502, 503}, sync and async. Each asserts exactly one request and anAPIErrorwith the billing message. Also covers the image probe message, a single-request success, and the wallet-rail unsigned probe still retried (sync and async).tests/unit/test_apikey.py: the same for the BaseLLMClient/AsyncLLMClient, the Base wallet-rail probe retry, and unit tests for the guard (no-op off-rail and on non-5xx; status andRetry-Afterkept).tests/unit: 1118 passed (1054 before).client.pyandsolana_client.pyreverted tomain, 51 of the new tests fail, which is every no-replay assertion. They pass again with the fix restored.black --check,ruff checkandpython3.9 -m py_compileon the changed files all pass.🤖 Generated with Claude Code