Repository navigation
Conversation
…vent loop asyncio.get_event_loop() raises when the thread has no current event loop: in worker threads, after asyncio.run() returned, and on Python 3.14+. Reuse or create the thread's loop instead, raise an actionable UnitxtError when called inside a running loop (IBM#1647), and recreate the LiteLLM semaphore/rate limiter when inference runs in a different loop. Signed-off-by: Waleed Khalid <wak327@gmail.com>
Contributor
Author
|
@elronbandel @yoavkatz @martinscooper could you take a look when you have a moment? This fixes the nested event loop error from #1647, plus the related |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LiteLLMInferenceEngine._inferandWMLInferenceEngineChat(async path) drive their coroutines withasyncio.get_event_loop().run_until_complete(...). That call raisesRuntimeError: There is no current event loop in thread ...whenever the thread has no current loop, which happens:asyncio.run()in the main thread (unitxt itself callsasyncio.runinmetrics.py), on every supported Python version;ThreadPoolExecutor,asyncio.to_thread);get_event_loop()no longer creates a loop implicitly.Inside an already running loop (Jupyter, FastAPI, ...) it fails with the bare
RuntimeError: This event loop is already running, which is what #1647 asked to turn into an actionable message.Changes
get_or_create_event_loop()/run_coroutine_synchronously()ininference.py: reuse the thread's current loop, or create and set one if there is none or it is closed. When called inside a running loop that was not patched bynest_asyncio, close the coroutine and raise aUnitxtErrorthat suggestsasyncio.to_thread(engine.infer, dataset)ornest_asyncio.apply().LiteLLMInferenceEngine._inferandWMLInferenceEngineChat.LiteLLMInferenceEngine'sasyncio.SemaphoreandAsyncTokenBucketlock get bound to the first loop they are contended in, so reusing the engine in another loop failed with... is bound to a different event loop(this also affectednest_asynciousers whose engine had already run outside the notebook loop). They are now recreated when_infer_asyncruns in a different loop. The loop is tracked byid()instead of a reference so the engine stays picklable; a primitive bound to a loop keeps that loop alive, so its id cannot be reused.Before / after
Same
LiteLLMInferenceEngineinstance with_completionstubbed out, Python 3.12:mainasyncio.run(...)RuntimeError: There is no current event loop in thread 'MainThread'.RuntimeError: There is no current event loop in thread 'Thread-1'.RuntimeError: This event loop is already runningUnitxtErrorwith instructionsnest_asyncio.apply()RuntimeError: <Semaphore> is bound to a different event loopFixes #1647
Test plan
tests/library/test_inference_utils.pycovers: plain call, afterasyncio.run, worker thread, loop reuse with a contended semaphore across calls, and the error inside a running loop (and that the coroutine is closed, not left un-awaited).LiteLLMInferenceEngine.pre-commit runon the changed files.