Check if main event loop is running before scheduling

[kigawas]

Fixes #525

After some debugging, I found that main_event_loop is sometimes closed. If it's closed, we should not call_soon_threadsafe

[andersk]

If the user called EventLoop.stop, we should respect their choice. We shouldn’t overreact by silently creating a new event loop in a new thread—that’s going to create confusing bugs with async objects associated with the wrong loop, thread unsafety, and programs failing to stop.

[kigawas]

It's already "failing to stop" now due to a potential deadlock, do you have another solution?

[andersk]

Is there a reason the answer is any more complicated than “don’t call EventLoop.stop if you want it to keep running”?

[kigawas]

In my case, EventLoop.stop was not called explicitly, the event loop just stopped somewhere

[TEST1] main event loop <_UnixSelectorEventLoop running=True closed=False debug=False>
in main event loop, running? True
exc_info (None, None, None) awaitable <coroutine object convert_exception_to_response.<locals>.inner at 0x123d415d0>
result <HttpResponse status_code=401, "application/json">
running_in_main_event_loop True
PASSED
[TEST2] main event loop <_UnixSelectorEventLoop running=True closed=False debug=False>
in main event loop, running? True
exc_info (None, None, None) awaitable <coroutine object convert_exception_to_response.<locals>.inner at 0x122c91f30>
result <HttpResponse status_code=200, "application/json; charset=utf-8">
running_in_main_event_loop True
main event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
in main event loop, running? False
[...stuck...]

[andersk]

Event loops don’t arbitrarily stop themselves. You can look at the code here:

https://github.com/python/cpython/blob/v3.13.7/Lib/asyncio/base_events.py

The only way for BaseEventLoop.is_running() to change from True to False is if BaseEventLoop._thread_id is set to None, which only happens in BaseEventLoop._run_forever_cleanup, which is only called by BaseEventLoop.run_forever if BaseEventLoop._stopping became True, which only happens if BaseEventLoop.stop() was called. (Perhaps by another library function like BaseEventLoop.run_until_complete → _run_until_complete_cb → BaseEventLoop.stop, although it would be weird to be calling both run_forever and run_until_complete on the same event loop.)

Without a reproducible test case, there’s not really anything more I can say.

[kigawas]

<_UnixSelectorEventLoop running=False closed=False debug=False>

I misinterpreted.
It's actually not closed but neither running.

[kigawas]

I see, if loop is not running, loop.create_task will never create a task and it'll just seem getting stuck

[kigawas]

The problem is still there and can be fixed via this one-line change

[carltongibson]

@kigawas Can you add a minimal test case that fails without this change to demonstrate the issue?

(It's not clear why you're seeing what you are.)

[kigawas]

@carltongibson Please see this repo

https://github.com/kigawas/asgiref-stale-loop-repro

[kigawas]

Test added

[andersk]

Your code mutates the private variable SyncToAsync.threadlocal.main_event_loop. You claim that this is “matching what asgiref's own thread_handler does after a real sync_to_async dispatch”—however, the real SyncToAsync.thread_handler can only be called with a running loop (because it was called via loop = asyncio.get_running_loop(); …; loop.run_in_executor(…)).

Is there some misbehavior you can demonstrate using only the public interface?

[kigawas]

I encountered this problem when running pytest-asyncio in a Django project which has some sync_to_async and async_to_sync mixture, not tinkering its private API.
It's a real problem.

[andersk]

Okay, then, can you show a reproduction recipe using the public API?

[carltongibson]

@kigawas your last comment goes beyond the code of conduct. Please be civil. Or you will be bared from the organisation.

This is your final warning.

[carltongibson]

@andersk just to underline, your contributions here are always of high quality and are greatly appreciated. Sorry you have to take negative comments 🎁

[carltongibson]

@kigawas I managed to dig in and make progress on this. It looks like a regression in #507. Will likely pursue in #558, which has a clearer reproduction case.