I was looking for a way to delay module imports until some AsyncIO code has run.

Before anyone calls me mad, the reason I want to do this: is to check for an ipykernel.comm.Comm to have setup. I am waiting on an RPC call that verifies my Jupyter extension has loaded; this solves the problem that Jupyter extensions load asynchronously. When JavaScript comm target it registered it sends a success message back.

def create_future():
    future = asyncio.Future()
    comm = Comm('my_ext')

    @comm.on_msg
    def on_message(msg):
        future.set_result(comm)
        comm.on_msg(None)

    return future

async def wait_for_future():
    return await create_future()

# Something along the lines of this...
loop.run_until_complete(wait_for_future())

In the context of a Jupyter Notebook: I wanted to have the above code block my module import (with a short timeout, obviously) until on_message was invoked. I tried wrapping that code in an asyncio.Future, but ran into problems actually executing it because Jupyter has an event loop already running.

I tried to work around this with threads and loop.call threadsafe; then waiting for the thread to complete. I also tried something similar with a nested event loop (async_nested).

What I think the core problem is

The problem is that I can't wait for the asyncio.Future without waiting on first run_cell_async, pausing all other execution on the event loop. The deadlock can not be released because it depends on the on_message future resolving, but Comm messages are sent by that same event loop.

The bigger problem

I think it's impossible to wait on Comms.

You can't even do this:

def create_comm_future():
    """
    Wait for the extension to send back
    it's heartbeat.
    """
    future = asyncio.Future()
    comm = Comm('my_ext')

    @comm.on_msg
    def on_message(msg):
        future.set_result(comm)
        # Unset the callback
        comm.on_msg(None)

    return future

Then in a Jupyter notebook:

[1]: from my_module import create_comm_future

# This will never complete - because on message will never be called. Infuriating
[2]: await create_comm_future()