Problem

Complementary to #3268. That issue points out AsyncTable.search() at least wraps the embedding call in run_in_executor, while AsyncTable.add() doesn't. However, the search() side still has a separate problem under concurrent load: it uses the default executor.

Evidence

lancedb/table.py — AsyncTable.search():

async def make_embedding(embedding, query):
    if embedding is not None:
        loop = asyncio.get_running_loop()
        # This function is likely to block, since it either calls an expensive
        # function or makes an HTTP request to an embeddings REST API.
        return (
            await loop.run_in_executor(
                None,                                       # <-- default executor
                embedding.function.compute_query_embeddings_with_retry,
                query,
            )
        )[0]

Passing None uses asyncio's default ThreadPoolExecutor, which is:

1. Shared with every other run_in_executor(None, …) / asyncio.to_thread(…) caller in the process — rerankers, pyarrow, fsspec, boto3 elsewhere, user code.
2. Sized at min(32, cpu_count + 4) by default — as small as ~10 on low-vCPU containers.

Under concurrent load (e.g. 100 users against a FastAPI service), a blocking embedding provider (Bedrock, OpenAI, Cohere over sync HTTP) will saturate that shared pool. We measured query throughput drop from ~60 RPS (fake embeddings) to ~10 RPS (real Bedrock invoke_model via sync boto3), with the default pool being the bottleneck.

Suggested improvements

1. Short-term — expose an override. Let AsyncTable.search() (and the future async add() path from EmbeddingFunction lacks async support, blocks event loop in AsyncTable.add() #3268) accept an explicit executor argument, falling back to None for today's behavior. Also expose a module-level setter (e.g. lancedb.set_embedding_executor(pool)) so an app can isolate embedding I/O on a dedicated, sized pool without threading the arg through every call site.
2. Long-term — the async ABC from EmbeddingFunction lacks async support, blocks event loop in AsyncTable.add() #3268 eliminates the executor entirely for providers with real async clients. Pair it with (1) so the degraded sync fallback is still isolated from the shared default pool.

Environment

Same as #3268 — lancedb 0.30.2, Python 3.13, FastAPI.