Skip to content

EmbeddingFunction lacks async support, blocks event loop in AsyncTable.add() #3268

Description

@JonnyTran

Problem

EmbeddingFunction methods (compute_source_embeddings, compute_query_embeddings) are synchronous-only. When used with AsyncTable, this blocks the event loop in production async frameworks like FastAPI.

Inconsistency between add() and search()

AsyncTable.search() correctly wraps embedding calls in run_in_executor:

# table.py – AsyncTable.search()
async def make_embedding(embedding, query):
    loop = asyncio.get_running_loop()
    return (await loop.run_in_executor(
        None, embedding.function.compute_query_embeddings_with_retry, query,
    ))[0]

But AsyncTable.add() calls embeddings synchronously on the event loop via _append_vector_columns():

# table.py – _append_vector_columns()
col_data = func.compute_source_embeddings_with_retry(batch[conf.source_column])

This blocks all concurrent requests when ingesting data with registered embedding functions.

Broader issue

The EmbeddingFunction ABC has no async interface at all. The TypeScript SDK already supports async computeSourceEmbeddings() / async computeQueryEmbeddings(), but the Python SDK has no equivalent. Most embedding providers (OpenAI, Cohere, Bedrock, etc.) offer async clients that could be leveraged.

Suggested improvements

  1. Short-term: Wrap compute_source_embeddings_with_retry in run_in_executor inside AsyncTable.add(), matching the pattern already used in AsyncTable.search()
  2. Long-term: Add async variants to EmbeddingFunction (async_compute_source_embeddings, async_compute_query_embeddings) so implementations can use native async HTTP clients instead of thread pool workarounds

Environment

  • lancedb 0.30.2
  • Python 3.13
  • FastAPI

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions