[Model Monitoring] Fix event loop blocking during model endpoint creation (ML-11826) (#9186)

alxtkr77 · Alex Toker · web-flow · commit d33efd570789 · 2026-01-11T12:13:15.000+02:00
## Summary
Fix event loop blocking during model endpoint creation by using
`run_in_threadpool` to wrap synchronous DB operations.

## Changes Made
- Wrap `run_function_with_new_db_session` call in `run_in_threadpool` in
`_create_model_endpoint_limited`
- Add `await` keyword for the async operation
- Add explanatory comment

## Why This Change
When deploying a serving function with many models (e.g., 5000), the
synchronous DB operations for creating model endpoints would block the
async event loop. This prevented the API server from handling other
requests and could cause timeouts.

Using `run_in_threadpool` moves the synchronous DB work to a thread
pool, keeping the event loop responsive while still processing the model
endpoint creation.

## Testing
- Ran `test_app_flow[True-True]` system test successfully
- Deployed serving function with 5000 models - completed in ~7 minutes
with API remaining responsive

## Reference
- Jira: ML-11826

Co-authored-by: Alex Toker &lt;alex_toker@mckinsey.com&gt;
diff --git a/server/py/services/api/crud/model_monitoring/deployment.py b/server/py/services/api/crud/model_monitoring/deployment.py
@@ -1985,8 +1985,11 @@ async def _create_model_endpoint_limited(
         delete_background_task: fastapi.BackgroundTasks,
     ):
         async with semaphore:
-            result = framework.db.session.run_function_with_new_db_session(
-                func=services.api.crud.ModelEndpoints().create_model_endpoints,
+            # Use run_in_threadpool to avoid blocking the event loop
+            # while performing synchronous DB operations
+            result = await run_in_threadpool(
+                framework.db.session.run_function_with_new_db_session,
+                services.api.crud.ModelEndpoints().create_model_endpoints,
                 model_endpoints_instructions=model_endpoints_instructions,
                 project=project,
                 function_name=function_name,
