Cache pipeline schemas and plugin list responses#645
Conversation
The /api/v1/pipelines/schemas and /api/v1/plugins endpoints were recomputing everything on every request (schema generation, distribution scanning, update check subprocesses). Since these don't change until a plugin install/uninstall (which restarts the server), cache the responses after first computation. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafal Leszko <[email protected]>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughAdds two module-level caches in Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/scope/server/app.py (1)
2203-2227:⚠️ Potential issue | 🟠 MajorSerialize the cold-cache fill for
/api/v1/plugins.The early
Nonecheck happens beforeawait plugin_manager.list_plugins_async(), so concurrent first requests can all miss and each run the expensive plugin scan/update checks. Guard the miss path with anasyncio.Lockor memoized task.🔒 Suggested fix
+_plugins_list_cache_lock = asyncio.Lock() + `@app.get`("/api/v1/plugins") `@cloud_proxy`() async def list_plugins( @@ global _plugins_list_cache if _plugins_list_cache is not None: return _plugins_list_cache - - try: - plugin_manager = get_plugin_manager() - plugins_data = await plugin_manager.list_plugins_async() + async with _plugins_list_cache_lock: + if _plugins_list_cache is not None: + return _plugins_list_cache + try: + plugin_manager = get_plugin_manager() + plugins_data = await plugin_manager.list_plugins_async() - plugins = [_convert_plugin_dict_to_info(p) for p in plugins_data] + plugins = [_convert_plugin_dict_to_info(p) for p in plugins_data] - failed = [ - FailedPluginInfoSchema( - package_name=f.package_name, - entry_point_name=f.entry_point_name, - error_type=f.error_type, - error_message=f.error_message, + failed = [ + FailedPluginInfoSchema( + package_name=f.package_name, + entry_point_name=f.entry_point_name, + error_type=f.error_type, + error_message=f.error_message, + ) + for f in plugin_manager.get_failed_plugins() + ] + + response = PluginListResponse( + plugins=plugins, total=len(plugins), failed_plugins=failed ) - for f in plugin_manager.get_failed_plugins() - ] - - response = PluginListResponse( - plugins=plugins, total=len(plugins), failed_plugins=failed - ) - _plugins_list_cache = response - return response - except Exception as e: - logger.error(f"Error listing plugins: {e}") - raise HTTPException(status_code=500, detail=str(e)) from e + _plugins_list_cache = response + return response + except Exception as e: + logger.error(f"Error listing plugins: {e}") + raise HTTPException(status_code=500, detail=str(e)) from e🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/scope/server/app.py` around lines 2203 - 2227, The code checks _plugins_list_cache before awaiting plugin_manager.list_plugins_async, so concurrent cold requests cause multiple expensive scans; fix by serializing the cold-cache fill using an asyncio.Lock (e.g., a module-level _plugins_list_lock) or by memoizing the in-progress task: acquire the lock (or check/create the shared task) before calling get_plugin_manager() / plugin_manager.list_plugins_async(), re-check _plugins_list_cache after acquiring to avoid race, populate _plugins_list_cache with the created PluginListResponse (plugins, total, failed_plugins) and release the lock (or let the awaited task set the cache) so only one scan runs for concurrent first requests while others await the result.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/scope/server/app.py`:
- Around line 110-113: The pipeline schema and plugins-list caches
(_pipeline_schemas_cache and _plugins_list_cache) are never invalidated, so
reload_plugin and plugin install/uninstall paths continue to serve stale data;
update the plugin install, uninstall and reload handlers (where reload_plugin is
invoked) to call the existing helper _clear_catalog_caches() immediately after a
successful plugin change and before returning the success response so the next
request rebuilds fresh data for /api/v1/plugins and /api/v1/pipelines/schemas.
---
Outside diff comments:
In `@src/scope/server/app.py`:
- Around line 2203-2227: The code checks _plugins_list_cache before awaiting
plugin_manager.list_plugins_async, so concurrent cold requests cause multiple
expensive scans; fix by serializing the cold-cache fill using an asyncio.Lock
(e.g., a module-level _plugins_list_lock) or by memoizing the in-progress task:
acquire the lock (or check/create the shared task) before calling
get_plugin_manager() / plugin_manager.list_plugins_async(), re-check
_plugins_list_cache after acquiring to avoid race, populate _plugins_list_cache
with the created PluginListResponse (plugins, total, failed_plugins) and release
the lock (or let the awaited task set the cache) so only one scan runs for
concurrent first requests while others await the result.
| # Cached responses for pipeline schemas and plugin list. | ||
| # Server restarts after plugin install/uninstall, so these are naturally reset. | ||
| _pipeline_schemas_cache: PipelineSchemasResponse | None = None | ||
| _plugins_list_cache: object | None = None |
There was a problem hiding this comment.
Invalidate these caches when plugins change.
These responses are now cached forever, but reload_plugin can change plugin metadata and add/remove pipelines without restarting the process. After the first hit, /api/v1/plugins and /api/v1/pipelines/schemas will keep serving the pre-reload snapshot.
🧹 Suggested fix
_pipeline_schemas_cache: PipelineSchemasResponse | None = None
_plugins_list_cache: object | None = None
+
+def _clear_catalog_caches() -> None:
+ global _pipeline_schemas_cache, _plugins_list_cache
+ _pipeline_schemas_cache = None
+ _plugins_list_cache = NoneThen call _clear_catalog_caches() after successful plugin install/uninstall/reload paths, before returning the success response.
Also applies to: 655-678, 2203-2227
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/scope/server/app.py` around lines 110 - 113, The pipeline schema and
plugins-list caches (_pipeline_schemas_cache and _plugins_list_cache) are never
invalidated, so reload_plugin and plugin install/uninstall paths continue to
serve stale data; update the plugin install, uninstall and reload handlers
(where reload_plugin is invoked) to call the existing helper
_clear_catalog_caches() immediately after a successful plugin change and before
returning the success response so the next request rebuilds fresh data for
/api/v1/plugins and /api/v1/pipelines/schemas.
🚀 fal.ai Preview Deployment
TestingConnect to this preview deployment by running this on your branch: 🧪 E2E tests will run automatically against this deployment. |
✅ E2E Tests passed
Test ArtifactsCheck the workflow run for screenshots. |
The cached plugin list response from a previous test was being returned instead of the mock, causing test_returns_plugin_list to fail. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafal Leszko <[email protected]>
The /api/v1/pipelines/schemas and /api/v1/plugins endpoints were recomputing everything on every request (schema generation, distribution scanning, update check subprocesses). Since these don't change until a plugin install/uninstall (which restarts the server), cache the responses after first computation.
Summary by CodeRabbit
Refactor
Tests