Skip to content

Cache pipeline schemas and plugin list responses#645

Merged
leszko merged 2 commits intomainfrom
rafal/speed-up-loading
Mar 10, 2026
Merged

Cache pipeline schemas and plugin list responses#645
leszko merged 2 commits intomainfrom
rafal/speed-up-loading

Conversation

@leszko
Copy link
Copy Markdown
Collaborator

@leszko leszko commented Mar 10, 2026

The /api/v1/pipelines/schemas and /api/v1/plugins endpoints were recomputing everything on every request (schema generation, distribution scanning, update check subprocesses). Since these don't change until a plugin install/uninstall (which restarts the server), cache the responses after first computation.

Summary by CodeRabbit

  • Refactor

    • Added in-process caching for pipeline schemas and plugin lists to avoid reconstructing them on repeated requests.
  • Tests

    • Test setup now clears internal caches before each run to ensure consistent, isolated test behavior.

The /api/v1/pipelines/schemas and /api/v1/plugins endpoints were
recomputing everything on every request (schema generation, distribution
scanning, update check subprocesses). Since these don't change until a
plugin install/uninstall (which restarts the server), cache the responses
after first computation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Rafal Leszko <[email protected]>
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 10, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2e4175fd-d8c6-45a6-be19-75f8acc826fa

📥 Commits

Reviewing files that changed from the base of the PR and between 6b450a0 and d643343.

📒 Files selected for processing (1)
  • tests/test_plugin_api.py

📝 Walkthrough

Walkthrough

Adds two module-level caches in src/scope/server/app.py and updates get_pipeline_schemas and list_plugins to return cached results when present; tests clear those caches in tests/test_plugin_api.py during setup.

Changes

Cohort / File(s) Summary
Server caching
src/scope/server/app.py
Introduces module-level caches (`_pipeline_schemas_cache: PipelineSchemasResponse
Tests — cache reset
tests/test_plugin_api.py
Clears the new module caches (_pipeline_schemas_cache and _plugins_list_cache) in the test setup after importing the app to ensure tests run with a fresh cache state.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

Hop, hop, hooray! 🐰 I stash what's known,
Two little caches keep answers grown.
Tests wake me fresh at every start,
Quick returns now play their part! ✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title 'Cache pipeline schemas and plugin list responses' clearly and concisely summarizes the main change: implementing caching for two specific API endpoints' responses.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch rafal/speed-up-loading

Comment @coderabbitai help to get the list of available commands and usage tips.

@leszko leszko requested a review from thomshutt March 10, 2026 10:41
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/scope/server/app.py (1)

2203-2227: ⚠️ Potential issue | 🟠 Major

Serialize the cold-cache fill for /api/v1/plugins.

The early None check happens before await plugin_manager.list_plugins_async(), so concurrent first requests can all miss and each run the expensive plugin scan/update checks. Guard the miss path with an asyncio.Lock or memoized task.

🔒 Suggested fix
+_plugins_list_cache_lock = asyncio.Lock()
+
 `@app.get`("/api/v1/plugins")
 `@cloud_proxy`()
 async def list_plugins(
@@
     global _plugins_list_cache
     if _plugins_list_cache is not None:
         return _plugins_list_cache
-
-    try:
-        plugin_manager = get_plugin_manager()
-        plugins_data = await plugin_manager.list_plugins_async()
+    async with _plugins_list_cache_lock:
+        if _plugins_list_cache is not None:
+            return _plugins_list_cache
+        try:
+            plugin_manager = get_plugin_manager()
+            plugins_data = await plugin_manager.list_plugins_async()
 
-        plugins = [_convert_plugin_dict_to_info(p) for p in plugins_data]
+            plugins = [_convert_plugin_dict_to_info(p) for p in plugins_data]
 
-        failed = [
-            FailedPluginInfoSchema(
-                package_name=f.package_name,
-                entry_point_name=f.entry_point_name,
-                error_type=f.error_type,
-                error_message=f.error_message,
+            failed = [
+                FailedPluginInfoSchema(
+                    package_name=f.package_name,
+                    entry_point_name=f.entry_point_name,
+                    error_type=f.error_type,
+                    error_message=f.error_message,
+                )
+                for f in plugin_manager.get_failed_plugins()
+            ]
+
+            response = PluginListResponse(
+                plugins=plugins, total=len(plugins), failed_plugins=failed
             )
-            for f in plugin_manager.get_failed_plugins()
-        ]
-
-        response = PluginListResponse(
-            plugins=plugins, total=len(plugins), failed_plugins=failed
-        )
-        _plugins_list_cache = response
-        return response
-    except Exception as e:
-        logger.error(f"Error listing plugins: {e}")
-        raise HTTPException(status_code=500, detail=str(e)) from e
+            _plugins_list_cache = response
+            return response
+        except Exception as e:
+            logger.error(f"Error listing plugins: {e}")
+            raise HTTPException(status_code=500, detail=str(e)) from e
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/scope/server/app.py` around lines 2203 - 2227, The code checks
_plugins_list_cache before awaiting plugin_manager.list_plugins_async, so
concurrent cold requests cause multiple expensive scans; fix by serializing the
cold-cache fill using an asyncio.Lock (e.g., a module-level _plugins_list_lock)
or by memoizing the in-progress task: acquire the lock (or check/create the
shared task) before calling get_plugin_manager() /
plugin_manager.list_plugins_async(), re-check _plugins_list_cache after
acquiring to avoid race, populate _plugins_list_cache with the created
PluginListResponse (plugins, total, failed_plugins) and release the lock (or let
the awaited task set the cache) so only one scan runs for concurrent first
requests while others await the result.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/scope/server/app.py`:
- Around line 110-113: The pipeline schema and plugins-list caches
(_pipeline_schemas_cache and _plugins_list_cache) are never invalidated, so
reload_plugin and plugin install/uninstall paths continue to serve stale data;
update the plugin install, uninstall and reload handlers (where reload_plugin is
invoked) to call the existing helper _clear_catalog_caches() immediately after a
successful plugin change and before returning the success response so the next
request rebuilds fresh data for /api/v1/plugins and /api/v1/pipelines/schemas.

---

Outside diff comments:
In `@src/scope/server/app.py`:
- Around line 2203-2227: The code checks _plugins_list_cache before awaiting
plugin_manager.list_plugins_async, so concurrent cold requests cause multiple
expensive scans; fix by serializing the cold-cache fill using an asyncio.Lock
(e.g., a module-level _plugins_list_lock) or by memoizing the in-progress task:
acquire the lock (or check/create the shared task) before calling
get_plugin_manager() / plugin_manager.list_plugins_async(), re-check
_plugins_list_cache after acquiring to avoid race, populate _plugins_list_cache
with the created PluginListResponse (plugins, total, failed_plugins) and release
the lock (or let the awaited task set the cache) so only one scan runs for
concurrent first requests while others await the result.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6939dafe-4c91-4cac-b5a6-4a4c71ecd168

📥 Commits

Reviewing files that changed from the base of the PR and between 3a76284 and 6b450a0.

📒 Files selected for processing (1)
  • src/scope/server/app.py

Comment on lines +110 to +113
# Cached responses for pipeline schemas and plugin list.
# Server restarts after plugin install/uninstall, so these are naturally reset.
_pipeline_schemas_cache: PipelineSchemasResponse | None = None
_plugins_list_cache: object | None = None
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Invalidate these caches when plugins change.

These responses are now cached forever, but reload_plugin can change plugin metadata and add/remove pipelines without restarting the process. After the first hit, /api/v1/plugins and /api/v1/pipelines/schemas will keep serving the pre-reload snapshot.

🧹 Suggested fix
 _pipeline_schemas_cache: PipelineSchemasResponse | None = None
 _plugins_list_cache: object | None = None
+
+def _clear_catalog_caches() -> None:
+    global _pipeline_schemas_cache, _plugins_list_cache
+    _pipeline_schemas_cache = None
+    _plugins_list_cache = None

Then call _clear_catalog_caches() after successful plugin install/uninstall/reload paths, before returning the success response.

Also applies to: 655-678, 2203-2227

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/scope/server/app.py` around lines 110 - 113, The pipeline schema and
plugins-list caches (_pipeline_schemas_cache and _plugins_list_cache) are never
invalidated, so reload_plugin and plugin install/uninstall paths continue to
serve stale data; update the plugin install, uninstall and reload handlers
(where reload_plugin is invoked) to call the existing helper
_clear_catalog_caches() immediately after a successful plugin change and before
returning the success response so the next request rebuilds fresh data for
/api/v1/plugins and /api/v1/pipelines/schemas.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 10, 2026

🚀 fal.ai Preview Deployment

App ID daydream/scope-pr-645--preview
WebSocket wss://fal.run/daydream/scope-pr-645--preview/ws
Commit d643343

Testing

Connect to this preview deployment by running this on your branch:

uv run build && SCOPE_CLOUD_APP_ID="daydream/scope-pr-645--preview/ws" uv run daydream-scope

🧪 E2E tests will run automatically against this deployment.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 10, 2026

✅ E2E Tests passed

Status passed
fal App daydream/scope-pr-645--preview
Run View logs

Test Artifacts

Check the workflow run for screenshots.

The cached plugin list response from a previous test was being returned
instead of the mock, causing test_returns_plugin_list to fail.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Signed-off-by: Rafal Leszko <[email protected]>
@leszko leszko merged commit 9e7a370 into main Mar 10, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants