fix: handle orphan files and return appropriate responses when syncing connectors#1715
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughThis PR filters trashed/orphaned connector files before syncing (Drive-aware), reconciles and skips orphan IDs, reports connectors that only had deletions, adjusts preview counts, updates the frontend response type, and adds unit tests covering these behaviors. ChangesOrphan Reconciliation in Connector Sync
Sequence Diagram(s)sequenceDiagram
participant Client
participant SyncAll
participant ReconcileOrphans
participant SyncSpecific
Client->>SyncAll: POST /sync_all
SyncAll->>ReconcileOrphans: reconcile_orphans_for_connector_type(existing_file_ids)
ReconcileOrphans-->>SyncAll: orphan_ids
SyncAll->>SyncAll: existing_file_ids -= orphan_ids
alt existing_file_ids empty
SyncAll-->>Client: 200 { status: "no_files", deleted_only_connectors: [...] }
else
SyncAll->>SyncSpecific: sync_specific_files(existing_file_ids)
SyncSpecific-->>SyncAll: tasks/results
SyncAll-->>Client: 200 { status: "ok", tasks: [...] }
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
src/api/connectors.py (2)
699-720:⚠️ Potential issue | 🟠 Major | ⚡ Quick win
max_filesskips orphan cleanup without actually limiting this resync.In this branch,
body.max_filesis never passed toreconcile_orphans_for_connector_type()orsync_specific_files(). Setting it only disables orphan filtering, but the endpoint still enqueues every indexed ID, including stale ones.🐛 Proposed fix
- if body.max_files is None: - orphan_ids = await reconcile_orphans_for_connector_type( - connector_type=connector_type, - user_id=user.user_id, - connector_service=connector_service, - session_manager=session_manager, - jwt_token=jwt_token, - existing_file_ids=existing_file_ids, - id_field=id_field, - ) - if orphan_ids: - orphan_id_set = set(orphan_ids) - ids_to_sync = [fid for fid in existing_file_ids if fid not in orphan_id_set] + orphan_ids = await reconcile_orphans_for_connector_type( + connector_type=connector_type, + user_id=user.user_id, + connector_service=connector_service, + session_manager=session_manager, + jwt_token=jwt_token, + existing_file_ids=existing_file_ids, + id_field=id_field, + ) + if orphan_ids: + orphan_id_set = set(orphan_ids) + ids_to_sync = [fid for fid in existing_file_ids if fid not in orphan_id_set]🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/connectors.py` around lines 699 - 720, The resync currently ignores body.max_files — pass body.max_files into reconcile_orphans_for_connector_type so orphan cleanup respects the limit, and ensure ids_to_sync is truncated to body.max_files (or pass max_files into sync_specific_files) before enqueuing; update references in this block to use body.max_files when computing ids_to_sync and when calling reconcile_orphans_for_connector_type or sync_specific_files so only up to max_files IDs are processed.
699-724: 🛠️ Refactor suggestion | 🟠 Major | 🏗️ Heavy liftExtract orphan sync planning out of the FastAPI handlers.
The new orphan-filter / deleted-only branching is now duplicated across two routes and is already drifting in behavior. Moving this planning into
connector_serviceor a dedicated helper would keep the handlers as HTTP adapters and centralize one sync policy.As per coding guidelines,
src/api/**/*.py: "No business logic in route handlers."Also applies to: 1188-1204, 1248-1257
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/connectors.py` around lines 699 - 724, The orphan-filtering and deleted-only branching logic (the reconcile_orphans_for_connector_type call and the ids_to_sync build-up) should be moved out of the FastAPI handler into connector_service (or a small helper) so the handler is just an HTTP adapter; add a method like connector_service.plan_ids_to_sync(existing_file_ids, max_files, connector_type, user_id, session_manager, jwt_token, id_field) that encapsulates the current logic (call reconcile_orphans_for_connector_type when max_files is None, compute orphan_id_set and final ids_to_sync, and return the ids_to_sync or an empty list/None to indicate no files), then replace the in-handler block that references reconcile_orphans_for_connector_type, ids_to_sync, and the subsequent if-not-ids_to_sync response with a simple call to that new method and handle the no-files response; do the same replacement at the other duplicated locations (the other handler blocks that currently duplicate this logic).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/connectors/google_drive/connector.py`:
- Around line 347-349: The shortcut check is ineffective because
_resolve_shortcut() never requests the target file's "trashed" field, so
resolved.get("trashed") is always falsy; update _resolve_shortcut to include the
target's trashed attribute when fetching the shortcut target (e.g., add
"trashed" to the fields param used in the files().get or files().list call that
resolves the shortcut) so resolved contains the real trashed value, and apply
the same fix for the other shortcut check site (the similar guard around lines
425-426) so both guards correctly detect trashed targets.
---
Outside diff comments:
In `@src/api/connectors.py`:
- Around line 699-720: The resync currently ignores body.max_files — pass
body.max_files into reconcile_orphans_for_connector_type so orphan cleanup
respects the limit, and ensure ids_to_sync is truncated to body.max_files (or
pass max_files into sync_specific_files) before enqueuing; update references in
this block to use body.max_files when computing ids_to_sync and when calling
reconcile_orphans_for_connector_type or sync_specific_files so only up to
max_files IDs are processed.
- Around line 699-724: The orphan-filtering and deleted-only branching logic
(the reconcile_orphans_for_connector_type call and the ids_to_sync build-up)
should be moved out of the FastAPI handler into connector_service (or a small
helper) so the handler is just an HTTP adapter; add a method like
connector_service.plan_ids_to_sync(existing_file_ids, max_files, connector_type,
user_id, session_manager, jwt_token, id_field) that encapsulates the current
logic (call reconcile_orphans_for_connector_type when max_files is None, compute
orphan_id_set and final ids_to_sync, and return the ids_to_sync or an empty
list/None to indicate no files), then replace the in-handler block that
references reconcile_orphans_for_connector_type, ids_to_sync, and the subsequent
if-not-ids_to_sync response with a simple call to that new method and handle the
no-files response; do the same replacement at the other duplicated locations
(the other handler blocks that currently duplicate this logic).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 71b33e51-d08c-41ab-a2ce-804f30683879
📒 Files selected for processing (4)
frontend/app/api/mutations/useSyncConnector.tssrc/api/connectors.pysrc/connectors/google_drive/connector.pytests/unit/api/test_reconcile_orphans_for_connector_type.py
89287e6 to
055ec30
Compare
…and update tests for shortcut handling
l
Summary by CodeRabbit
Bug Fixes
New Features
Tests