Skip to content

fix: handle orphan files and return appropriate responses when syncing connectors (backport of #1785)#2016

Merged
ricofurtado merged 2 commits into
mainfrom
cpd-backport-1785-orphan-files
Jul 3, 2026
Merged

fix: handle orphan files and return appropriate responses when syncing connectors (backport of #1785)#2016
ricofurtado merged 2 commits into
mainfrom
cpd-backport-1785-orphan-files

Conversation

@ricofurtado

@ricofurtado ricofurtado commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Backport of #1785 from release-cpd to main.

Most of the original PR's content (orphan reconciliation, filename-vs-hash checks) was already independently implemented on main via #1592 and #1715. The only net-new change here is the Google Drive OAuth refresh request actually enforcing its timeout (_TimeoutSession) — the version on main set session.timeout, which requests.Session does not honor.

Test plan

  • python3 -m py_compile on touched files
  • Full unit suite for touched areas passes

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Improved the reliability of Google Drive authentication refresh and revocation requests by consistently applying request timeouts.
    • Reduced the chance of stalled or long-running connection calls during credential updates.

…g connectors (#1785)

* feat: handle orphan files and return appropriate responses when syncing connectors

* feat: refactor GoogleDriveOAuth to separate required scopes and add handling for missing optional group scopes in tests

* style: ruff autofix (auto)

* feat: add handling for skipped files and warnings in task processing and UI components

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
(cherry picked from commit aca865a)
@github-actions github-actions Bot added backend 🔷 Issues related to backend services (OpenSearch, Langflow, APIs) bug 🔴 Something isn't working. labels Jul 3, 2026
@coderabbitai

coderabbitai Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Refactors the bounded-timeout HTTP transport in Google OAuth credential handling by introducing a _TimeoutSession subclass of requests.Session that injects a default timeout on every request, replacing the prior approach of setting a timeout attribute directly on a plain session.

Changes

Timeout Session Refactor

Layer / File(s) Summary
Timeout-injecting session and usage
src/connectors/google_drive/oauth.py
Adds _TimeoutSession class overriding request() to default timeout kwarg, and updates _make_timeout_request() to construct the google-auth Request using this session instead of setting a session.timeout attribute.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: lucaseduoli

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Title check ⚠️ Warning The title describes orphan-file sync behavior, but the actual change only refactors Google Drive OAuth timeout handling. Rename it to reflect the Google Drive OAuth timeout fix, e.g. "fix: enforce timeout on Google Drive OAuth refresh requests".
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cpd-backport-1785-orphan-files

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 3, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/connectors/google_drive/oauth.py`:
- Around line 18-27: The OAuth transport timeout override in
_TimeoutSession.request is being bypassed because setdefault() preserves the
120s timeout passed by google.auth.transport.requests.Request. Update the
request override so it always enforces _REFRESH_TIMEOUT_SECONDS for calls made
through load_credentials() and revoke_credentials(), using the
_TimeoutSession.request hook to replace any existing timeout instead of leaving
kwargs["timeout"] unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 53cf922b-e216-4d67-be4b-42a19f24939c

📥 Commits

Reviewing files that changed from the base of the PR and between bdd873f and 9fdc6b4.

📒 Files selected for processing (1)
  • src/connectors/google_drive/oauth.py

Comment on lines +18 to +27
class _TimeoutSession(req_lib.Session):
def __init__(self, timeout: float) -> None:
super().__init__()
self._timeout = timeout

def request(self, method, url, **kwargs):
kwargs.setdefault("timeout", self._timeout)
return super().request(method, url, **kwargs)


Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

google-auth python Request.__call__ default timeout parameter behavior

💡 Result:

In the google-auth Python library, the Request.call method, specifically in the implementation provided by google.auth.transport.requests, uses a default timeout of 120 seconds [1][2][3]. Key behavioral details regarding this timeout parameter include: 1. Default Value: The method defines a constant (commonly referred to as _DEFAULT_TIMEOUT in the source code) set to 120 seconds [4][5][2]. 2. Parameter Behavior: If you do not specify a timeout, or if you explicitly pass None, the library defaults to this 120-second value when making the underlying HTTP request using the requests library [1][4][6]. 3. Implementation: This timeout value is passed directly to the requests.Session.request method [4][2]. Consequently, it follows the behavior of the requests library's timeout parameter, which can accept either a single float/int for the total timeout or a tuple representing (connect_timeout, read_timeout) [1][7][8]. 4. Abstract Base Class: It is important to note that google.auth.transport.Request is an abstract base class that defines the call interface with a default timeout of None [9][10]. Implementations of this abstract class (like the requests-based transport mentioned above) are responsible for defining their own specific default timeout behavior [9][10]. If you need to override this behavior, you should explicitly pass a different value (or a tuple) to the timeout parameter when calling the Request object [4][2].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the OAuth helper and its usage site.
git ls-files src/connectors/google_drive/oauth.py src/connectors/google_drive | sed -n '1,20p'
echo '--- file outline ---'
ast-grep outline src/connectors/google_drive/oauth.py --view expanded || true
echo '--- relevant lines ---'
nl -ba src/connectors/google_drive/oauth.py | sed -n '1,140p'
echo '--- search for Request/session usage ---'
rg -n "Request\\(|_TimeoutSession|_REFRESH_TIMEOUT_SECONDS|revoke\\(|refresh\\(" src/connectors/google_drive -n

Repository: langflow-ai/openrag

Length of output: 1467


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
path = Path("src/connectors/google_drive/oauth.py")
for start, end in [(1, 130), (130, 260)]:
    print(f"--- lines {start}-{end} ---")
    for i, line in enumerate(path.read_text().splitlines(), start=1):
        if start <= i <= end:
            print(f"{i:4d}: {line}")
PY

Repository: langflow-ai/openrag

Length of output: 11223


Force the OAuth transport timeout override

google.auth.transport.requests.Request already forwards timeout=120 into session.request(), so setdefault() never replaces it. load_credentials() and revoke_credentials() will still wait up to 120s instead of _REFRESH_TIMEOUT_SECONDS (30s).

🐛 Proposed fix
-        kwargs.setdefault("timeout", self._timeout)
+        kwargs["timeout"] = self._timeout
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
class _TimeoutSession(req_lib.Session):
def __init__(self, timeout: float) -> None:
super().__init__()
self._timeout = timeout
def request(self, method, url, **kwargs):
kwargs.setdefault("timeout", self._timeout)
return super().request(method, url, **kwargs)
class _TimeoutSession(req_lib.Session):
def __init__(self, timeout: float) -> None:
super().__init__()
self._timeout = timeout
def request(self, method, url, **kwargs):
kwargs["timeout"] = self._timeout
return super().request(method, url, **kwargs)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/connectors/google_drive/oauth.py` around lines 18 - 27, The OAuth
transport timeout override in _TimeoutSession.request is being bypassed because
setdefault() preserves the 120s timeout passed by
google.auth.transport.requests.Request. Update the request override so it always
enforces _REFRESH_TIMEOUT_SECONDS for calls made through load_credentials() and
revoke_credentials(), using the _TimeoutSession.request hook to replace any
existing timeout instead of leaving kwargs["timeout"] unchanged.

@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 3, 2026
@github-actions github-actions Bot added the lgtm label Jul 3, 2026
@ricofurtado ricofurtado merged commit 79b33f0 into main Jul 3, 2026
34 of 46 checks passed
@github-actions github-actions Bot deleted the cpd-backport-1785-orphan-files branch July 3, 2026 21:44
ricofurtado added a commit that referenced this pull request Jul 5, 2026
…to release-saas-ga-0.6.2 (#2021)

* fix: error in skip duplicates functionality for folder ingest for connectors (backport of #1941) (#2018)

* fix: error in skip duplicates functionality for folder ingest for connectors 75616(#1842) (#1941)

* fix: enhance duplicate handling and indexing for connector file uploads

* style: ruff autofix (auto)

* fix: improve duplicate handling in connector sync functionality

* chore: trigger CodeRabbit

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
(cherry picked from commit effaf02)

* style: ruff autofix (auto)

* fix: remove duplicate test definitions from main merge

The automated merge-from-main on this branch concatenated two tests
(test_connector_processor_indexes_cleaned_filename,
test_langflow_connector_processor_uses_cleaned_filename) that already
existed under both names, tripping ruff F811. Keep one copy of each,
preferring the version wired with connector_id_exists=True to match
current check_document_exists behavior.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
(cherry picked from commit 286e9db)

* fix: Ingestion of buckets not working right after connection is tested - CPD #75506 (#1942) (#2019)

* fix: update bucket ingestion messages for clarity and consistency

* fix: guard bucket pre-selection against connection ID mismatch

The defaults query returns the first S3/COS connection regardless of active
status. Only pre-select saved bucket_names when the defaults connection_id
matches the current connector.connectionId to avoid seeding the wrong
buckets if a stale connection exists alongside the active one.

* fix: prevent defaults from overwriting user bucket selections

Two issues in the initial-selection effect:
1. hasAppliedInitial was only set when valid buckets were found, leaving the
   effect live across bucket refreshes and allowing stale defaults to overwrite
   selections made after a refetch.
2. No guard against the async race where buckets loads before initialSelectedBuckets:
   user clicks buckets, defaults arrive later and silently overwrite them.

Fix: mark hasAppliedInitial unconditionally on first evaluation, and only apply
defaults when selectedBuckets is still empty (user hasn't acted yet).

---------

(cherry picked from commit f3823b8)

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
(cherry picked from commit 36f115f)

* fix: handle orphan files and return appropriate responses when syncing connectors (#1785) (#2016)

* feat: handle orphan files and return appropriate responses when syncing connectors

* feat: refactor GoogleDriveOAuth to separate required scopes and add handling for missing optional group scopes in tests

* style: ruff autofix (auto)

* feat: add handling for skipped files and warnings in task processing and UI components

---------

(cherry picked from commit aca865a)

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
(cherry picked from commit 79b33f0)

* fix: enhance error handling for document processing and add OCR requirement check for image files (#1859) (#2017)

* fix: enhance error handling for document processing and add OCR requirement check for image files

* style: ruff autofix (auto)

* remove clickhouse connect from langflow image (#1801)

* chore: fix ci (#1889)

* Update processors.py

* test fix

* fix ci

---------

(cherry picked from commit a286581)

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: Edwin Jose <[email protected]>
(cherry picked from commit 4c6acd8)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: Edwin Jose <[email protected]>
lucaseduoli added a commit that referenced this pull request Jul 7, 2026
…model option (#2031)

* fix: remediate image_scan CVEs in backend, frontend, and langflow images (#2000)

* fix light color text catergory chips in light mode (#2013)

Co-authored-by: Olfa Maslah <[email protected]>

* fix: upgraded version of docling (#1995)

* changed version of docling

* fix logs not being the correct ones

* go back to sequential building to save space

* add cache to builds

* remove single use results from docling manager

* revert single use results

* change the ray tenant id header

* added env

* changed cache to hit if no changes were made

* fix hash for cache

* style: ruff autofix (auto)

* fixed coderabbit

* style: ruff autofix (auto)

* added comment on docling manager

* style: ruff autofix (auto)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* feat: add chunk level page source (#2015)

* Changed export docling document to append page number

* update result on frontend to append page number

* document how to get page number in readmes

* Update flows/components/export_docling_document.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update flows/components/export_docling_document.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix: apply CodeRabbit auto-fixes

Fixed 1 file(s) based on 1 unresolved review comment.

Co-authored-by: CodeRabbit <[email protected]>

* update component code

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <[email protected]>

* fix: error in skip duplicates functionality for folder ingest for connectors (backport of #1941) (#2018)

* fix: error in skip duplicates functionality for folder ingest for connectors 75616(#1842) (#1941)

* fix: enhance duplicate handling and indexing for connector file uploads

* style: ruff autofix (auto)

* fix: improve duplicate handling in connector sync functionality

* chore: trigger CodeRabbit

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
(cherry picked from commit effaf02)

* style: ruff autofix (auto)

* fix: remove duplicate test definitions from main merge

The automated merge-from-main on this branch concatenated two tests
(test_connector_processor_indexes_cleaned_filename,
test_langflow_connector_processor_uses_cleaned_filename) that already
existed under both names, tripping ruff F811. Keep one copy of each,
preferring the version wired with connector_id_exists=True to match
current check_document_exists behavior.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix: Ingestion of buckets not working right after connection is tested - CPD #75506 (#1942) (#2019)

* fix: update bucket ingestion messages for clarity and consistency

* fix: guard bucket pre-selection against connection ID mismatch

The defaults query returns the first S3/COS connection regardless of active
status. Only pre-select saved bucket_names when the defaults connection_id
matches the current connector.connectionId to avoid seeding the wrong
buckets if a stale connection exists alongside the active one.



* fix: prevent defaults from overwriting user bucket selections

Two issues in the initial-selection effect:
1. hasAppliedInitial was only set when valid buckets were found, leaving the
   effect live across bucket refreshes and allowing stale defaults to overwrite
   selections made after a refetch.
2. No guard against the async race where buckets loads before initialSelectedBuckets:
   user clicks buckets, defaults arrive later and silently overwrite them.

Fix: mark hasAppliedInitial unconditionally on first evaluation, and only apply
defaults when selectedBuckets is still empty (user hasn't acted yet).



---------


(cherry picked from commit f3823b8)

Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix: handle orphan files and return appropriate responses when syncing connectors (#1785) (#2016)

* feat: handle orphan files and return appropriate responses when syncing connectors

* feat: refactor GoogleDriveOAuth to separate required scopes and add handling for missing optional group scopes in tests

* style: ruff autofix (auto)

* feat: add handling for skipped files and warnings in task processing and UI components

---------


(cherry picked from commit aca865a)

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix: enhance error handling for document processing and add OCR requirement check for image files (#1859) (#2017)

* fix: enhance error handling for document processing and add OCR requirement check for image files

* style: ruff autofix (auto)

* remove clickhouse connect from langflow image (#1801)

* chore: fix ci (#1889)

* Update processors.py

* test fix

* fix ci

---------




(cherry picked from commit a286581)

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: Edwin Jose <[email protected]>

* fix: bump golang.org/x/net to v0.55.0 to remediate CVE-2026-25680 and CVE-2026-39821 (#2014)

* feat: add new design for messages and add inline references (#2020)

* update opensearch multimodal and flows to support parser, chunk size and chunk overlap

* update backend to support citations and chunk metadata

* update frontend to support citations and metadata, implement new design for assistant message

* changed the prompt for the agent

* changed the prompt for the agent

* remove hallucinated chunk references

* adjust design of citations and where popover appears

* adjusted design for message, popover and cards

* fix React Doctor picks

* fix: apply CodeRabbit auto-fixes

Fixed 5 file(s) based on 4 unresolved review comments.

Co-authored-by: CodeRabbit <[email protected]>

* style: apply biome auto-fixes [skip ci]

* updated colors in ibm mode

* fixed react doctor picks

* added correct type and validation for function calls

* remove style for message when on onboarding

* fix message not being w full

* remove search query in end of the message

* change popup to use shadcn popover

* fix coderabbit's suggestion

* increase prompt limit

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* apply new settings on retry (#2010)

* fix: preserve HTTPException status in update_docling_preset (#1586) (#2004)

* fix: report actual synced connection count in connector_sync (#1547) (#2006)

* fix: restore LLM model values in reapply_all_settings fallback (#1587) (#2008)

* fix: prevent partial matches for exact token searches (#2003)

* fix: prevent partial matches for exact token searches (#1937)

* style: ruff autofix (auto)

* chore: remove unused json import

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix: target agent component when updating chat flow system prompt (#2001)

* fix: target langflow agent component for system prompt update instead of provider LLM component

Signed-off-by: vchen7629 <[email protected]>

* fix: removed unused llm_provider arg from _update_langflow_system_prompt

Signed-off-by: vchen7629 <[email protected]>

* test: add unit tests for update_chat_flow_system_prompt

Signed-off-by: vchen7629 <[email protected]>

* style: ruff autofix (auto)

* fix(test): resolve flow path relative to repo root in test_langflow_agent_system_prompt

Signed-off-by: vchen7629 <[email protected]>

* style: ruff autofix (auto)

---------

Signed-off-by: vchen7629 <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix: replaced hardcoded docker.io prefix with a env variable prefix for private repo compatibility (#2002)

Signed-off-by: vchen7629 <[email protected]>

* fix: resolve Pydantic delta serialization warnings in agent streaming (#2005)

* fix: updated agent model dump to exclude delta field

Signed-off-by: vchen7629 <[email protected]>

* test: added regression unit test to verify response field + no warnings

* style: ruff autofix (auto)

* test: added additional unit test to verify that model_dump without exclusion raises the warning

Signed-off-by: vchen7629 <[email protected]>

* style: ruff autofix (auto)

* test: tighten delta warning assertion to match specific Pydantic error

Signed-off-by: vchen7629 <[email protected]>

* style: ruff autofix (auto)

---------

Signed-off-by: vchen7629 <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix: prevent exception details from leaking in API error responses (#2007)

* fix: removed str(e) in exceptions to prevent internals from leaking

Signed-off-by: vchen7629 <[email protected]>

* style: ruff autofix (auto)

* fix: added missing status_code

Signed-off-by: vchen7629 <[email protected]>

* style: ruff autofix (auto)

---------

Signed-off-by: vchen7629 <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* Changed docling options to include local support and be shown in the Langflow tab

* style: ruff autofix (auto)

* enable ollama and delete unused file

* fix next lint

---------

Signed-off-by: vchen7629 <[email protected]>
Co-authored-by: Gautham N Pai <[email protected]>
Co-authored-by: Wallgau <[email protected]>
Co-authored-by: Olfa Maslah <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <[email protected]>
Co-authored-by: Rico Furtado <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Edwin Jose <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mike Fortman <[email protected]>
Co-authored-by: hunterxtang <[email protected]>
Co-authored-by: NishadA05 <[email protected]>
Co-authored-by: Zephyrus <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend 🔷 Issues related to backend services (OpenSearch, Langflow, APIs) bug 🔴 Something isn't working. lgtm

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants