Skip to content

fix: google to support WEBHOOK_BASE_URL#1849

Merged
edwinjosechittilappilly merged 3 commits into
mainfrom
feat-add-webhook-base-google
Jun 11, 2026
Merged

fix: google to support WEBHOOK_BASE_URL#1849
edwinjosechittilappilly merged 3 commits into
mainfrom
feat-add-webhook-base-google

Conversation

@edwinjosechittilappilly

@edwinjosechittilappilly edwinjosechittilappilly commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Google to support WEBHOOK_BASE_URL .

Summary by CodeRabbit

  • Improvements

    • Google Drive connector: webhook URL selection now uses multiple fallbacks (including a legacy env override) for more flexible setup.
    • Webhook handling: incoming channel IDs are normalized/trimmed for more reliable processing.
    • Google Drive file processing skips trashed items when explicit file IDs are provided, reducing noise.
  • Imacts to Integrations

    • OneDrive and SharePoint subscriptions now use the configured webhook URL verbatim.
  • Tests

    • New and updated tests cover webhook URL resolution and subscription behaviors.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: eefe7cf6-fb13-41ab-ad71-300d13b240e6

📥 Commits

Reviewing files that changed from the base of the PR and between 1e49838 and 85afecc.

📒 Files selected for processing (6)
  • src/connectors/google_drive/connector.py
  • src/connectors/onedrive/connector.py
  • src/connectors/sharepoint/connector.py
  • tests/unit/connectors/test_connector_file_type_validation.py
  • tests/unit/connectors/test_langflow_connector_txt_to_md.py
  • tests/unit/connectors/test_webhook_notification_url.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/connectors/google_drive/connector.py

Walkthrough

Centralizes Google Drive webhook URL resolution and header extraction, skips trashed Drive items when iterating explicit file_ids, uses configured webhook_url verbatim for OneDrive/SharePoint Graph subscriptions, adds a GOOGLE_DRIVE_WEBHOOK_URL setting, and adds tests covering webhook resolution and subscription payloads.

Changes

Connectors webhook & tests

Layer / File(s) Summary
Settings constant and Google Drive webhook helpers
src/config/settings.py, src/connectors/google_drive/connector.py
Adds GOOGLE_DRIVE_WEBHOOK_URL; implements _resolve_webhook_address() with precedence (config["webhook_url"], GOOGLE_DRIVE_WEBHOOK_URL, WEBHOOK_BASE_URL + /connectors/google_drive/webhook, cfg.webhook_address); adds extract_webhook_channel_id(); updates setup_subscription() messaging/use.
Google Drive: skip trashed items
src/connectors/google_drive/connector.py
_iter_selected_items() now early-skips file metadata marked trashed when processing explicit file_ids.
Graph subscription notificationUrl usage
src/connectors/onedrive/connector.py, src/connectors/sharepoint/connector.py
OneDrive and SharePoint setup_subscription() now pass webhook_url directly as notificationUrl (no appended /webhook/<type> path).
Unit tests and small test fixes
tests/unit/connectors/*
Adds test_webhook_notification_url.py covering Graph subscription payloads and Google Drive webhook resolution precedence; updates small test fixes in existing connector tests (MagicMock request arg, session_manager = None).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested labels

tests

Suggested reviewers

  • ricofurtado
  • lucaseduoli
  • mfortman11
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 32.14% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix: google to support WEBHOOK_BASE_URL' directly and concisely describes the main change—adding WEBHOOK_BASE_URL support to the Google Drive connector.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat-add-webhook-base-google

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added backend 🔷 Issues related to backend services (OpenSearch, Langflow, APIs) bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jun 11, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/connectors/google_drive/connector.py`:
- Around line 866-868: Replace the direct environment access
os.getenv("GOOGLE_DRIVE_WEBHOOK_URL") (the variable named legacy) with a value
loaded from the central configuration module: add GOOGLE_DRIVE_WEBHOOK_URL =
os.getenv("GOOGLE_DRIVE_WEBHOOK_URL") to config/settings.py and then import and
use that setting in place of the os.getenv call in connector.py (remove the
direct os.getenv usage and reference settings.GOOGLE_DRIVE_WEBHOOK_URL or the
imported symbol instead).
- Around line 875-877: The code references self.cfg.webhook_address but
GoogleDriveConfig lacks that field, so either remove the unreachable getter
lines in connector.py that read webhook_address, or add the field to
GoogleDriveConfig (e.g., add webhook_address: Optional[str] = None to the
dataclass and import typing.Optional) so getattr(self.cfg, "webhook_address",
None) can return a configured value; update GoogleDriveConfig (the dataclass
definition) or delete the webhook_address handling in the connector accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7b862478-f114-4a1f-af5b-7ccfbf09f6d9

📥 Commits

Reviewing files that changed from the base of the PR and between 5bf8fe4 and 8be1721.

📒 Files selected for processing (1)
  • src/connectors/google_drive/connector.py

Comment thread src/connectors/google_drive/connector.py Outdated
Comment thread src/connectors/google_drive/connector.py
@github-actions github-actions Bot added the lgtm label Jun 11, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jun 11, 2026
@github-actions github-actions Bot added tests bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jun 11, 2026
@edwinjosechittilappilly edwinjosechittilappilly merged commit 5ed2d68 into main Jun 11, 2026
20 checks passed
@github-actions github-actions Bot deleted the feat-add-webhook-base-google branch June 11, 2026 23:39
ricofurtado pushed a commit that referenced this pull request Jun 12, 2026
* Update connector.py

* update env in settings

* fix webhook url of sharepoint
ricofurtado added a commit that referenced this pull request Jun 12, 2026
…dingly (#1851)

* fix: handle document processing errors and update task statuses accordingly

* fix: add OCR handling for raster images during file ingestion

* style: ruff autofix (auto)

* fix: google to support WEBHOOK_BASE_URL (#1849)

* Update connector.py

* update env in settings

* fix webhook url of sharepoint

* feat: crd report scaleddown phase status (#1846)

* crd report scaleddown phase status

* improve status message

* chore: release 0.4.0.dev0 of sdks (#1847)

* release 0.4.1 of sdks

* dev 0

* dev 0

* 0.4.0-dev0

* feat: sdk scripts (#1848)

* sdk scripts

* Update main.py

* fix: put correct file type in langflow-less ingestion (#1842)

* FIxed no langflow ingestion

* style: ruff autofix (auto)

* add fallback for mime types

* style: ruff autofix (auto)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* feat: add Python SDK smoke-test suite for remote/IBM SaaS deployments

Standalone uv project under scripts/test_scripts/sdk/python that exercises
every openrag-sdk functionality (settings, models, documents, search, chat,
knowledge filters, error handling) against a remote instance using IBM SaaS
header auth (X-Username / X-Api-Key), with pass/fail/skip reporting to
console, markdown, JSON, and a run log.

* style: ruff autofix (auto)

* Potential fix for pull request finding 'CodeQL / Clear-text logging of sensitive information'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* style: ruff autofix (auto)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Edwin Jose <[email protected]>
Co-authored-by: ming <[email protected]>
Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Vchen7629 pushed a commit to Vchen7629/openrag that referenced this pull request Jun 15, 2026
…dingly (langflow-ai#1851)

* fix: handle document processing errors and update task statuses accordingly

* fix: add OCR handling for raster images during file ingestion

* style: ruff autofix (auto)

* fix: google to support WEBHOOK_BASE_URL (langflow-ai#1849)

* Update connector.py

* update env in settings

* fix webhook url of sharepoint

* feat: crd report scaleddown phase status (langflow-ai#1846)

* crd report scaleddown phase status

* improve status message

* chore: release 0.4.0.dev0 of sdks (langflow-ai#1847)

* release 0.4.1 of sdks

* dev 0

* dev 0

* 0.4.0-dev0

* feat: sdk scripts (langflow-ai#1848)

* sdk scripts

* Update main.py

* fix: put correct file type in langflow-less ingestion (langflow-ai#1842)

* FIxed no langflow ingestion

* style: ruff autofix (auto)

* add fallback for mime types

* style: ruff autofix (auto)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* feat: add Python SDK smoke-test suite for remote/IBM SaaS deployments

Standalone uv project under scripts/test_scripts/sdk/python that exercises
every openrag-sdk functionality (settings, models, documents, search, chat,
knowledge filters, error handling) against a remote instance using IBM SaaS
header auth (X-Username / X-Api-Key), with pass/fail/skip reporting to
console, markdown, JSON, and a run log.

* style: ruff autofix (auto)

* Potential fix for pull request finding 'CodeQL / Clear-text logging of sensitive information'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* style: ruff autofix (auto)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Edwin Jose <[email protected]>
Co-authored-by: ming <[email protected]>
Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
ricofurtado pushed a commit that referenced this pull request Jun 15, 2026
* Update connector.py

* update env in settings

* fix webhook url of sharepoint
ricofurtado added a commit that referenced this pull request Jun 15, 2026
…dingly (#1851)

* fix: handle document processing errors and update task statuses accordingly

* fix: add OCR handling for raster images during file ingestion

* style: ruff autofix (auto)

* fix: google to support WEBHOOK_BASE_URL (#1849)

* Update connector.py

* update env in settings

* fix webhook url of sharepoint

* feat: crd report scaleddown phase status (#1846)

* crd report scaleddown phase status

* improve status message

* chore: release 0.4.0.dev0 of sdks (#1847)

* release 0.4.1 of sdks

* dev 0

* dev 0

* 0.4.0-dev0

* feat: sdk scripts (#1848)

* sdk scripts

* Update main.py

* fix: put correct file type in langflow-less ingestion (#1842)

* FIxed no langflow ingestion

* style: ruff autofix (auto)

* add fallback for mime types

* style: ruff autofix (auto)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* feat: add Python SDK smoke-test suite for remote/IBM SaaS deployments

Standalone uv project under scripts/test_scripts/sdk/python that exercises
every openrag-sdk functionality (settings, models, documents, search, chat,
knowledge filters, error handling) against a remote instance using IBM SaaS
header auth (X-Username / X-Api-Key), with pass/fail/skip reporting to
console, markdown, JSON, and a run log.

* style: ruff autofix (auto)

* Potential fix for pull request finding 'CodeQL / Clear-text logging of sensitive information'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* style: ruff autofix (auto)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Edwin Jose <[email protected]>
Co-authored-by: ming <[email protected]>
Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
ricofurtado pushed a commit that referenced this pull request Jun 25, 2026
* Update connector.py

* update env in settings

* fix webhook url of sharepoint
ricofurtado added a commit that referenced this pull request Jul 1, 2026
* feat:  add connectors permissions for Saas (#1782)

* feat: add SaaS workspace connector permissions with RBAC gates
Introduce admin Connectors Permission settings (cloud-only) backed by
workspace_config, enforce availability on connector APIs and OAuth init,
and sync the new connectors:manage:access permission at startup without
an Alembic migration. OSS/dev OSS brand bypasses policy; add dev role
toggle and brand-aware proxy headers for local RBAC testing.

* style: ruff autofix (auto)

* address CodeRabbit comments and fix backend lint unused imports

* fix backend lint

* feat: workspace connector permissions with aligned SaaS policy guards
Add admin Connectors Permission (workspace_config-backed) with RBAC
connectors:manage:access, enforce policy on connector APIs and OAuth init,
and keep the permission list independent of the live connectors tab.
Consolidate frontend connector-access hooks into useGetConnectorsQuery and
settings tab helpers into brand.ts. Fix Connectors Permission tab redirects
(RSC dev-brand default, wait for permissionsResolved). Let explicitly
enabled connectors override deployment visibility filters; add
OPENRAG_DEV_CONNECTOR_POLICY for local OSS dev backend enforcement Snapshot and restore all cached connector query keys in connect/disconnect
mutations so optimistic updates stay correct when policy context changes
mid-mutation (brand toggle, permissions resolving).

* style: ruff autofix (auto)

* addressed coderabbit comments

* remove mentions of dev only envs in the env.example

---------

Co-authored-by: Olfa Maslah <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Edwin Jose <[email protected]>

* fix: google to support WEBHOOK_BASE_URL (#1849)

* Update connector.py

* update env in settings

* fix webhook url of sharepoint

* fix: make connector webhooks work end-to-end (Drive routing, Graph subscriptions, change discovery) (#1867)

* fix: alias legacy /connectors/google/webhook to google_drive and reload connections store on webhook channel lookup miss

* Fix Sharepoint and Google Drive

* fix: propagate deleted items from SharePoint/OneDrive webhook delta query

Main's delete-event coverage (#1852) expects handle_webhook to return
deleted file ids so sync_specific_files can run its deleted-at-source
cleanup (get_file_content -> 404 -> delete indexed chunks). The delta
implementation skipped deleted items; now deleted files propagate and
deleted folders stay excluded.

* use logging _config

* Update test_webhook_type_alias.py

* fix: forward end-user JWT to MCP/API callers via gateway-placed X-OpenRAG-API-JWT header (#1874)

* use fallback header

* Notes on new header

* raises http exception for missing jwt

* feat: Add shared flag for COS ingestion to allow documents to be indexed without an owner

* style: ruff autofix (auto)

* Update src/api/connectors.py

Improving error return

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* feat: Implement shared flag for document replacement in OpenSearch queries

* style: ruff autofix (auto)

* refactor: Remove shared flag from document ingestion options in SDK

* feat: Update resolve_shared_owner_fields to return anonymous user details for shared documents

* style: ruff autofix (auto)

* feat: Add anonymous delete permission for shared documents and update related functionality

* feat: Update delete permission to include anonymous access for shared documents

* style: ruff autofix (auto)

* style: apply ruff autofix

* feat: Restrict shared flag usage to ibm_cos connector in sync_connector_files method

* fix: clarify behavior of documents with owner=null in DLS tests

* fix button types (#1875)

* refactor: no derived state (#1886)

* no derivied state

* coderabbit comments

* OPENSEARCH_NODE_COUNT_CHECK_ENABLED as deafult flag (#1975)

What changed:

  - Renamed the env var read by the backend to OPENSEARCH_NODE_COUNT_CHECK_ENABLED in src/config/settings.py.
  - Updated Docker Compose to pass OPENSEARCH_NODE_COUNT_CHECK_ENABLED in docker-compose.yml.
  - Added the canonical flag to the example env file in .env.example.
  - Documented the flag in the OpenSearch config table in docs/docs/reference/configuration.mdx.
  - Updated the existing readiness test wording in tests/unit/test_opensearch_wait_node_count.py.
  - Added a regression test that loads config.settings with OPENSEARCH_NODE_COUNT_CHECK_ENABLED=false and verifies the canonical flag is what the code reads in tests/unit/config/
    test_opensearch_node_count_check_enabled.py.

* chore: Segment improvements (#1971)

* chore: Segment improvements (#1963)

* Update static properties

* and knowledge/setting actions

* avoid duplicate events

* coderabbit comments

* fix: onboarding timeout cleanup issue (#1977)

* fix the cleanup timeout issue for the embedding step

* final step fix

* feat: add shared parameter to connector_sync function

* feat: add shared parameter to TaskProcessor and update connector_sync for permission checks

* fix: improve response handling in delete_document_endpoint for better status reporting

* fix: clear completeTimeoutRef after completion to prevent memory leaks

* fix: update OPENSEARCH_NODE_COUNT_CHECK_ENABLED to use fallback from OPENSEARCH_NODE_COUNT_CHECK

* fix: onboarding timeout cleanup issue (#1977)

* fix the cleanup timeout issue for the embedding step

* final step fix

* fix: support big csv files on langflow-less ingestion (#1968)

* fix max tokens when splitting for langflow-less ingestion

* fix coderabbit picks

* style: ruff autofix (auto)

* added non lf ingestion test

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix: upgrade OpenSearch to 3.6.0 (#1972)

* upgrade OpenSearch to 3.6.0

* fixed PR review comments

* fix nitpicks

* Added ibm watsonx ai and langchain ibm to Langflow image (#1989)

* OPENSEARCH_NODE_COUNT_CHECK_ENABLED as deafult flag (#1975)

What changed:

  - Renamed the env var read by the backend to OPENSEARCH_NODE_COUNT_CHECK_ENABLED in src/config/settings.py.
  - Updated Docker Compose to pass OPENSEARCH_NODE_COUNT_CHECK_ENABLED in docker-compose.yml.
  - Added the canonical flag to the example env file in .env.example.
  - Documented the flag in the OpenSearch config table in docs/docs/reference/configuration.mdx.
  - Updated the existing readiness test wording in tests/unit/test_opensearch_wait_node_count.py.
  - Added a regression test that loads config.settings with OPENSEARCH_NODE_COUNT_CHECK_ENABLED=false and verifies the canonical flag is what the code reads in tests/unit/config/
    test_opensearch_node_count_check_enabled.py.

* fix: onboarding timeout cleanup issue (#1977)

* fix the cleanup timeout issue for the embedding step

* final step fix

* fix: universal file names (#1982)

* ascii support for ingest files

* update the flow json

* style: ruff autofix (auto)

* Update langflow_headers.py

* Update test_langflow_ingest_callback.py

* Update test-ci.yml

* Update test_onboarding_sample_docs.py

* style: ruff autofix (auto)

* Update test_ascii_safe_header_value.py

* Update langflow_file_service.py

* re trigger commit

* fix: purge config_manager singleton in non-langflow ingestion test

The _purge_modules() list was missing "config.config_manager", so the
ConfigManager singleton retained its cached _config from a previous test.
Subsequent calls to get_openrag_config() returned the old config where
disable_ingest_with_langflow=False, causing the router to use the Langflow
path even though DISABLE_INGEST_WITH_LANGFLOW=true was set in the environment.
Adding config.config_manager to the purge list forces a fresh load on the
next import, picking up the correct env-var values.

* fix: use hash_id to derive expected document_id in CSV ingestion test

DocumentFileProcessor ignores Docling's binary_hash field and computes
document_id = hash_id(file_path) from the actual file content. The test
was asserting against the literal mock value "sha-csv-integration-123"
which never appears in the indexed documents, giving 0 hits and a false
failure. Now the test derives expected_document_id via hash_id(csv_path)
to match what the production code actually indexes.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

---------

Co-authored-by: Wallgau <[email protected]>
Co-authored-by: Olfa Maslah <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Edwin Jose <[email protected]>
Co-authored-by: Edwin Jose <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Mike Fortman <[email protected]>
Co-authored-by: Sebastián Estévez <[email protected]>
Co-authored-by: Lucas Oliveira <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend 🔷 Issues related to backend services (OpenSearch, Langflow, APIs) bug 🔴 Something isn't working. lgtm tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants