feat(file-processors): add async/fallback support to docling-serve by sahana-sreeram · Pull Request #6014 · ogx-ai/ogx

sahana-sreeram · 2026-06-03T16:57:38Z

What does this PR do?

Integrates AsyncDoclingServiceClient from docling-slim to enable async endpoints for both IBM Docling SaaS and local docling-serve. This enables IBM Docling SaaS and improves performance for local deployments.

Problem: Before this PR, docling-serve provider had critical limitations:

IBM Docling SaaS was completely incompatible (it only provides async endpoints, OGX docling-serve only used sync)
Local deployments experienced timeouts on large PDFs under concurrent load
No async capabilities or WebSocket-based status updates

Solution:

Use Docling's new AsyncDoclingServiceClient for async conversion with automatic fallback to sync
Handle dual response formats: presigned S3 URLs (IBM SaaS) and direct documents (local Docling-Serve)
Error handling approach for IBM SaaS' chunking limitation: catch HTTP 404/405 errors and throw clear InvalidParameterError

This solution is lightweight and requires no infrastructure changes. Users can:

Continue using local docling-serve with async mode (now the default)
Switch to IBM Docling SaaS by updating base_url and api_key in their config

Test Plan

Validated async functionality using both unit tests and integration testing against a local Docling Serve instance.

Unit Tests

23 passed in 1.49s

  Unit Tests
  New tests added:
  tests/unit/providers/file_processor/test_docling_serve.py::TestIBMSaaSCompatibility::test_ibm_saas_blocks_chunking_with_clear_error PASSED [ 91%]
  tests/unit/providers/file_processor/test_docling_serve.py::TestIBMSaaSCompatibility::test_ibm_saas_allows_conversion_without_chunking PASSED [ 95%]
  tests/unit/providers/file_processor/test_docling_serve.py::TestIBMSaaSCompatibility::test_local_docker_allows_chunking PASSED [100%]
  
  ======================== 23 passed, 1 warning in 1.49s =========================

Integration Testing

Tested against local docling-serve Docker (v1.24.0) and IBM's Docling SaaS (both in async mode) with real 37KB PDF (ogx_and_models.pdf):

Test 1: IBM Docling SaaS (Async Conversion)
  {
    "endpoint": "IBM SaaS",
    "base_url": "https://api.aws-c1.dcls.saas.ibm.com/...",
    "chunks_count": 1,
    "conversion_method": "async",
    "processing_time_ms": 934,
    "total_chars": 1486,
    "chunk_sizes": [1486]
  } 
  
  Test 2: Local docling-serve (Async Conversion + Chunking)
  {
    "endpoint": "Local Docker",
    "base_url": "http://localhost:5001",
    "chunks_count": 3,
    "conversion_method": "async",
    "processing_time_ms": 5072,
    "total_chars": 1457,
    "chunk_sizes": [792, 541, 120]
  }

Conversion works as expected for IBM SaaS; Local Docling-Serve can run both convert and chunk methods. Confirms API endpoints are hit correctly.

Test Setup

Docling Serve: Local instance via Docker (quay.io/docling-project/docling-serve)
IBM SaaS Docling API key and base url configured
Test file: ogx_and_models.pdf (86 pages, 63MB, from integration test fixtures)

Docling team confirmed async endpoints work on both local and SaaS. Default to async mode with automatic fallback to sync on failure. This eliminates timeout issues under load and enables compatibility with IBM Docling SaaS async-only endpoints. - Added async submit/poll/result workflow methods - Implemented automatic fallback from async to sync - Added unit tests for async mode and fallback behavior Will need rebase after ogx-ai#5627 merges. Signed-off-by: Sahana Sreeram <[email protected]>

…g-serve using docling-slim's new asyncio capabilities. Adds support for async endpoints using AsyncDoclingServiceClient with automatic fallback to sync. Updates base_url to remove /v1 suffix (SDK adds it internally). Adds unit tests for async mode. Signed-off-by: Sahana Sreeram <[email protected]>

Fix api_key type (str not Optional), remove unsupported chunking_max_tokens, fix ChunkedDocumentResultItem handling (object not dict). Signed-off-by: Sahana Sreeram <[email protected]>

Signed-off-by: Sahana Sreeram <[email protected]>

mergify · 2026-06-17T11:05:14Z

This pull request has merge conflicts that must be resolved before it can be merged. @sahana-sreeram please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

…g-serve Integrate AsyncDoclingServiceClient to support async endpoints for both IBM Docling SaaS and local docling-serve. - Use AsyncDoclingServiceClient for async conversion - Handle dual response formats (presigned URLs for SaaS, direct documents for local) - Block chunking for IBM SaaS with clear error (not supported) - Require docling-slim[service-client]>=2.103.0 This enables IBM Docling SaaS, which only provides async endpoints. Signed-off-by: Sahana Sreeram <[email protected]>

…M SaaS chunking Fixed URL matching by catching HTTP 404/405 responses instead of checking for 'ibm.com' in URL. This resolves the CodeQL security warning and makes the code self-correcting if IBM adds chunking support later. Signed-off-by: Sahana Sreeram <[email protected]>

alinaryan · 2026-06-24T17:25:51Z

+    mode: str = Field(
+        default="async",
+        description=(
+            "API mode: 'async' (use asynchronous submit/poll endpoints, recommended for both local and SaaS), "
+            "'sync' (use synchronous endpoints, fallback option), or 'auto' (detect server capabilities)"
+        ),
+    )


Use Literal["async", "sync", "auto"] to get Pydantic validation at config load time

alinaryan · 2026-06-24T17:36:04Z

+                    sdk_module="docling.service_client",
+                )
+                conversion_method = "async"
+            except Exception as e:


This catches all exceptions including InvalidParameterError, which _convert_and_chunk_async raises when IBM SaaS returns 404/405 for unsupported chunking. That user-facing error gets silently swallowed, a redundant sync request hits the same 404/405, and only then does the error reach the caller.

Also, mode='async' and mode='auto' behave identically here — both fall back to sync on any failure. If a user sets mode='async', they probably don't expect a silent sync fallback.

Consider narrowing the catch and only falling back in auto mode:

except (httpx.ConnectError, httpx.TimeoutException) as e: if self.config.mode == "auto": log.warning("Async failed, falling back to sync", error=str(e)) chunks = None else: raise

Fixed in 0d54af1, now using your exact code suggestion which:

Catches network errors (httpx.ConnectError, httpx.TimeoutException)

In auto mode: falls back to sync on network errors. In async mode: raises immediately

InvalidParameterError raised up immediately (without sync retry)

Add Literal type validation, narrow exception handling to network errors only, and only fallback in auto mode. Signed-off-by: Sahana Sreeram <[email protected]>

Signed-off-by: Sahana Sreeram <[email protected]>

mattf · 2026-06-25T11:50:01Z

@alinaryan pls ping me when you've approved this

alinaryan

LGTM. I have one nit, but non-blocking

alinaryan · 2026-06-25T14:48:50Z

+        )
+
+    @pytest.fixture
+    def config_async(self) -> DoclingServeFileProcessorConfig:


test_fallback_from_async_to_sync takes config_async in its signature but creates its own config_auto inside the body. The fixture is instantiated but never used.

fixed (2c222f6) - the parameter was left over from earlier testing. Also renamed the test to test_auto_mode_falls_back_to_sync to be more representative of its behavior after review code fixes

Signed-off-by: Sahana Sreeram <[email protected]>

…oclingServiceClient Signed-off-by: Sahana Sreeram <[email protected]>

sahana-sreeram force-pushed the feat/docling-serve-asynch branch from e4c4cdd to c39f590 Compare June 16, 2026 19:53

sahana-sreeram added 4 commits June 16, 2026 15:57

fix deps. add docling-slim to unit test deps

2dce4f2

chore: trigger GitHub conflict check refresh

2c26574

sahana-sreeram force-pushed the feat/docling-serve-asynch branch from cafe291 to 2c26574 Compare June 16, 2026 19:58

sahana-sreeram added 2 commits June 16, 2026 16:10

fix(mypy): resolve AsyncDoclingServiceClient type errors

8403ebf

Fix api_key type (str not Optional), remove unsupported chunking_max_tokens, fix ChunkedDocumentResultItem handling (object not dict). Signed-off-by: Sahana Sreeram <[email protected]>

fix(ruff): remove unused max_tokens variable

762f552

Signed-off-by: Sahana Sreeram <[email protected]>

mergify Bot added the needs-rebase label Jun 17, 2026

sahana-sreeram added 2 commits June 17, 2026 15:16

Merge upstream/main into feat/docling-serve-asynch

a719231

github-advanced-security AI found potential problems Jun 17, 2026

View reviewed changes

Comment thread src/ogx/providers/remote/file_processor/docling_serve/docling_serve.py Fixed

mergify Bot removed the needs-rebase label Jun 17, 2026

sahana-sreeram and others added 3 commits June 17, 2026 16:19

Merge branch 'main' into feat/docling-serve-asynch

1c157da

Merge branch 'main' into feat/docling-serve-asynch

d423101

sahana-sreeram marked this pull request as ready for review June 23, 2026 18:52

sahana-sreeram requested review from bbrowning, cdoern, franciscojavierarceo, leseb, mattf, raghotham and skamenan7 as code owners June 23, 2026 18:52

alinaryan reviewed Jun 24, 2026

View reviewed changes

sahana-sreeram and others added 3 commits June 24, 2026 16:49

refactor(file-processors): address review feedback for docling-serve

0d54af1

Add Literal type validation, narrow exception handling to network errors only, and only fallback in auto mode. Signed-off-by: Sahana Sreeram <[email protected]>

Merge branch 'main' into feat/docling-serve-asynch

6d1a3cc

chore: trigger CI re-run

40b63c4

Signed-off-by: Sahana Sreeram <[email protected]>

sahana-sreeram requested a review from alinaryan June 24, 2026 21:15

alinaryan approved these changes Jun 25, 2026

View reviewed changes

sahana-sreeram and others added 3 commits June 25, 2026 11:17

rename test and remove unused parameter

2c222f6

Signed-off-by: Sahana Sreeram <[email protected]>

Merge branch 'main' into feat/docling-serve-asynch

9adb2e3

Merge branch 'main' into feat/docling-serve-asynch

ac5de92

mattf requested changes Jun 26, 2026

View reviewed changes

Comment thread src/ogx/providers/remote/file_processor/docling_serve/docling_serve.py Outdated

Comment thread src/ogx/providers/remote/file_processor/docling_serve/config.py

sahana-sreeram added 2 commits June 29, 2026 09:56

tempfile logic to auto-delete instead of manual cleanup

ab1ede2

Signed-off-by: Sahana Sreeram <[email protected]>

strip /v1 suffix from base_url for backward compatibility with AsyncD…

3f1f160

…oclingServiceClient Signed-off-by: Sahana Sreeram <[email protected]>

sahana-sreeram requested a review from mattf June 29, 2026 15:01

mattf approved these changes Jun 29, 2026

View reviewed changes

mattf added this pull request to the merge queue Jun 29, 2026

mattf removed this pull request from the merge queue due to a manual request Jun 29, 2026

Merge branch 'main' into feat/docling-serve-asynch

71d2cd2

mattf enabled auto-merge June 29, 2026 17:36

mattf added this pull request to the merge queue Jun 29, 2026

github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Jun 29, 2026

mattf added this pull request to the merge queue Jun 29, 2026

Merged via the queue into ogx-ai:main with commit 5a21c2a Jun 29, 2026
63 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(file-processors): add async/fallback support to docling-serve#6014

feat(file-processors): add async/fallback support to docling-serve#6014
mattf merged 20 commits into
ogx-ai:mainfrom
sahana-sreeram:feat/docling-serve-asynch

sahana-sreeram commented Jun 3, 2026 •

edited

Loading

Uh oh!

mergify Bot commented Jun 17, 2026

Uh oh!

Uh oh!

alinaryan Jun 24, 2026

Uh oh!

alinaryan Jun 24, 2026

Uh oh!

sahana-sreeram Jun 24, 2026 •

edited

Loading

Uh oh!

Uh oh!

mattf commented Jun 25, 2026

Uh oh!

alinaryan left a comment

Uh oh!

alinaryan Jun 25, 2026

Uh oh!

sahana-sreeram Jun 25, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

sahana-sreeram commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Test Plan

Unit Tests

Integration Testing

Uh oh!

mergify Bot commented Jun 17, 2026

Uh oh!

Uh oh!

alinaryan Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

alinaryan Jun 24, 2026

Choose a reason for hiding this comment

Uh oh!

sahana-sreeram Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mattf commented Jun 25, 2026

Uh oh!

alinaryan left a comment

Choose a reason for hiding this comment

Uh oh!

alinaryan Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

sahana-sreeram Jun 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sahana-sreeram commented Jun 3, 2026 •

edited

Loading

sahana-sreeram Jun 24, 2026 •

edited

Loading