Skip to content

perf(core): parallelize recursive retrieval using run_jobs - #23127

Open
veerjain-1 wants to merge 2 commits into
run-llama:mainfrom
veerjain-1:perf/parallel-recursive-retrieval
Open

veerjain-1 wants to merge 2 commits into
run-llama:mainfrom
veerjain-1:perf/parallel-recursive-retrieval

Conversation

@veerjain-1

@veerjain-1 veerjain-1 commented Sep 18, 2026 •

Copy link
Copy Markdown

Change

Resolve the recursive-retrieval TODO in BaseRetriever by scheduling independent async object lookups through the existing bounded run_jobs helper. Flatten results in input order, then preserve the existing first-node-ID deduplication behavior.

Add three regression tests through the public aretrieve API:

  • Event-coordinated child retrievers must overlap even when the second finishes first; returned nodes retain input order.
  • Direct and unresolved index nodes pass through, and duplicate node IDs keep the first result and score.
  • Nested retrieval errors propagate to the caller.

The follow-up adds tests only; it does not change the production implementation further.

Validation

  • uv run --no-sync pytest tests/retrievers -q: 8 passed on the updated feature branch.
  • Counterfactual check: the new concurrency/order test fails against the original sequential BaseRetriever; the other two compatibility tests pass (1 failed, 2 passed).
  • Ruff, Ruff formatting, Black with Python 3.10 target, and git diff --check pass.
  • Python-applicable repository pre-commit hooks pass when invoked individually. Full pre-commit bootstrap is not claimed as passed: the unrelated legacy Prettier/nodeenv setup fails parsing an upstream Node prerelease version (0-alpha). Repository hook configuration was not modified or disabled.

Tests use static/mock retrievers and asyncio events, with no live model calls. The five-second timeout only bounds a broken sequential run; no throughput benchmark or full-core-suite result is claimed.

Upstream workflow authorization (November 11): the current-head Unit Testing, Core Typecheck, Build Package, Linting, and CodeQL workflow runs are awaiting maintainer approval (action_required). For example: https://github.com/run-llama/llama_index/actions/runs/42699760985. No upstream CI pass is claimed.

AI assistance: the follow-up regression tests were prepared with Codex assistance and validated with the checks above. Upstream review and merge remain with the maintainers.

@veerjain-1

Copy link
Copy Markdown
Author

Hi there! Just a gentle ping to check in on this. Let me know if there's anything else needed from my side to move this forward!

Copy link
Copy Markdown
Author

Hi maintainers, could someone authorize the current-head CI runs and review this change?

Head f80c99a includes the concurrency/order, pass-through/deduplication, and error-propagation regression tests added on November 9. The retriever suite passed locally (8 tests), and the new concurrency test fails against the original sequential implementation. The PR body records the validation and limitations, including the unrelated pre-commit bootstrap problem.

Unit Testing, Core Typecheck, Build Package, Linting, and CodeQL are still action_required, so I cannot check the upstream results yet. I can address any failures or review feedback once those run. The tests use static retrievers and make no live model calls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant