Repository navigation
fix(rag): stop indexing on a broken vision model and name it - #7154
Conversation
A vision model that was installed but crashing turned a scanned PDF into blank pages: every page waited out the failure in turn, and the run ended with a hint to download the model that was already there. Indexing a PDF or PowerPoint now stops once the model has failed twice in a row without reading any image, with an error naming the model and its last error. index_document reports it as error_type "vision_model_failed", and the agent tool says the model failed, not the file. Closes amd#4741
|
Verdict: Approve with suggestions If the vision model is installed but crashes, indexing a scanned PDF or PowerPoint now stops after two images fail with nothing read. Before, it waited out the failure on every page and ended with advice to download a model that was already installed. The error now names the model and its last error, and the agent tells the user the model failed, not the file. Nothing blocks the merge. The two suggestions below are small.
Real-world evidenceN/A: this runner produced no evidence bundle. The PR description has a before/after run of the SDK on a 10-page scan. On 🔍 Technical detailsIssues 🟢 Indexing a folder asks the broken model twice per scanned file ( 🟢 Error code documented under the wrong field ( Notes, not findings
Strengths
|
A page whose second image failed counted as the vision model reading nothing, though it read the images before it, so a model that works but fails on some images could stop indexing as broken. The tools spec now lists vision_model_failed under error_type.
|
Applied two of the suggestions in ee13333:
I left folder indexing as it is. Stopping the folder on this error would also skip its text files and text-heavy PDFs, which index fine without the vision model, so the user would lose documents that work to save a few seconds per scan. 🔍 Technical details
|
|
The one failing check, GAIA CLI on Linux, isn't from this change: 🔍 Runs and one gap in the outage check
The step reports this as "add-apt-repository failed … for a reason other than a Launchpad outage". |
|
@kovtcharov-amd this is ready for your review. The bot's suggestions are applied in ee13333. The one red check, GAIA CLI on Linux, is the Launchpad outage that is also failing |
Summary
A vision model that is installed but crashing no longer turns a scanned PDF into blank pages. Indexing stops after the model has failed twice in a row without reading any image, and the error names the model and its last error. Before, every page waited out the failure in turn (minutes for a 10-page scan), and the run ended with a hint to download the model that was already there.
Why
The issue's Strix Halo box reset the Qwen3-VL backend on every image. Each page's failure became a page warning, so the document indexed as text-only, or failed as "no extractable text, enable VLM image extraction by downloading …", which sends the user after the wrong fix. The bot's suggested fix on the issue (stop once the model has failed with nothing read, and name it) is what this does.
Linked issue
Closes #4741
Changes
degraded_pages, as before.index_documentreturnserror_type: "vision_model_failed", and the agent tool's hint says the model failed, not the file, instead of "suggest they check the file".Test plan
Run in a
python:3.12container:pytest tests/unit/rag/test_failing_vision_model.py: 7 passed. It indexes real reportlab PDFs and python-pptx decks with a vision model that fails, works, or fails after working.pytest tests/unit/test_rag_file_resolution.py::TestIndexDocumentVisionModelFailed: passes; the tool keeps the error and doesn't blame the file.pytest tests/unit/rag tests/unit/test_vlm_extraction_failures.py tests/unit/test_rag_file_resolution.py tests/unit/test_vision_uses_downloaded_model.py tests/test_rag.py: 909 passed, 35 skipped.pytest tests/unit: 38,761 passed, 590 skipped, 15 failed, of 39,270 collected, with no crashed workers. The 15 fail the same way onmainin this container: 14 shell-guardrail and skill-grant tests refuseghandnpmbecause neither is installed there, and one expects a folder it can't create, but the container runs as root. 44 tests have::inside their parameters, so they were rerun by file to account for them.python util/lint.py --all: passes.The agent tool's reply for this failure gains a hint, so
rag_qualityis the eval category to watch on the next RC.Evidence
The same 10-page scan, indexed with a vision model that fails every image with
backend_watchdog_reset:Checklist
Closes #N/Fixes #N/Refs #N).python util/lint.py --all,pytest tests/unit/).