Repository navigation
fix: pass the caller's limit to redisvl in RedisDb.keyword_search - #10781
Conversation
keyword_search accepted a limit and never used it. The TextQuery it builds was missing num_results, so redisvl applied its own default of 10 and every keyword search returned at most 10 documents regardless of the request. vector_search and hybrid_search in the same file, with the same signature, already pass num_results=limit.
VANDRANKI
left a comment
There was a problem hiding this comment.
Community review, not a maintainer review and it does not clear the merge gate. I read the full diff and ran it on Windows with Python 3.13.3, redisvl 0.28.0 and redis-py 8.1.0, against main at 1f2aac5. I did not run it against a live Redis server.
I ran the bug on main and the fix on the PR, and I think the change is correct.
What I ran:
- With the PR's
tests/unit/vectordb/test_redisdb.pyon main's source,test_keyword_search_passes_the_limit_to_redisvlfails and the other 11 tests pass. On the PR branch all 12 pass. - Built the redisvl
TextQuerythe waykeyword_searchdoes, with and withoutnum_results. Without it the generated arguments end inLIMIT 0 10, withnum_results=3they end inLIMIT 0 3. So on main,keyword_searchalways asked Redis for 10 results, whateverlimitwas. The vector and hybrid paths already passnum_results=limit, andkeyword_searchis the onlyTextQuery(call site inredisdb.py.
One visible effect: RedisDB.search defaults to limit=5, so on main a default keyword search returned up to 10 documents, and with the fix it returns at most 5. That is the documented behavior, but anyone relying on the accidental 10 will see fewer results. limit values above 10 now work as well.
I did not check what convert_bytes or Document.from_dict do with the query results, since the PR does not touch them.
|
Thanks for taking it this far. Rebuilding the You are right about the visible effect, and it is worth saying out loud for whoever merges this. All four methods on this class declare the same default: The breakage that is actually reachable runs the other way: anything above 10 was silently capped. Ask for 50, get 10, no exception and no log. Happy to add a changelog line about the default keyword search returning 5 if you would rather surface it to users. |
|
Thanks @Ayoubhm07 for your contribution! |
RedisDb.keyword_searchtakes alimitand never uses it : theTextQueryit builds omitsnum_resultsso redisvl applies its own default of 10It went unnoticed because two defaults collide :
Knowledge.max_resultsis 10 atknowledge/knowledge.py:92andTextQuery.num_resultsis 10 as wellOn the stock path you ask for 10 and get 10 : the right answer from the wrong mechanism
Move
max_resultsand it breaks both ways with no exception and no log : ask for 50 get 10 — ask for 3 get 10Nothing downstream compensates and the comment at
knowledge/knowledge.py:245-247says so :vector_searchatredisdb.py:662andhybrid_searchat:731share the same signature and both passnum_results=limitEvery query in the file passes it except this one : even the bulk reads annotate why they want everything with
num_results=1000, # Get all matching documentsRedis is also the only one of the 8
keyword_searchimplementations undervectordb/that drops itThe fix
text_query = TextQuery( text=query, text_field_name="content", + num_results=limit, filter_expression=filter_expression, )Proof
test_keyword_search_passes_the_limit_to_redisvladded besidetest_search_vector_keyword_hybridBefore :
After :
The existing
test_search_vector_keyword_hybridcould not have caught this : it setsidx.query.return_valueitself solen(docs) == 1holds because the mock was told to return one row and not becauselimit=1reached redisvlThe new test asserts on
idx.query.call_args: what the adapter asked for rather than what the fake returnedOne limitation worth naming : the test reads
_num_resultswhich is privateredisvl 0.27.2 exposes no public accessor and the only alternative is asserting on a mocked
len()— the exact assertion that let this throughSearched the open pull requests : the two touching
vectordb/are #10545 and #10689 and neither touchesredisdb.pyDisclosure per
CONTRIBUTING.mdpoint 5 : this change was produced with my claudeEvery line number above and the redisvl default were read off the source rather than assumed