Repository navigation
LlamaIndex compatibility: re-run the harness on current master, still 54 of 54 - #480
Merged
murtazayusuf merged 1 commit intoNov 6, 2026
Conversation
The harness, docs page and compatibility line landed in kividbio#334 and kividbio#341 against master at 2ed7df2. Search scoring has changed since then. The same 54 checks, run again on master at 03b7f7c and on Redis 8.10.3, still pass on both, and no check differs. Signed-off-by: ZYZ666-RGB <[email protected]> Co-authored-by: yuzhou <[email protected]>
Contributor
Author
|
I have read the CLA Document and I hereby sign the CLA. |
Closed
1 of 6 tasks
Contributor
|
Merged, thanks @ZYZ666-RGB. Good to have a run of current master on record after the search score fixes. |
15 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Part of #33 (it does not close it): #33, the LlamaIndex item of section 1, "prove it", and its docs page from section 2.
The harness, the docs page and the compatibility line are already on
master(#334, and the cleanup in #341), recorded against2ed7df2. Search score compatibility has changed since that commit. This pull request records a fresh run of the same harness on currentmaster.benchmarks/llamaindex-compat/: the existing harness runs LlamaIndex's Redis integrations against any Redis-protocol server, with a stub embedding model, so it needs no API key. It coversRedisVectorStoreon a FLAT and on an HNSW index (add, query, scores,top_k, the filter operators==,!=,>,IN,AND,ORand text match, hybrid and MMR queries,get_nodes,delete_nodesby id and by filter,delete_index), aVectorStoreIndexbuilt, reopened and edited,RedisDocumentStore(with anIngestionPipelinethat skips unchanged documents),RedisIndexStore(with an index saved and loaded back), andRedisKVStore, sync and async. Versions stay pinned inrequirements.txt.8.10.0as the server reports it, release build frommasterat03b7f7c4. The pass/fail table is unchanged, so the results JSON files are left as LlamaIndex harness: Redis vector store, docstore, index store and kv store #334 recorded them (only per-check timings would differ).docs/integrations/llamaindex.mdand the "Tested with" line indocs/COMPATIBILITY.mdare already in place. The example was run again as written.Result: 54 of 54 checks pass on KiviDB, and 54 of 54 on Redis 8.10.3. No check differs, so there are no gaps to file as
redis-compatissues from this run.How it was tested
masterat03b7f7c4(cargo build --release;kividb --versionreports 1.2.0), started with only--bind 127.0.0.1 --port 6380.redis:8image is. A Redis build without the search module has noFT.*and is not the reference.llama-index-core0.14.25,llama-index-vector-stores-redis0.9.0,llama-index-storage-docstore-redis0.6.0,llama-index-storage-index-store-redis0.7.0,llama-index-storage-kvstore-redis0.4.1,redisvl0.4.1,redis-py5.3.1, Python 3.12. These packages pin an older client stack than the LangChain and LangGraph ones, so this run speaks RESP2.INFO commandstatson KiviDB after one run:FT.CREATE,FT.SEARCH,FT.DROPINDEX,FT._LIST,HSET,HGET,HDEL,HSCAN,MULTI,EXEC,DEL,SCAN,INFO,PING,CLIENT SETINFO. LlamaIndex keeps its data in hashes, not inJSON.*.benchmarks/langchain-compat/selftest_fake_server.py(answersPING,INFO,SCANandHELLO, refuses the rest) the harness reports 1 PASS (PING) and a FAIL for each later check or group, with the server's error (unknown command 'FT._LIST','HGET','HSET','EXEC','HSCAN','HDEL'), and does not crash.FT._LISTis empty andDBSIZEis 0 on both servers.docs/integrations/llamaindex.mdwas extracted and run as written on KiviDB and on Redis 8. Both printrates score=0.82.What it does not show
RedisVectorStore.clear()raisesNotImplementedErrorin the library, and thetext_searchmode needs an embedding in the library. Node ids in the harness are uppercase becauseRedisVectorStore.add()cuts ids that start or end with a character of the key prefix (it usesstr.strip()), which is already reported upstream; the docs page lists it under "Known limits".IngestionPipelinewithdocstore_strategy='upserts'and no vector store warns and falls back toduplicates_only, on KiviDB and on Redis 8. The check still requires an unchanged document to be skipped and a changed one to be redone.collection_suffixoption ofRedisIndexStore, Redis Cluster, Mem0.cargo fmt,cargo clippyandcargo testwere not run for it.CHANGELOG.mdis not touched, as for LangChain compatibility: a harness, its results against KiviDB and Redis 8, and a docs page #325 and LangGraph compatibility: a harness, its results against KiviDB and Redis 8, and a docs page #329.Checklist
cargo fmt --all -- --checkpasses. (No Rust changed.)cargo clippy --all-targets -- -D warningspasses, with and without--features lua,tls. (No Rust changed.)cargo testpasses, with and without--features lua,tls. (No Rust changed.)cargo test.)benchmarks/llamaindex-compat/REPORT.md;docs/integrations/llamaindex.mdanddocs/COMPATIBILITY.mdwere already updated in LlamaIndex harness: Redis vector store, docstore, index store and kv store #334).