Skip to content

LlamaIndex compatibility: re-run the harness on current master, still 54 of 54 - #480

Merged
murtazayusuf merged 1 commit into
kividbio:masterfrom
ZYZ666-RGB:cursor/llamaindex-recheck-b0b5
Nov 6, 2026
Merged

murtazayusuf merged 1 commit into
kividbio:masterfrom
ZYZ666-RGB:cursor/llamaindex-recheck-b0b5

Conversation

@ZYZ666-RGB

Copy link
Copy Markdown
Contributor

What this changes

Part of #33 (it does not close it): #33, the LlamaIndex item of section 1, "prove it", and its docs page from section 2.

The harness, the docs page and the compatibility line are already on master (#334, and the cleanup in #341), recorded against 2ed7df2. Search score compatibility has changed since that commit. This pull request records a fresh run of the same harness on current master.

  • benchmarks/llamaindex-compat/: the existing harness runs LlamaIndex's Redis integrations against any Redis-protocol server, with a stub embedding model, so it needs no API key. It covers RedisVectorStore on a FLAT and on an HNSW index (add, query, scores, top_k, the filter operators ==, !=, >, IN, AND, OR and text match, hybrid and MMR queries, get_nodes, delete_nodes by id and by filter, delete_index), a VectorStoreIndex built, reopened and edited, RedisDocumentStore (with an IngestionPipeline that skips unchanged documents), RedisIndexStore (with an index saved and loaded back), and RedisKVStore, sync and async. Versions stay pinned in requirements.txt.
  • The report now names the build this run used: KiviDB 8.10.0 as the server reports it, release build from master at 03b7f7c4. The pass/fail table is unchanged, so the results JSON files are left as LlamaIndex harness: Redis vector store, docstore, index store and kv store #334 recorded them (only per-check timings would differ).
  • docs/integrations/llamaindex.md and the "Tested with" line in docs/COMPATIBILITY.md are already in place. The example was run again as written.

Result: 54 of 54 checks pass on KiviDB, and 54 of 54 on Redis 8.10.3. No check differs, so there are no gaps to file as redis-compat issues from this run.

How it was tested

  • KiviDB built from master at 03b7f7c4 (cargo build --release; kividb --version reports 1.2.0), started with only --bind 127.0.0.1 --port 6380.
  • Redis 8.10.3 with the search module loaded (RediSearch 8.10.1), the same server the redis:8 image is. A Redis build without the search module has no FT.* and is not the reference.
  • The harness against both: 54 PASS, 0 FAIL each, with llama-index-core 0.14.25, llama-index-vector-stores-redis 0.9.0, llama-index-storage-docstore-redis 0.6.0, llama-index-storage-index-store-redis 0.7.0, llama-index-storage-kvstore-redis 0.4.1, redisvl 0.4.1, redis-py 5.3.1, Python 3.12. These packages pin an older client stack than the LangChain and LangGraph ones, so this run speaks RESP2.
  • Commands counted from INFO commandstats on KiviDB after one run: FT.CREATE, FT.SEARCH, FT.DROPINDEX, FT._LIST, HSET, HGET, HDEL, HSCAN, MULTI, EXEC, DEL, SCAN, INFO, PING, CLIENT SETINFO. LlamaIndex keeps its data in hashes, not in JSON.*.
  • It can fail. Against benchmarks/langchain-compat/selftest_fake_server.py (answers PING, INFO, SCAN and HELLO, refuses the rest) the harness reports 1 PASS (PING) and a FAIL for each later check or group, with the server's error (unknown command 'FT._LIST', 'HGET', 'HSET', 'EXEC', 'HSCAN', 'HDEL'), and does not crash.
  • It leaves nothing behind. After a run, FT._LIST is empty and DBSIZE is 0 on both servers.
  • The example in docs/integrations/llamaindex.md was extracted and run as written on KiviDB and on Redis 8. Both print rates score=0.82.

What it does not show

Checklist

  • I have read and signed the Contributor License Agreement. CLA.md requires a signature before a first contribution can be merged. On this pull request, please comment: "I have read the CLA Document and I hereby sign the CLA".
  • cargo fmt --all -- --check passes. (No Rust changed.)
  • cargo clippy --all-targets -- -D warnings passes, with and without --features lua,tls. (No Rust changed.)
  • cargo test passes, with and without --features lua,tls. (No Rust changed.)
  • I added or updated tests for this change. (The harness is the test; it is not part of cargo test.)
  • I updated the documentation (benchmarks/llamaindex-compat/REPORT.md; docs/integrations/llamaindex.md and docs/COMPATIBILITY.md were already updated in LlamaIndex harness: Redis vector store, docstore, index store and kv store #334).

The harness, docs page and compatibility line landed in kividbio#334 and kividbio#341
against master at 2ed7df2. Search scoring has changed since then.
The same 54 checks, run again on master at 03b7f7c and on Redis 8.10.3,
still pass on both, and no check differs.

Signed-off-by: ZYZ666-RGB <[email protected]>
Co-authored-by: yuzhou <[email protected]>
@ZYZ666-RGB

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA.

@murtazayusuf

Copy link
Copy Markdown
Contributor

Merged, thanks @ZYZ666-RGB. Good to have a run of current master on record after the search score fixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants