fix(redis): scope RedisSemanticCache entries by llm_string - #220

Fixes #216.

limjoobin this has the migration utility you asked for on the issue: explicit, cold by default, with an optional backfill.

What

RedisSemanticCache passed llm_string to RedisVL only as metadata, which isn't part of the entry key. The same prompt cached for two llm_strings landed on one key and the second write replaced the first. lookup() also fetched a single nearest hit and compared llm_string afterwards, so a closer entry from another model hid a valid one.

Change

Entries now carry an llm_string_hash tag field (an MD5, since raw llm_strings contain tag separators). update() passes it as filters, which also puts it in the key, and lookup() filters on it in the search. Same for the async methods.

An index created by an earlier version lacks that field, so the constructor now raises a ValueError that points to migrate_index(). Any other schema mismatch raises the same error as before.

RedisSemanticCache.migrate_index(
    redis_url="redis://localhost:6379",
    name="llmcache",
    backfill=False,  # True also moves existing entries to their new keys
)

It's a classmethod because the constructor raises on a legacy index, and it doesn't need embeddings: backfill reuses the stored vectors.

Index upgrade: I used FT.ALTER ... SKIPINITIALSCAN rather than the overwrite=True I suggested on the issue. Keys are untouched, there's no moment without an index, and nothing gets re-indexed (legacy entries don't have the field anyway). A plain FT.ALTER re-indexes every document in the background; with 20k entries, max_doc_id went from 20000 to 40000.

Backfill: for each legacy entry, all fields are copied as raw bytes to the key the new write path uses, with llm_string_hash added and entry_id updated. The write and the PEXPIREAT of the original deadline share a MULTI, so a replacement never exists without its expiration. Entries with no expiration keep none. The source is deleted only after that.

To rerun it safely, tagged entries are skipped. If the replacement already exists (interrupted run or newer write), it's kept and only the source is removed.

A few things you didn't specify, so please check them:

• Entries whose prompt or llm_string can't be read stay in place, since their new key can't be computed. They're logged and counted in the returned {"backfilled": n, "skipped": m}.
• Backfill ignores keys that match the pattern but belong to another cache (foo:tenant:<id> when migrating foo, which the name + prefix mapping can produce).
• There's no async variant, since this is a one-off operation.

Testing

• New unit and integration tests in test_semantic_cache_llm_string.py. The migration ones cover cold start, rerun, field and deadline preservation, decode_responses clients, unreadable entries, and failures both before the write and before the source delete.
• Unit and integration suites pass on Redis 8.4 with redisvl 0.28.0. The semantic cache tests also pass on Redis 8.2, and with redisvl 0.27.1 and redis 7.1.0 (the minimums in pyproject.toml).
• I also migrated two indexes written by the code on main, 5,000 entries each. Every entry was a hit after backfill, and all deadlines were unchanged.
• Not tested on Redis Cluster.
• test_chat_message_history.py fails the same way on unmodified main in my environment ('dict' object has no attribute 'index' when parsing FT.INFO), so I left it out of the runs above.
• ruff check, mypy and codespell pass. ruff format --check . only flags README.md, as it does on main.

Release note

RedisSemanticCache now keeps entries separate per llm_string, so different models or parameter sets no longer overwrite each other's entries or hide each other's hits.

Action required for existing caches: an index created by an earlier version now raises a ValueError. Stop the applications using the cache, run RedisSemanticCache.migrate_index(), then deploy the new version. Entries cached before the upgrade are no longer returned unless you pass backfill=True. Old and new versions can't share a migrated index.