Repository navigation
Conversation
`metadata_filters_to_sql` rendered GT/GTE/LT/LTE comparisons against `metadata ->> 'key'`, which is text. A numeric filter value produced `text > integer`, raising `UndefinedFunction` on PostgreSQL, while a string value compared lexically. Compare int/float filter values as floats behind a `jsonb_typeof(metadata -> 'key') = 'number'` guard so rows with a missing or non-numeric value are excluded instead of raising. String filter values keep the existing text comparison. Adds unit tests that render the SQL and bumps the package version to 0.3.1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Summary
Fix
metadata_filters_to_sqlin the Azure PostgreSQL vector store so thatGT,GTE,LT, andLTEfilters with anintorfloatvalue comparemetadata values as numbers instead of text. String filter values keep the
existing text comparison.
This is a direct bug fix with a local reproduction; there is no linked issue.
Problem
metadata_filters_to_sqlbuilds the column reference asmetadata ->> 'key',which is
text. ForGT/GTE/LT/LTEit compared that text directly withsql.Literal(value):psycopg.errors.UndefinedFunction: operator does not exist: text > integer.A float value raises the same error with
text > numeric.'30' > '9'is falseeven though 30 > 9 numerically, and numeric metadata cannot be ordered
correctly.
Reproduction (PostgreSQL 16.15, table
(id int, metadata jsonb)containing{"price": 30},{"price": 9},{"price": "unknown"},{},{"price": null},{"price": 30.5},{"price": 100}, and{"price": [1, 2]}):Change
For
GT/GTE/LT/LTEonly, anintorfloatfilter value (excludingbool) now renders a numeric comparison guarded by the stored JSON type:Rows where the key is missing,
null, a string, an array, or an object donot match the guard and are skipped without a type error.
String values keep the existing text comparison, unchanged:
(metadata ->> 'price') > '9'.EQ,NE,IN,NIN,CONTAINS,TEXT_MATCH,TEXT_MATCH_INSENSITIVE, andIS_EMPTYare untouched.Added
tests/llama_index/test_metadata_filters_sql.py, which renders thegenerated SQL and needs no database.
Bumped the package version from 0.3.0 to 0.3.1.
Verification
Environment: macOS, PostgreSQL 16.15 (Homebrew), Python 3.14, psycopg 3.3.4.
New unit tests:
The same test module against the unmodified baseline (619d746) fails:
Local PostgreSQL probe executing the generated SQL against the table above:
The rows with
{"price": "unknown"}, a missingprice,null, and anarray were present for every case and never raised. On the unmodified
baseline, the five numeric cases raise
UndefinedFunction.Full package suite:
The 7 errors are a pre-existing teardown problem in
tests/conftest.py(
fixture function has more than one 'yield') and are unrelated to thischange; the same error occurs on the unmodified baseline (4 errors there,
the count varying with connection-pool/test order). DB-backed vector store
tests were skipped because Azure AD credentials and the
vectorextensionare not available in this local environment. Package coverage was therefore
not measured locally; the new unit tests exercise every changed branch.
Lint:
New Package?
Version Bump?
Type of Change
How Has This Been Tested?
Suggested Checklist:
AI Assistance
AI assistance was used to draft this change and its tests. The reproduction,
test runs, and results reported above were executed locally.