Skip to content

fix(analyticdb): emit native jsonb filter syntax instead of jsonpath - #300

Open
Lubaoshuai wants to merge 1 commit into
spring-ai-alibaba:mainfrom
Lubaoshuai:fix/analyticdb-jsonb-filter
Open

Lubaoshuai wants to merge 1 commit into
spring-ai-alibaba:mainfrom
Lubaoshuai:fix/analyticdb-jsonb-filter

Conversation

@Lubaoshuai

Copy link
Copy Markdown

Fixes alibaba/spring-ai-alibaba#4962

Describe what this PR does / why we need it

AdVectorFilterExpressionConverter rendered the filter as jsonpath syntax ($.key, &&/||, ==, !()), with double-quoted string values. QueryCollectionData and DeleteCollectionData splice that string straight into a PostgreSQL WHERE clause, and ADB PG FastANN vector search rejects it, so any filtered similarity search fails with a SQL error.

This PR emits native syntax instead:

  • keys as metadata->>'key' jsonb text extractions
  • numeric comparisons as (metadata->>'key')::numeric, since PostgreSQL defines no comparison operators between text and numeric types
  • AND / OR keywords, = comparisons, and NOT (...) for NIN
  • single-quoted string literals with '' escaping (double quotes are identifiers in SQL, so they were equally invalid)
  • IN still expands to an OR chain of equalities, preserving the previous semantics

Does this pull request fix one issue?

Fixes alibaba/spring-ai-alibaba#4962

Testing

New AdVectorFilterExpressionConverterTest (12 cases, no real ADB credentials needed): equality/comparison, AND/OR, group nesting, IN/NIN, numeric casts, boolean values and quote escaping. mvn test passes (12 green), spring-javaformat:apply and spotless:check are clean.

The filter expression was rendered with jsonpath syntax ($.key, &&, ||,
==, !()) and double-quoted string values, but QueryCollectionData and
DeleteCollectionData splice it straight into a PostgreSQL WHERE clause,
where FastANN vector search rejects it (alibaba/spring-ai-alibaba#4962).

Emit native syntax instead: metadata->>'key' text extractions, AND/OR
keywords, = comparisons, NOT (...) for NIN, and single-quoted string
literals with '' escaping. Comparisons against numeric operands cast the
extracted text with ::numeric, since PostgreSQL defines no operators
between text and numeric types.
@Lubaoshuai

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@Lubaoshuai

Copy link
Copy Markdown
Author

Heads-up on the failing CLA check: it looks like an infrastructure issue rather than anything about this change. The workflow run errored with "Could not retrieve repository contents. Status: 401" when accessing the signatures repository, which suggests the CLA_REPO_ACCESS_TOKEN secret may have expired or lost access. I have already posted the sign comment, so once the token is fixed a "recheck" should turn this green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] AnalyticDB PG vector store AdVectorFilterExpressionConverter hardcode $. jsonpath, ADB PG FastANN vector search does not support jsonpath filter

1 participant