Skip to content

Commit 555c46b

Browse files
authored
Python: Stop doubling apostrophes in Chroma and Weaviate filter values (#14560)
### Motivation and Context The Chroma and Weaviate `_lambda_parser` implementations double apostrophes in string constants (`O'Brien` becomes `O''Brien`). That is SQL-style escaping, but these connectors pass the value as data (a Chroma `where` dict, a Weaviate `Filter` object), so the doubled value never equals the stored one. A record with `name="O'Brien"` cannot be found with `x.name == "O'Brien"`. I reproduced it against a real in-process Chroma client: the filter built `{'name': "O''Brien"}` and the search returned no results, while a filter on `"Smith"` returned the Smith record. Fixes #14559 ### Description - Return string (and decoded bytes) constants unchanged in `ChromaCollection._lambda_parser` and `WeaviateCollection._lambda_parser`, the same way the MongoDB, Pinecone and Qdrant connectors already do. - Connectors that build query or SQL strings (Azure AI Search, Postgres, SQL Server, Oracle) are not touched. - Add a unit test for each connector. Both fail on `main` and pass with this change. After the change the same real-Chroma script returns record `1`. - Chroma, Weaviate and `tests/unit/data` unit tests pass, and ruff reports nothing on the changed files. ### Contribution Checklist - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
1 parent f37f308 commit 555c46b

4 files changed

Lines changed: 26 additions & 4 deletions

File tree

‎python/semantic_kernel/connectors/chroma.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -424,9 +424,9 @@ def _lambda_parser(self, node: ast.AST) -> dict[str, Any] | str | int | float |
424424
case ast.Constant():
425425
value = node.value
426426
if isinstance(value, str):
427-
return value.replace("'", "''")
427+
return value
428428
if isinstance(value, bytes):
429-
return value.decode("utf-8").replace("'", "''")
429+
return value.decode("utf-8")
430430
if isinstance(value, (int, float, bool)) or value is None:
431431
return value
432432
raise VectorStoreOperationException(f"Unsupported constant type: {type(value)}")

‎python/semantic_kernel/connectors/weaviate.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -471,9 +471,9 @@ def _lambda_parser(self, node: ast.AST) -> "_Filters | FilterValues":
471471
case ast.Constant():
472472
value = node.value
473473
if isinstance(value, str):
474-
return value.replace("'", "''")
474+
return value
475475
if isinstance(value, bytes):
476-
return value.decode("utf-8").replace("'", "''")
476+
return value.decode("utf-8")
477477
if isinstance(value, (int, float, bool)) or value is None:
478478
return value
479479
raise VectorStoreOperationException(f"Unsupported constant type: {type(value)}")

‎python/tests/unit/connectors/memory/test_chroma.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,3 +113,9 @@ async def test_chroma_collection_search(chroma_collection, mock_client, include_
113113
async for res in results.results:
114114
assert res.record["id"] == "1"
115115
assert res.score == 0.1
116+
117+
118+
def test_chroma_collection_build_filter_keeps_apostrophes(chroma_collection):
119+
"""Chroma filters are dicts, not SQL, so the value must reach Chroma unchanged."""
120+
assert chroma_collection._build_filter('lambda x: x.content == "O\'Brien"') == {"content": "O'Brien"}
121+
assert chroma_collection._build_filter('lambda x: x.content != "O\'Brien"') == {"content": {"$ne": "O'Brien"}}

‎python/tests/unit/connectors/memory/weaviate/test_weaviate_collection.py‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -427,3 +427,19 @@ async def test_weaviate_collection_deserialize_data(
427427
await collection.get(key=data.id)
428428

429429
mock_inner_get.assert_called_once_with([data.id], include_vectors=False, options=None)
430+
431+
432+
def test_weaviate_collection_build_filter_keeps_apostrophes(clear_weaviate_env, record_type, definition) -> None:
433+
"""Weaviate filters are objects, not query strings, so the value must be passed through unchanged."""
434+
collection = WeaviateCollection(
435+
record_type=record_type,
436+
definition=definition,
437+
collection_name="TestCollection",
438+
async_client=AsyncMock(spec=WeaviateAsyncClient),
439+
env_file_path="fake_env_file_path.env",
440+
)
441+
442+
result = collection._build_filter('lambda x: x.content == "O\'Brien"')
443+
444+
assert result.target == "content"
445+
assert result.value == "O'Brien"

0 commit comments

Comments
 (0)