Skip to content

fix: set element_type for ARRAY fields in insert and upsert requests - #3872

Open
Thanatos9404 wants to merge 2 commits into
milvus-io:masterfrom
Thanatos9404:fix/2269-array-element-type
Open

Thanatos9404 wants to merge 2 commits into
milvus-io:masterfrom
Thanatos9404:fix/2269-array-element-type

Conversation

@Thanatos9404

@Thanatos9404 Thanatos9404 commented Oct 19, 2026 •

Copy link
Copy Markdown

issue: #2269

Insert and upsert requests write the rows of an ARRAY field into scalars.array_data.data, but scalars.array_data.element_type is never set, so it is left at its default, DataType.NONE. This happens on the column and row paths for both insert and upsert. The vector array path next to it already sets its element type.

The server reads this value to size array rows when it splits insert messages (see milvus-io/milvus#36029). Since milvus-io/milvus#36373, the fix for that issue, the proxy fills it in from the schema, so this mostly makes the client send a complete request instead of relying on that fallback.

The change adds a small set_array_element_type helper that copies the element type declared in the schema, and calls it in entity_to_field_data and _pack_scalar_row. This PR also covered the ARRAY sub-fields of struct fields, but master sets those since #3882, so after merging master it keeps only the top-level paths.

New unit tests cover entity_to_field_data, row_insert_param and row_upsert_param, including struct sub-fields. They fail on master and pass with the fix. After the merge, tests/unit passes locally on Windows, apart from three symlink tests in test_volume_upload_policy.py that need a privilege Windows does not grant by default, and black and ruff are clean.

ARRAY fields had their rows written to scalars.array_data.data, but
scalars.array_data.element_type was never set, so requests left it at
its default, DataType.NONE. This affected column and row based inserts
and upserts, and the ARRAY sub-fields of struct fields.

Set it from the element type declared in the schema wherever array
rows are written, the same way the vector array path already sets its
element type. For struct sub-fields it is only set when data is
appended, so a null struct row still leaves the payload empty.

issue: milvus-io#2269

Signed-off-by: Yashvardhan Thanvi <[email protected]>
@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Thanatos9404
To complete the pull request process, please assign tedxu after the PR has been reviewed.
You can assign the PR to them by writing /assign @tedxu in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot

Copy link
Copy Markdown

Welcome @Thanatos9404! It looks like this is your first PR to milvus-io/pymilvus 🎉

@mergify

mergify Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Master now sets the element type on the ARRAY sub-fields of struct
fields (milvus-io#3882), so prepare.py keeps the master version. The top-level
ARRAY paths in entity_helper.py, column and row based, still need the
fix and keep it.

Signed-off-by: Yashvardhan Thanvi <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants