Skip to content

[VECTOR_FLOAT16] Add bulk copy support for vector float16 and added tests#2898

Merged
muskan124947 merged 4 commits intomainfrom
users/muskgupta/vectorFloat16_bulkcopySupport
Mar 5, 2026
Merged

[VECTOR_FLOAT16] Add bulk copy support for vector float16 and added tests#2898
muskan124947 merged 4 commits intomainfrom
users/muskgupta/vectorFloat16_bulkcopySupport

Conversation

@muskan124947
Copy link
Copy Markdown
Contributor

@muskan124947 muskan124947 commented Feb 23, 2026

Description

Building on the existing VECTOR(FLOAT16) support in [VECTOR_FLOAT16] Add support for vector float16 data type
#2899
, this PR adds full end-to-end support for the bulk copy operations (bulk inserts with useBulkCopyForBatchInsert, table-to-table bulk copies, CSV-to-table bulk copies).

Below are some example scenarios:

  1. Table to table bulk copy
Vector vector = new Vector(3, Vector.VectorDimensionType.FLOAT16, new Float[]{1.0f, 2.0f, 3.0f});
try (Statement stmt = con.createStatement();
        SQLServerBulkCopy bulkCopy = new SQLServerBulkCopy(con)) {

        stmt.executeUpdate("create table destinationTable (vector_col vector(3, float16))");

        bulkCopy.setDestinationTableName("destinationTable");
        bulkCopy.writeToServer(vector);
    }
  1. CSV-to-table bulk copies
    vector.csv
vector_col
"[1.0,2.0,3.0]"
"[4.0,5.0,6.0]"
try (Statement stmt = con.createStatement();
    SQLServerBulkCopy bulkCopy = new SQLServerBulkCopy(con);
    SQLServerBulkCSVFileRecord fileRecord = new SQLServerBulkCSVFileRecord("vectors.csv", null, ",", true)) {

        stmt.executeUpdate("create table destinationTable (vector_col vector(3, float16))");

        // Add column metadata for the CSV file
        fileRecord.addColumnMetadata(1, "vector_col", microsoft.sql.Types.VECTOR, 3, 2);
        fileRecord.setEscapeColumnDelimitersCSV(true);

        bulkCopy.setDestinationTableName("destinationTable");
        bulkCopy.writeToServer(fileRecord);
}
  1. Bulk inserts with useBulkCopyForBatchInsert
        String connStr = connectionString + ";vectorTypeSupport=v2" + ";useBulkCopyForBatchInsert=true;";
        String sqlString = "insert into " + AbstractSQLGenerator.escapeIdentifier(bulkCopyTableName)
                + " (vectorCol) values (?)";

        try (SQLServerConnection conn = (SQLServerConnection) DriverManager.getConnection(connStr);
                SQLServerPreparedStatement pstmt = (SQLServerPreparedStatement) conn.prepareStatement(sqlString);
                Statement stmt = conn.createStatement()) {

            Float[] vectorData = createTestData(4.0f, 5.0f, 6.0f);
            Vector vector = new Vector(vectorData.length, getVectorDimensionType(), vectorData);

            pstmt.setObject(1, vector, microsoft.sql.Types.VECTOR);
            pstmt.addBatch();
            pstmt.executeBatch();
    }

Note: Currently, tests tagged with vectorFloat16Test are excluded from ADO runs and Github checks. These will be enabled post the official build rollout.

@muskan124947 muskan124947 self-assigned this Feb 23, 2026
@github-project-automation github-project-automation Bot moved this to In progress in MSSQL JDBC Feb 23, 2026
@muskan124947 muskan124947 marked this pull request as ready for review February 25, 2026 09:48
@muskan124947 muskan124947 added this to the 13.3.2 milestone Feb 25, 2026
@muskan124947 muskan124947 added the Under Review Used for pull requests under review label Feb 25, 2026
# Conflicts:
#	src/test/java/com/microsoft/sqlserver/jdbc/datatypes/vector/VectorFloat16Test.java
#	src/test/java/com/microsoft/sqlserver/jdbc/datatypes/vector/VectorFloat32Test.java
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds bulk copy support for the VECTOR FLOAT16 data type in the JDBC driver. It extends the existing FLOAT32 vector bulk copy support to also handle FLOAT16 vectors, adding version-checking logic to enforce that FLOAT16 is only available when the server negotiates vector version 2 (v2). The PR also reorganizes vector bulk copy tests from scattered locations into a dedicated abstract base class hierarchy.

Changes:

  • Production code in SQLServerBulkCopy.java adds version checks that reject FLOAT16 vectors on v1 connections and throw errors when vector support is entirely disabled
  • SQLServerBulkCSVFileRecord.java fixes a bug where the scale passed to getVectorDimensionType() was bytes-per-dimension (4/2) but the method expected a TDS scale byte (0/1)
  • Test code is reorganized: vector bulk copy tests moved from BatchExecutionWithBulkCopyTest, BulkCopyCSVTest, and DatabaseMetaDataTest into a new abstract VectorBulkCopyTest base class with VectorFloat32BulkCopyTest and VectorFloat16BulkCopyTest concrete subclasses

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
SQLServerBulkCopy.java Adds getNegotiatedVectorVersion() checks in writeTypeInfo and getDestTypeFromSrcType to reject unsupported vector types; adds FLOAT16 column definition
SQLServerBulkCSVFileRecord.java Fixes bug: converts bytes-per-dimension scale to TDS scale byte before calling getVectorDimensionType()
VectorBulkCopyTest.java New abstract base class with all shared bulk copy test logic and inner helper implementations
VectorFloat32BulkCopyTest.java FLOAT32-specific test class; adds tests for FLOAT16-on-v1-connection error cases
VectorFloat16BulkCopyTest.java FLOAT16-specific test class; delegates all tests to base class
BatchExecutionWithBulkCopyTest.java Removes FLOAT32-only vector bulk copy tests (moved to new test hierarchy)
BulkCopyCSVTest.java Removes FLOAT32-only vector CSV bulk copy tests (moved to new test hierarchy)
DatabaseMetaDataTest.java Removes FLOAT32-only vector metadata test (presumably moved elsewhere or removed as duplicate)

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 3, 2026

Codecov Report

❌ Patch coverage is 40.00000% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.65%. Comparing base (53b6a4a) to head (b7e4654).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...om/microsoft/sqlserver/jdbc/SQLServerBulkCopy.java 35.71% 5 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##               main    #2898   +/-   ##
=========================================
  Coverage     60.64%   60.65%           
- Complexity     4913     4921    +8     
=========================================
  Files           151      151           
  Lines         34986    35000   +14     
  Branches       5849     5854    +5     
=========================================
+ Hits          21218    21229   +11     
- Misses        10927    10937   +10     
+ Partials       2841     2834    -7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…lkcopy/VectorBulkCopyTest.java

Co-authored-by: Copilot <[email protected]>
@muskan124947 muskan124947 merged commit e7f08a8 into main Mar 5, 2026
18 of 19 checks passed
@github-project-automation github-project-automation Bot moved this from In progress to Closed/Merged PRs in MSSQL JDBC Mar 5, 2026
@muskan124947 muskan124947 removed the Under Review Used for pull requests under review label Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Closed/Merged PRs

Development

Successfully merging this pull request may close these issues.

4 participants