Repository navigation
[Bug] Arrow Flight SQL: first VARIANT read on a pre-existing connection returns an empty string instead of {} #67367
Description
Search before asking
- I had searched in the issues and found no similar issues.
Version
Apache Doris 4.1.3-rc02, commit 31263df4dc1d4d3a27517d264802cd4d6b92c874
Client: Python + ADBC Flight SQL driver (adbc_driver_flightsql), FE arrow_flight_sql_port = 41070.
The MySQL/JDBC protocol is used as the control path for comparison.
What's Wrong?
If an ADBC Flight SQL connection is opened before a VARIANT table is created and written, the first read of a persisted empty JSON object on that connection can return an empty string instead of {}.
Running exactly the same query a second time on the same connection returns {}, and a freshly opened ADBC connection also returns {} on its first try.
The raw Arrow schema is int32, string, string:
- pre-existing connection, first read:
["1", "", ""] - same connection, second read:
["1", "{}", "{}"] - new connection after the write:
["1", "{}", "{}"]
What You Expected?
The first read and all subsequent reads should be identical. An empty JSON object should come back as {} for both nullable and NOT NULL VARIANT columns.
How to Reproduce?
- Open the ADBC Flight SQL connection first, without querying the target table yet.
- Over a separate MySQL/JDBC connection, run the DDL and
INSERTbelow. - On the pre-existing ADBC connection, run the same
SELECTtwice in a row. - Compare the two Arrow results, then open a new ADBC connection and run it once more as a control.
DROP TABLE IF EXISTS adbc_variant_first_read;
CREATE TABLE adbc_variant_first_read (
id INT,
v_nullable VARIANT NULL,
v_not_null VARIANT NOT NULL
) DUPLICATE KEY(id)
DISTRIBUTED BY HASH(id) BUCKETS 1
PROPERTIES("replication_num"="1");
INSERT INTO adbc_variant_first_read
VALUES (1, PARSE_JSON('{}'), PARSE_JSON('{}'));
SELECT id, CAST(v_nullable AS STRING), CAST(v_not_null AS STRING)
FROM adbc_variant_first_read ORDER BY id;Client side:
import adbc_driver_flightsql.dbapi as flight_sql
# Open this connection BEFORE the JDBC session runs CREATE TABLE and INSERT.
conn = flight_sql.connect(uri="grpc://127.0.0.1:41070",
db_kwargs={"username": "root", "password": ""})
cur = conn.cursor()
sql = ("SELECT id, CAST(v_nullable AS STRING), CAST(v_not_null AS STRING) "
"FROM adbc_variant_first_read ORDER BY id")
cur.execute(sql)
print(cur.fetch_arrow_table().to_pylist()) # first read may contain empty strings
cur.execute(sql)
print(cur.fetch_arrow_table().to_pylist()) # returns {}Anything Else?
This looks like stale per-connection schema/metadata state on the Flight SQL session: the connection was established before the table existed, and the first read resolves the variant subcolumns against that stale state.
Workaround: retry the identical read, or re-establish the ADBC connection after the write completes. A silent retry was deliberately not added to the test path, because it would hide the consistency problem.
Also reproducible through the variant_p0/variant_hirachinal regression case, in addition to the isolated table above.
Tracking issue: #65615
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
Doris-Breakwater commentedon Sep 1, 2026
Breakwater-GitHub-Analysis-Slot: slot_0480a2cd30b5
Initial triage
This should be treated as a correctness bug, not only a client-display issue: a persisted JSON object is reported as a different, valid string value on the first read. The identical Arrow schema on the bad and good executions (
int32, string, string), together with both nullable and non-nullable columns being affected, makes an Arrow type mismatch or outer-nullability handling unlikely.The proposed root cause (stale table/schema metadata held by the pre-existing Flight SQL connection) is not yet established. In the supplied sequence, the new-connection control runs only after the old connection has executed the query twice. It therefore changes both connection age and query order: a first-reader/storage initialization or timing effect could produce the same observation.
What the 4.1.3-rc02 code supports
I inspected commit
31263df4dc1d4d3a27517d264802cd4d6b92c874read-only.FlightSqlConnectContext,DorisFlightSqlProducer, orFlightSqlConnectProcessorthat would directly explain a table created after connection establishment.CAST(VARIANT AS STRING)is evaluated before the Arrow result conversion. The Flight result writer receives an ordinary Doris string column, so Arrow schema metadata alone cannot turn{}into an empty string.ColumnVariantserializer explicitly emits{}when a row has no visible paths, and the hierarchical reader comments/code preserve an empty root placeholder as a present empty object rather than SQLNULL.{}, while an absent subtree is SQLNULL.These are verified code-path facts, but they do not identify the component that first produces the zero-length value. The remaining hypotheses include a first-read storage/Variant initialization path, a session-state difference, and the ADBC/Arrow conversion boundary. There is currently insufficient evidence to rank one as the root cause.
Information needed to isolate the fault
Please attach:
adbc_driver_flightsql, PyArrow, and the Arrow C++ libraries used by the driver.SHOW CREATE TABLE, FE/BE topology and deployment mode, plus the effective Variant settings—especiallyenable_variant_v2and the table's document/hierarchical-storage settings. Please do not infer these from release defaults.LENGTH(...)andHEX(...)on the first execution. If the server-computed length is already0, the fault precedes Arrow string-buffer materialization; if it is2while the fetched value is empty, focus on the Flight/ADBC boundary.The most important reproduction control is to remove the query-order confounder after each fresh table creation:
CAST(... AS STRING)query over MySQL/JDBC and recordLENGTH/HEXas a storage/control-path check.{"a":1}and an empty array to determine whether this is specific to the empty-object root placeholder.Suggested maintainer next steps
Reproduce on the reported commit with the controlled ordering above, then inspect the value at three boundaries for the bad query ID: after Variant row assembly, after
VARIANT -> STRINGserialization, and immediately before the Doris block is converted to an Arrow batch. The first boundary at which the bytes change from7b 7dto zero length will determine ownership without assuming a connection cache.Once isolated, add an end-to-end regression that opens the Flight connection before DDL, persists
{}, and asserts the first read, repeated read, and first read from a second connection. A lower-level persisted-segment scan/cast test should accompany it if the corruption is already present before the result sink.Retrying or reconnecting is a reasonable temporary workaround, but a silent retry would mask a correctness failure and should not be considered the fix. This issue can remain linked to #65615, but it should not be closed as a duplicate without identifying the failing boundary. No labels are currently attached;
kind/bug,area/rpc, and the repository's existing VARIANT/storage ownership label (if available) would make routing clearer.