Repository navigation
[fix](arrow) Reject out-of-range Flight timestamps - #68604
Merged
Gabriel39 merged 3 commits intoSep 29, 2026
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Flight SQL can publish out-of-range timestamps from legacy INT96 date materialization. For example, a non-null zero DATETIME becomes
timestamp[us]with value-62169984000000000: Arrow validation succeeds, but PyArrow scalar conversion raisesOverflowError.Validate timestamps in
ArrowFlightArrowBlockConvertorbefore publishing each batch. Check the actual timestamp unit and both UTC/local calendar bounds for zoned values. Recursively check arrays, map keys/values, and structs while skipping values masked by a NULL parent. Reject unsupported timestamps with an error identifying the field path and block row. Other Arrow consumers and Parquet scan compatibility remain unchanged.Related PR: #68596 adds separate UTF-8 validation. This PR is independent of that change; both checks must be retained when integrating the shared Flight conversion entry point.
Testing
ArrowFlightTimestampTest.*:DataTypeSerDeArrowTest.*passed under ASAN. Recompiled the changed converter and new tests against existing ASAN BE libraries, with C++ access control enabled. Test dates are constructed through the public unchecked setter so invalid calendar values reach the validator.connectshortcut. The HDFS suite discovers the live FE Flight endpoint through MySQL because external clusters can override the configured port. The HDFS suite requiresenableHiveTestand the external test environment.getAt()failure by consuming the JDBC array directly.Live external regression validation
8d1158d3f5in an isolated local cluster and rebuilt the regression framework from this branch. This follow-up changes only test scripts and expected results.test_flight_int96_timestamp_range,test_remote_doris_all_types_select,test_remote_doris_statistics, andtest_remote_doris_table_stats.Release note
Flight SQL now returns a server-side error for timestamps outside the supported 0001–9999 calendar range instead of publishing values that fail client conversion. Valid timestamps and NULLs retain their existing values.
Check List (For Author)