Skip to content

Commit daaf6e4

Browse files
committed
test: cover a non-UTC coerce_int96_tz read; note protobuf rejection in upgrade guide
1 parent 046e41c commit daaf6e4

2 files changed

Lines changed: 32 additions & 3 deletions

File tree

‎datafusion/sqllogictest/test_files/parquet.slt‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -898,9 +898,37 @@ drop table int96_from_spark;
898898
statement ok
899899
reset datafusion.execution.time_zone;
900900

901+
# A non-UTC offset keeps the stored instant and renders it at that offset,
902+
# eight hours ahead of the UTC values above.
901903
statement ok
902904
set datafusion.execution.parquet.coerce_int96 = ns;
903905

906+
statement ok
907+
set datafusion.execution.parquet.coerce_int96_tz = '+08:00';
908+
909+
statement ok
910+
CREATE EXTERNAL TABLE int96_from_spark
911+
STORED AS PARQUET
912+
LOCATION '../../parquet-testing/data/int96_from_spark.parquet';
913+
914+
query TTT
915+
describe int96_from_spark;
916+
----
917+
a Timestamp(ns, "+08:00") YES
918+
919+
query P
920+
select * from int96_from_spark
921+
----
922+
2024-01-02T04:34:56.123456+08:00
923+
2024-01-01T09:00:00+08:00
924+
1816-03-29T16:56:08.066277376+08:00
925+
2024-12-31T07:00:00+08:00
926+
NULL
927+
1815-11-09T00:01:01.191053312+08:00
928+
929+
statement ok
930+
drop table int96_from_spark;
931+
904932
statement ok
905933
reset datafusion.execution.parquet.coerce_int96_tz;
906934

‎docs/source/library-user-guide/upgrading/56.0.0.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -407,9 +407,10 @@ let tz: Option<&str> = options.execution.time_zone.as_ref().map(|tz| tz.as_str()
407407

408408
`ParquetOptions::coerce_int96_tz` changed from `Option<String>` to
409409
`Option<ConfigTimeZone>`, the same type as `ExecutionOptions::time_zone`. An
410-
invalid time zone is now rejected by `SET` and by table options instead of on
411-
the first Parquet read. This applies even when `coerce_int96` is not set, where
412-
the time zone has no effect.
410+
invalid time zone is now rejected by `SET`, by table options, and by protobuf
411+
decoding instead of on the first Parquet read. This applies even when
412+
`coerce_int96` is not set, where the time zone has no effect. Valid values and
413+
the protobuf wire format are unchanged.
413414

414415
**Who is affected:**
415416

0 commit comments

Comments
 (0)