Repository navigation
Commit 7de2708
committed
test: align the system-function corpus write with Spark 4 datetime defaults
CometIcebergSystemFunctionSuite aborted in beforeAll on Spark 3.4 and 3.5.
Three Spark 3.x defaults differ from Spark 4's and each blocks writing the
corpus:
- datetimeJava8ApiEnabled is off, so the row encoder rejects the java.time
values sourceData supplies (ClassCastException: LocalDate -> java.sql.Date)
- datetimeRebaseModeInWrite is EXCEPTION, which rejects the deliberately
pre-epoch timestamps (the corpus reaches back to 1843)
- outputTimestampType is INT96, which has its own ancient-timestamp check
Set all three to Spark 4's values around the write only. Every test reads the
data back from parquet, so no result comparison is affected.1 parent f8db9b5 commit 7de2708
1 file changed
Lines changed: 21 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
85 | 85 | | |
86 | 86 | | |
87 | 87 | | |
88 | | - | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
89 | 109 | | |
90 | 110 | | |
91 | 111 | | |
| |||
0 commit comments