Repository navigation
Commit e14e086
committed
test: port Spark's string-to-timestamp cases; fold regexes onto TimestampPattern
Addresses review feedback on #5130.
Move the regex table from two order-coupled const arrays onto `TimestampPattern`
itself: `ALL` lists the shapes in match order and `regex_str` returns each shape's
regex, so a shape and its pattern are declared together. The lazy `RegexSet` is now
built from `ALL.map(regex_str)`. No behavior change - same patterns, same order, and
the differential test still fails if the classifier is perturbed.
Port the string-to-timestamp cases from Spark's `DateTimeUtilsSuite`:
`test("string to timestamp")`, `SPARK-35780` (full range), `SPARK-37326`
(`allowTimeZone`), and the `stringToTimestamp` half of `SPARK-15379`, plus a port of
the `permuteWithWhitespaceAndControl` helper. Expected values are computed from a
proleptic-Gregorian day count rather than from the parser, so they are independent of
the code under test and cover the years beyond chrono's range that the
`Long.MinValue`/`MaxValue` microsecond boundaries need. Each case is also checked
across eval modes and a small set of session timezones, dimensions Spark's suite does
not have.
All of Spark's expectations hold except two pre-existing divergences, unchanged from
main and unrelated to the shape classifier, tracked by #5165:
- ISO control characters are not trimmed (`str::trim` only strips Unicode whitespace,
where Spark strips whitespace and ISO control), so a control-padded timestamp
returns null.
- A leading '+' that is not a year sign returns null under ANSI where Spark raises.
Both are asserted at their current behavior with a pointer to the issue so they cannot
silently widen.1 parent aa38e34 commit e14e086
1 file changed
Lines changed: 582 additions & 52 deletions
0 commit comments