Skip to content

Commit e14e086

Browse files
committed
test: port Spark's string-to-timestamp cases; fold regexes onto TimestampPattern
Addresses review feedback on #5130. Move the regex table from two order-coupled const arrays onto `TimestampPattern` itself: `ALL` lists the shapes in match order and `regex_str` returns each shape's regex, so a shape and its pattern are declared together. The lazy `RegexSet` is now built from `ALL.map(regex_str)`. No behavior change - same patterns, same order, and the differential test still fails if the classifier is perturbed. Port the string-to-timestamp cases from Spark's `DateTimeUtilsSuite`: `test("string to timestamp")`, `SPARK-35780` (full range), `SPARK-37326` (`allowTimeZone`), and the `stringToTimestamp` half of `SPARK-15379`, plus a port of the `permuteWithWhitespaceAndControl` helper. Expected values are computed from a proleptic-Gregorian day count rather than from the parser, so they are independent of the code under test and cover the years beyond chrono's range that the `Long.MinValue`/`MaxValue` microsecond boundaries need. Each case is also checked across eval modes and a small set of session timezones, dimensions Spark's suite does not have. All of Spark's expectations hold except two pre-existing divergences, unchanged from main and unrelated to the shape classifier, tracked by #5165: - ISO control characters are not trimmed (`str::trim` only strips Unicode whitespace, where Spark strips whitespace and ISO control), so a control-padded timestamp returns null. - A leading '+' that is not a year sign returns null under ANSI where Spark raises. Both are asserted at their current behavior with a pointer to the issue so they cannot silently widen.
1 parent aa38e34 commit e14e086

1 file changed

Lines changed: 582 additions & 52 deletions

File tree

  • native/spark-expr/src/conversion_funcs

0 commit comments

Comments
 (0)