Skip to content

Commit b7e909b

Browse files
peterxcliclaude
andcommitted
docs: point the clock audit entries at the surviving benchmark, correct stale figures
Review of #5771 noted three leftovers. The hour, minute and second entries on the datetime audit page still named benches/extract_clock_fields.rs, which this PR folded into benches/extract_date_part.rs and deleted. They also quoted "83-96% faster" from before the harness repair; the matched results put every fast-path shape with a baseline of at least 2 us at 82-94% less time, with offset session zones and dictionary input on the general path and within noise on a repeat. The dayofweek and weekday entries still carried "8.4-10.2x faster" from before the dense-null and dictionary work. The final figures are about 9x on flat input with no or sparse nulls, 3-4x at 87.5% nulls, roughly 1.1x on an all-null batch or a cardinality-8 dictionary where the replaced path already did little work, and about 2x at cardinality 1024. The audit page is what the next contributor reads instead of the PR body, so it should carry the numbers that held up. Removing `val DayOfWeek = Value("dow")` from CometGetDateField left its two comment lines behind, so the Sunday-is-0 note sat above `val DayOfYear`. Co-Authored-By: Claude Opus 5 <[email protected]>
1 parent a47574a commit b7e909b

2 files changed

Lines changed: 5 additions & 7 deletions

File tree

‎docs/source/contributor-guide/expression-audits/datetime_funcs.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939

4040
## dayofweek
4141

42-
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): computed directly from the epoch day (`((days + 4).rem_euclid(7) + 1)`) by the native `spark_dayofweek` kernel, replacing `datepart('dow', ..)` -- which reconstructs a `NaiveDateTime` per row and recomputes the null mask via `unary_opt` -- plus a separate `+ 1` arithmetic node. 8.4-10.2x faster. Note the old path returned NULL for epoch days outside chrono's range; the kernel is correct across the full `i32` domain. Benchmark: `benches/dayofweek_weekday.rs`.
42+
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): computed directly from the epoch day (`((days + 4).rem_euclid(7) + 1)`) by the native `spark_dayofweek` kernel, replacing `datepart('dow', ..)` -- which reconstructs a `NaiveDateTime` per row and recomputes the null mask via `unary_opt` -- plus a separate `+ 1` arithmetic node. About 9x faster on flat input with no or sparse nulls, 3-4x at 87.5% nulls, and roughly 1.1x on an all-null batch or a cardinality-8 dictionary, where the replaced path already did little work (about 2x at cardinality 1024). Note the old path returned NULL for epoch days outside chrono's range; the kernel is correct across the full `i32` domain. Benchmark: `benches/dayofweek_weekday.rs`.
4343

4444
## from_utc_timestamp
4545

@@ -50,7 +50,7 @@
5050

5151
## hour
5252

53-
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): `hour`/`minute`/`second` take an integer fast path when no timezone offset applies -- `TimestampNTZ`, or a timezone-aware timestamp in a UTC session -- computing the field from the stored microseconds with Euclidean division instead of building a `chrono` datetime per row via `date_part`. Dictionaries, non-microsecond units and offset timezones keep the general path. 83-96% faster; the offset-timezone path is unchanged. Benchmark: `benches/extract_clock_fields.rs`.
53+
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): `hour`/`minute`/`second` take an integer fast path when no timezone offset applies -- `TimestampNTZ`, or a timezone-aware timestamp in a UTC session -- computing the field from the stored microseconds with Euclidean division instead of building a `chrono` datetime per row via `date_part`. Dictionaries, non-microsecond units and offset timezones keep the general path. Every fast-path shape with a baseline of at least 2 µs takes 82-94% less time; offset session zones and dictionary input keep the general path and stay within noise on a repeat. Benchmark: `benches/extract_date_part.rs`.
5454

5555
## make_timestamp_ltz
5656

@@ -62,7 +62,7 @@
6262

6363
## minute
6464

65-
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): `hour`/`minute`/`second` take an integer fast path when no timezone offset applies -- `TimestampNTZ`, or a timezone-aware timestamp in a UTC session -- computing the field from the stored microseconds with Euclidean division instead of building a `chrono` datetime per row via `date_part`. Dictionaries, non-microsecond units and offset timezones keep the general path. 83-96% faster; the offset-timezone path is unchanged. Benchmark: `benches/extract_clock_fields.rs`.
65+
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): `hour`/`minute`/`second` take an integer fast path when no timezone offset applies -- `TimestampNTZ`, or a timezone-aware timestamp in a UTC session -- computing the field from the stored microseconds with Euclidean division instead of building a `chrono` datetime per row via `date_part`. Dictionaries, non-microsecond units and offset timezones keep the general path. Every fast-path shape with a baseline of at least 2 µs takes 82-94% less time; offset session zones and dictionary input keep the general path and stay within noise on a repeat. Benchmark: `benches/extract_date_part.rs`.
6666

6767
## monthname
6868

@@ -74,7 +74,7 @@
7474

7575
## second
7676

77-
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): `hour`/`minute`/`second` take an integer fast path when no timezone offset applies -- `TimestampNTZ`, or a timezone-aware timestamp in a UTC session -- computing the field from the stored microseconds with Euclidean division instead of building a `chrono` datetime per row via `date_part`. Dictionaries, non-microsecond units and offset timezones keep the general path. 83-96% faster; the offset-timezone path is unchanged. Benchmark: `benches/extract_clock_fields.rs`.
77+
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): `hour`/`minute`/`second` take an integer fast path when no timezone offset applies -- `TimestampNTZ`, or a timezone-aware timestamp in a UTC session -- computing the field from the stored microseconds with Euclidean division instead of building a `chrono` datetime per row via `date_part`. Dictionaries, non-microsecond units and offset timezones keep the general path. Every fast-path shape with a baseline of at least 2 µs takes 82-94% less time; offset session zones and dictionary input keep the general path and stay within noise on a repeat. Benchmark: `benches/extract_date_part.rs`.
7878

7979
## to_date
8080

@@ -115,4 +115,4 @@
115115

116116
## weekday
117117

118-
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): computed directly from the epoch day (`(days + 3).rem_euclid(7)`) by the native `spark_weekday` kernel, replacing `datepart('isodow', ..)` plus a `- 1` arithmetic node. 8.4-10.2x faster. `weekday` numbers Monday = 0 through Sunday = 6, a different convention from `dayofweek`. Benchmark: `benches/dayofweek_weekday.rs`.
118+
- Performance (tuned 2026-09-08, PR [#5771](https://github.com/apache/datafusion-comet/pull/5771)): computed directly from the epoch day (`(days + 3).rem_euclid(7)`) by the native `spark_weekday` kernel, replacing `datepart('isodow', ..)` plus a `- 1` arithmetic node. About 9x faster on flat input with no or sparse nulls, 3-4x at 87.5% nulls, and roughly 1.1x on an all-null batch or a cardinality-8 dictionary, where the replaced path already did little work (about 2x at cardinality 1024). `weekday` numbers Monday = 0 through Sunday = 6, a different convention from `dayofweek`. Benchmark: `benches/dayofweek_weekday.rs`.

‎spark/src/main/scala/org/apache/comet/serde/datetime.scala‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,8 +40,6 @@ private object CometGetDateField extends Enumeration {
4040
val Year: Value = Value("year")
4141
val Month: Value = Value("month")
4242
val DayOfMonth: Value = Value("day")
43-
// Datafusion: day of the week where Sunday is 0, but spark sunday is 1 (1 = Sunday,
44-
// 2 = Monday, ..., 7 = Saturday).
4543
val DayOfYear: Value = Value("doy")
4644
val WeekOfYear: Value = Value("week")
4745
val Quarter: Value = Value("quarter")

0 commit comments

Comments
 (0)