Repository navigation
fix(dtype): preserve scaled generic time units and normalize the μs alias - #4342
Merged
d-v-b merged 6 commits intoSep 24, 2026
Merged
Conversation
Documentation build overview
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4342 +/- ##
==========================================
+ Coverage 94.36% 94.37% +0.01%
==========================================
Files 93 93
Lines 13142 13171 +29
==========================================
+ Hits 12401 12430 +29
Misses 741 741
🚀 New features to boost your workflow:
|
d-v-b
changed the title
…lias `DateTime64` and `TimeDelta64` read unit and scale from the dtype object rather than `dtype.name`, which drops a generic unit's scale. V2 metadata writes an explicit `[Ngeneric]` suffix; the constructor stores `μs` as `us`. NumPy allocation drops a generic time dtype's scale and byte order, and `astype` between generic time dtypes ignores the target. The CPU buffer restores the dtype with a view after allocation and converts endian-only casts through the integer counts. Integer fill values for generic datetimes decode through `datetime_from_int`, and structured defaults build a zero count for generic datetime fields. Assisted-by: ClaudeCode:claude-opus-5-5
d-v-b
force-pushed
the
claude/friendly-williams-cbd545
branch
from
September 24, 2026 15:25
df506a1 to
bc12e17
Compare
d-v-b
marked this pull request as ready for review
September 24, 2026 15:28
Generate the valid native dtypes and V2/V3 JSON for `DateTime64` and `TimeDelta64` from every unit, scales 1, 2 and 2**31 - 1, and both byte orders, instead of a separate round-trip test that repeated the harness's native and JSON round trips. Assisted-by: ClaudeCode:claude-opus-5-5
`DateTime64.default_scalar`, `TimeDelta64.default_scalar`, and the NaT branch of `TimeDelta64.cast_scalar` built NaT in the bare unit, dropping the scale factor. On NumPy < 2.2, casting a timedelta NaT between scales yields a count instead of NaT, so missing chunks of a `m8[2us]` array read back as 2**62 - 1 instead of NaT. Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Preserves the unit, scale, and byte order of NumPy
datetime64/timedelta64dtypes through native conversion, Zarr metadata, and CPU chunk IO. The largest gap is the generic unit with a scale factor, e.g.np.dtype("M8[2generic]").Changes
DateTime64andTimeDelta64read the unit and scale from the dtype object. They previously readdtype.name, which omits a generic unit's scale.dtype.stralso omits the generic scale, so V2 serialization appends an explicit suffix, e.g.<M8[2generic]. V3 already stores unit and scale in its configuration.μsalias. The constructor storesμsasus, NumPy's spelling. Without this an instance built withμscompares unequal to itself after a round trip through NumPy.μsis still accepted as input and in stored metadata.np.zeros(1, ">M8[2generic]").dtypeis<M8).cpu.NDBuffer.createandemptyrestore the requested dtype with a view before filling.astype. NumPy'sastypebetween generic time dtypes ignores the target and keeps the source byte order.cpu.NDBuffer.astypeconverts endian-only generic casts through the integer counts.datetime_from_int, since NumPy's scalar constructor rejects integer counts with a generic unit. Structured defaults build a zero count for generic datetime fields.default_scalarand the NaT branch ofTimeDelta64.cast_scalarbuilt NaT in the bare unit, dropping the scale factor. On NumPy < 2.2, casting a timedelta NaT between scales yields a count, so missing chunks of anm8[2us]array read back as2**62 - 1. NaT is now built in the dtype's unit and scale.μsand scaled generic units instead of excluding them.GPU buffers are not changed or validated.
Compatibility of the V2 suffix
NumPy 1.26, 2.0, 2.2, and 2.5 all parse
[Ngeneric]with the correct scale (up to2**31 - 1) and byte order. NumPy never emits this form, so a reader that re-serializes viadtype.strdrops the scale.zarr 2.18.7 cannot read these arrays correctly:
-9223372036854775808, which is what zarr writes by default. This already happens without the suffix (plain>M8), so it is not a regression.fill_value: nullit parses the dtype correctly and returns the right counts, but the result array has scale 1, so a scaled generic array is read with the wrong unit.Tests
2**31 - 1, and both byte orders, through native, V2 and V3 conversion.None/0/"NaT"fill values.The allocation,
astype, and NaT-scale regression tests fail when the corresponding fix is removed. The time and buffer tests also pass on NumPy 2.0 and 2.2.tests/test_array.py,tests/test_codecs,tests/test_buffer.py,tests/test_dtype, andtests/test_properties.pypass (6,254 passed).🤖 Generated with Claude Code