Skip to content

Keep the characters after the length number in char_dim_name - #11757

Merged
headtr1ck merged 4 commits into
pydata:mainfrom
LemonCondor:fix/char-dim-name-suffix
Oct 21, 2026
Merged

headtr1ck merged 4 commits into
pydata:mainfrom
LemonCondor:fix/char-dim-name-suffix

Conversation

@LemonCondor

@LemonCondor LemonCondor commented Oct 18, 2026 •

Copy link
Copy Markdown

Description

In xarray, a dimension name such as n3_chars was rebuilt as n3: the last number in the name was taken as the string length and everything after it was dropped. The last number is still read as the length (so the clash protection from #10395 still applies), but the characters after it are now kept. A matching name stays as is, and a mismatched one only has its number replaced (n4_chars with length 3 becomes n3_chars).

Checklist

  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst

AI Disclosure

  • This PR contains AI-generated content.
    • I have tested any AI-generated content in my PR.
    • I take responsibility for any AI-generated content in my PR.
      Tools: Claude Code. Prompt: find a reproducible bug in xarray's utilities/coding modules and propose a minimal fix with a regression test.

condor020-sudo and others added 2 commits October 18, 2026 19:45
validate_char_dim_name matched the last digit run anywhere in the name and
rebuilt the name as prefix + strlen, dropping everything after the digits.
A char dimension like "n3_chars" was therefore re-encoded as "n3", so a
decode/encode roundtrip silently renamed it. Only treat a trailing number
as the string length.

Co-authored-by: Claude <[email protected]>
@headtr1ck

Copy link
Copy Markdown
Collaborator

Thanks for the fix! The suffix was definitely getting lost, but I think the new regex goes a bit too far. A number that isn't at the end of the name is no longer treated as the length at all, so the clash protection from #10395 doesn't apply to those names anymore.

1. Name with a number in the middle, matching length (fixed by this PR)

import numpy as np
import xarray as xr

ds = xr.Dataset({"a": ("x", np.array([b"ab", b"cdef"]))})
ds.a.encoding["char_dim_name"] = "n4_chars"
ds.to_netcdf("ex1.nc")
xr.open_dataset("ex1.nc", decode_cf=False).sizes
# main: {'x': 2, 'n4': 4}
# PR:   {'x': 2, 'n4_chars': 4}

2. Two variables share the name, one is shorter (regression)

ds = xr.Dataset(
    {
        "a": ("x", np.array([b"ab", b"cdef"])),
        "b": ("x", np.array([b"ab", b"cde"])),
    }
)
ds.a.encoding["char_dim_name"] = "n4_chars"
ds.b.encoding["char_dim_name"] = "n4_chars"
ds.to_netcdf("ex2.nc")
# main: works (with a warning), dims 'n4': 4 and 'n3': 3
# PR:   ValueError: shape mismatch: objects cannot be broadcast to a single shape.
#       Mismatch is between arg 0 with shape (2, 3) and arg 1 with shape (2, 4).

3. Round trip, then shorten one variable

ds = xr.Dataset(
    {
        "a": ("x", np.array([b"ab", b"cdef"])),
        "b": ("x", np.array([b"ab", b"cdef"])),
    }
)
ds.a.encoding["char_dim_name"] = "n4_chars"
ds.b.encoding["char_dim_name"] = "n4_chars"
ds.to_netcdf("ex3.nc")
ds2 = xr.open_dataset("ex3.nc").load()
ds2["b"] = ds2.b.str[:3].astype("S")
ds2.b.encoding = {"char_dim_name": "n4_chars", "original_shape": (2, 4)}
ds2.to_netcdf("ex3b.nc")
xr.open_dataset("ex3b.nc", decode_cf=False).sizes
# main: {'x': 2, 'n4': 4, 'n3': 3}
# PR:   {'x': 2, 'n4_chars': 4, 'n4_chars3': 3}

So the question is what the dimension name should be when the length changes and the number isn't at the end. Any thoughts on what the expected behavior would be here?

Restore the previous rule that the last number in char_dim_name is the
string length, so the clash protection from pydata#10395 keeps working, and keep
the characters after that number. A mismatched "n4_chars" now becomes
"n3_chars" instead of "n3" (main) or "n4_chars3" (previous commit).
@LemonCondor

Copy link
Copy Markdown
Author

Thanks for the examples, you're right that the trailing-only regex dropped the clash protection.

I think the number should still be read as the length, wherever it is, and only the characters after it should be kept. So the last number gets replaced by the real length and the suffix stays. I pushed that in cacfc61:

  1. n4_chars, length 4: {'x': 2, 'n4_chars': 4} (unchanged name, no warning)
  2. a and b share n4_chars, b has length 3: {'x': 2, 'n4_chars': 4, 'n3_chars': 3}, with the naming-mismatch warning
  3. round trip, then shorten b: {'x': 2, 'n4_chars': 4, 'n3_chars': 3}

Only the last number is swapped, so n2_4_chars with length 3 gives n2_3_chars. I added these cases to test_CharacterArrayCoder_dim_mismatch_warnings.

@LemonCondor LemonCondor changed the title Keep full char_dim_name when it contains a non-trailing number Keep the characters after the length number in char_dim_name Oct 19, 2026

@headtr1ck headtr1ck left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@headtr1ck headtr1ck added the plan to merge Final call for comments label Oct 19, 2026
@headtr1ck
headtr1ck enabled auto-merge (squash) October 21, 2026 09:18
@headtr1ck
headtr1ck merged commit 9e2c57a into pydata:main Oct 21, 2026
62 of 63 checks passed
@welcome

welcome Bot commented Oct 21, 2026

Copy link
Copy Markdown

Congratulations on completing your first pull request! Welcome to Xarray! We are proud of you, and hope to see you again! celebration gif

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

plan to merge Final call for comments

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants