.str.split coerces dtype to object

Describe the issue:

The .str.split method coerces dtype from string to object. This behavior is also inconsistent with pandas (if that matters).

Minimal complete verifiable example:

Code

import dask.dataframe as dd
import pandas as pd

data = {"c": ["a,b,c", "d,e,f", "g,h,i"]}

# `pandas`
df = pd.DataFrame(data, dtype="string[pyarrow]")
print(df.dtypes)
print(df["c"].str.split(",", n=1, expand=True).dtypes)

# `dask`
ddf = dd.from_pandas(df)
print(ddf.dtypes)
print(ddf["c"].str.split(",", n=1, expand=True).dtypes)

Output

c    string[pyarrow]
dtype: object
0    string[pyarrow]
1    string[pyarrow]
dtype: object
c    string[pyarrow]
dtype: object
0    object
1    object
dtype: object

Environment:

Dask version: 2025.3.0
Python version: 3.10.16
Operating System: Ubuntu 24.04.2 LTS
Install method (conda, pip, source): pip

Note that after computing, the dtypes are correct:

In [6]: ddf["c"].str.split(",", n=1, expand=True).compute().dtypes
Out[6]: 
0    string[pyarrow]
1    string[pyarrow]
dtype: object

so it's just the _meta after .str.split.

dask/dask/dataframe/dask_expr/_str_accessor.py

Lines 186 to 189 in 0fa5e18

    
           meta = self.frame._meta._constructor( 
        
               [delimiter.join(["a"] * (self.n + 1))], 
        
               index=meta.iloc[:1].index, 
        
           )

looks a bit suspicious. I haven't stepped through it, but it seems like we're doing pd.Series(['list', 'of', 'strings']) which pandas will infer as object. Maybe we need to pass a dtype there? Mind taking a look?

Hey! I’ve prepared a fix for this issue and confirmed that it passes tests (including preservation of string[pyarrow] dtype). I’ll open a PR shortly

@TomAugspurger

@TomAugspurger @Tunahanyrd — happy to pick this up if it's still available. The fix scope looks clear from Tom's diagnosis above.

Preserve input dtype in str.split(expand=True) metadata

Fix: Preserve dtype in str.split(expand=True) (Fixes dask#11884)

@noahblakesmith

hii @noahblakesmith @TomAugspurger please review #12560 the pr.

Hi! I'd like to work on this issue. Please let me know if I can be assigned. Thanks!

Hi! I'd like to look into the str.split dtype coercion issue. I'll investigate and submit a fix. Thanks!

Hi! I'd like to work on this. I'll reproduce, identify the root cause, implement a minimal fix and add a regression test. Please let me know if I can proceed.

Navigation Menu

Repository navigation

`.str.split` coerces `dtype` to `object` #11884

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	meta = self.frame._meta._constructor(
	[delimiter.join(["a"] * (self.n + 1))],
	index=meta.iloc[:1].index,
	)

.str.split coerces dtype to object #11884

Description

Activity

TomAugspurger commented on Apr 11, 2025

Tunahanyrd commented on May 19, 2025

DresdenGman commented on Jul 10, 2026

Rajkaran-122 commented on Aug 22, 2026

LeonxLJX commented on Aug 27, 2026

LeonxLJX commented on Aug 31, 2026

LeonxLJX commented on Sep 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`.str.split` coerces `dtype` to `object` #11884