Skip to content

ARROW-15277: [Python] Use Make to create ChunkedArray and remove checks - #12096

Closed
edponce wants to merge 5 commits into
apache:masterfrom
edponce:ARROW-15277-Use-Make-to-create-ChunkedArray-and-remo
Closed

edponce wants to merge 5 commits into
apache:masterfrom
edponce:ARROW-15277-Use-Make-to-create-ChunkedArray-and-remo

Conversation

@edponce

@edponce edponce commented Jan 7, 2022 •

Copy link
Copy Markdown
Contributor

Replace Python explicit type/size checks for ChunkedArray with C++ ChunkedArray::Make().

@github-actions

github-actions Bot commented Jan 7, 2022

Copy link
Copy Markdown

@edponce
edponce marked this pull request as draft January 7, 2022 05:40
@edponce
edponce force-pushed the ARROW-15277-Use-Make-to-create-ChunkedArray-and-remo branch from d38f5ba to cbccfdb Compare April 22, 2022 00:59
@edponce
edponce marked this pull request as ready for review April 22, 2022 01:00
@edponce

edponce commented Apr 22, 2022

Copy link
Copy Markdown
Contributor Author

@jorisvandenbossche @pitrou I would appreciate your review on this PR when you get a chance.

@edponce
edponce force-pushed the ARROW-15277-Use-Make-to-create-ChunkedArray-and-remo branch 2 times, most recently from 2285758 to 4542cc7 Compare April 22, 2022 01:53
@pitrou

pitrou commented Apr 25, 2022

Copy link
Copy Markdown
Member

@edponce Can you rebase to try and get a clean CI build?

Comment thread python/pyarrow/tests/test_compute.py Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why you're changing this test?

@edponce edponce Apr 25, 2022 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In C++, a ChunkedArray has to have all Arrays of the same type, but the Python API allowed coercion of None to the first inferred type

["a", None] --> StringType
[None] --> NullType
[] --> NullType

Maybe we should allow such coercion in C++ as well.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this could be a slight API break. @jorisvandenbossche What do you think?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me double check carefully the current C++ and Python behavior before jumping to conclusions.

Comment thread python/pyarrow/table.pxi Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, are the error conditions checked by the test suite?

Comment thread python/pyarrow/includes/libarrow.pxd Resolved
@edponce
edponce force-pushed the ARROW-15277-Use-Make-to-create-ChunkedArray-and-remo branch from c2fc63f to 3e6f309 Compare April 25, 2022 10:54
@pitrou

pitrou commented Aug 17, 2022

Copy link
Copy Markdown
Member

@milesgranger Would you like to take this up perhaps?

pitrou added a commit that referenced this pull request Aug 25, 2022
zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants