Skip to content

MAINT: Remove __setslice__ and __getslice__ - #8592

Merged
charris merged 4 commits into
numpy:masterfrom
eric-wieser:remove-setslice
Feb 10, 2017
Merged

charris merged 4 commits into
numpy:masterfrom
eric-wieser:remove-setslice

Conversation

@eric-wieser

@eric-wieser eric-wieser commented Feb 9, 2017 •

Copy link
Copy Markdown
Member

This code was only here for Python 2.5 compatibility, but numpy requires 2.7 at minimum.

This involves:

  • Removing all the __[gs]etslice__ method from python files
  • Removing the sq_slice and sq_slice_ass slots from C files, so that the python files do not have to override the base implementations

@eric-wieser
eric-wieser force-pushed the remove-setslice branch 4 times, most recently from 28040f0 to 1a7fe64 Compare February 9, 2017 14:20
This code was only here for Python 2.5 compatibility, but numpy requires 2.7
at minimum
MaskedArray.__setitem__
MaskedArray.__delitem__
MaskedArray.__getslice__
MaskedArray.__setslice__

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that getslice and setslice are also in the index. Grep for the terms.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, but without the __. I was under the impression that was just a coincidental reference to something else. If that's not the case, I'll wipe that one too

@charris

charris commented Feb 9, 2017

Copy link
Copy Markdown
Member

I'm a bit concerned about backward compatibility, note the reference from @mhvk. Both Pandas and SciPy look good, however, so maybe not worth worrying about.

@charris

charris commented Feb 9, 2017

Copy link
Copy Markdown
Member

Needs a mention in the 1.13.0 release notes in the compatibility section.

@eric-wieser

Copy link
Copy Markdown
Member Author

I'm not convinced this would have any compatibility issues beyond super().__setslice__ failing

@eric-wieser

Copy link
Copy Markdown
Member Author

I'll do some experiments to characterize exactly what the effect is

@eric-wieser

eric-wieser commented Feb 9, 2017 •

Copy link
Copy Markdown
Member Author

@charris: Changelog added. Compatibility definitely only affects Python 2.7. Experiments so far:

  • Forgetting to implement MyClass.__getslice__ on ndarray subclasses would cause ndarray.__getslice__ to be called

    • Now: MyClass.__getitem__ is called instead, which is already the py3 behaviour
  • If __getslice__ is implemented on an ndarray subclass, it is still invoked in favor of __getitem__ if both exist

  • The above only applies because ndarray is a C-api class. If it were a pure-python class, that wouldn't be the case.

.. index::
triple: ndarray; special methods; getslice
triple: ndarray; special methods; setslice
triple: ndarray; special methods; get slice

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be an ndarray method. getitem and setitem are probably what should be here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not __getitem__ and __setitem__ ?

Replaces getslice and setslice which have been removed.
@charris

charris commented Feb 10, 2017

Copy link
Copy Markdown
Member

Made a small fix and will commit when tests finish. Yeah, I forgot [ci skip] :(

@eric-wieser

Copy link
Copy Markdown
Member Author

@charris: didn't know you could edit the PR! Is that new?

@charris

charris commented Feb 10, 2017

Copy link
Copy Markdown
Member

No, not new but not often used. You can disable it when you make the PR, there is a button somewhere...

@charris
charris merged commit 1b87725 into numpy:master Feb 10, 2017
@charris

charris commented Feb 10, 2017

Copy link
Copy Markdown
Member

Thanks @eric-wieser .

@charris

charris commented Feb 11, 2017

Copy link
Copy Markdown
Member

@pv Looks like that. Does that show up in Python3? If not, I think it indicates that getitem needs some improvement.

@seberg

seberg commented Feb 12, 2017

Copy link
Copy Markdown
Member

Frankly I doubt there is much you can do (easily). Slicing is a pretty fast operation, so simply creating a few extra python objects and more complex getitem logic may already add a substantial slowdown. But maybe its worth a look....

@eric-wieser

eric-wieser commented Feb 12, 2017 •

Copy link
Copy Markdown
Member Author

Perhaps there's an argument for __getitem__ trying to speed up the case when the input is a single slice object, at the expense of more complex slices

Py_DECREF(ret);
return NULL;
}
PyArray_UpdateFlags(ret, NPY_ARRAY_UPDATE_ALL);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this line exist here but not in get_view_from_index?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, no idea why this should be needed here. Maybe it was just being very careful, or a git blame might help to see when the line was added if it was. The array creation should set all the flags correctly already, except maybe writable, but that is correct.

@eric-wieser

Copy link
Copy Markdown
Member Author

Another bonus of killing this:

#python 2.7
>>> x = np.array(0)
>>> x[-1:]
TypeError: len() of unsized object
>>> x[np.s_[-1:]]
IndexError: too many indices for array

@juliantaylor

Copy link
Copy Markdown
Contributor

this change needs to be reverted.
There are programs out there calling setslice. E.g. astropy is massively broken.

we do not break backward compatibility for cosmetic reasons!

juliantaylor added a commit to juliantaylor/numpy that referenced this pull request Apr 14, 2017
This reverts commit 994f8a1 and
bec587c from numpygh-8592.

This is essentially a cosmetic change that breaks backward compatibility
with in use programs like astropy 1.3.
@charris charris changed the title MAINT: Remove __setslice__ and __getslice__ MAINT: Remove __setslice__ and __getslice__ May 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants