DEP: Deprecate silent ignoring of bad data in fromfile/fromstring by seberg · Pull Request #13605 · numpy/numpy

seberg · 2019-05-22T19:05:04Z

When the read functions reach bad data the loop is stopped. This adds
a check whether the file/string was actually read to its end, and
if not gives a DeprecationWarning.

Closes gh-6076

Thought it would close more issues, I am not a specialist on file objects, but since it is here and probably works...

eric-wieser · 2019-06-08T05:48:31Z

C code looks good now, although tests nits still apply.

CI is failing with:

=================================== FAILURES ===================================
________ TestCommaDecimalPointLocale.test_fromstring_best_effort_float _________

self = <numpy.core.tests.test_longdouble.TestCommaDecimalPointLocale object at 0xf057ac0c>

    def test_fromstring_best_effort_float(self):
>       assert_equal(np.fromstring("1,234", dtype=float, sep=" "),
                     np.array([1.]))
E       DeprecationWarning: string or file could not be read to its end due to unmatched data; this will raise an error in the future.

self       = <numpy.core.tests.test_longdouble.TestCommaDecimalPointLocale object at 0xf057ac0c>

/usr/local/lib/python3.6/dist-packages/numpy/core/tests/test_longdouble.py:195: DeprecationWarning
___________ TestCommaDecimalPointLocale.test_fromstring_best_effort ____________

self = <numpy.core.tests.test_longdouble.TestCommaDecimalPointLocale object at 0xf03bae2c>

    def test_fromstring_best_effort(self):
>       assert_equal(np.fromstring("1,234", dtype=np.longdouble, sep=" "),
                     np.array([1.]))
E       DeprecationWarning: string or file could not be read to its end due to unmatched data; this will raise an error in the future.

self       = <numpy.core.tests.test_longdouble.TestCommaDecimalPointLocale object at 0xf03bae2c>

/usr/local/lib/python3.6/dist-packages/numpy/core/tests/test_longdouble.py:199: DeprecationWarning
__________ TestCommaDecimalPointLocale.test_fromstring_foreign_value ___________

self = <numpy.core.tests.test_longdouble.TestCommaDecimalPointLocale object at 0xf058d4ec>

    def test_fromstring_foreign_value(self):
>       b = np.fromstring("1,234", dtype=np.longdouble, sep=" ")
E       DeprecationWarning: string or file could not be read to its end due to unmatched data; this will raise an error in the future.

self       = <numpy.core.tests.test_longdouble.TestCommaDecimalPointLocale object at 0xf058d4ec>

/usr/local/lib/python3.6/dist-packages/numpy/core/tests/test_longdouble.py:213: DeprecationWarning
--------- generated xml file: /numpy/build/test/junit/test-results.xml ---------
=========================== short test summary info ============================

Would be nice to finish this up.

seberg · 2019-06-08T19:49:21Z

Still not sure which version we would put this in. There was still a hairy issue with separator being invalid, and if the count was correct. Those should be fixed, but may warrant a bit closer look.

eric-wieser · 2019-06-13T05:05:23Z

Same CI failures as I mention above, should be as simple as adding assert_warns to those locations

seberg · 2019-06-13T14:46:46Z

Ack, yes sorry. I did not realize those tests were skipped because they need french locale.

seberg · 2019-06-13T15:34:51Z

Test failures now look like a CI issue.

tylerjereddy · 2019-06-13T17:30:47Z

Matti & I have seen that Azure issue before. It is sporadic, but perhaps more common as of late. Ideally, we'd not need to keep pulling in mingw from the internet to get gcc / gfortran runtime libs of a rather specific version.

It is possible to remove those dependencies upstream, but that discussion didn't go so well re: static links. Perhaps Microsoft might expand the breadth of mingw local install versions available some day or we might just host the file somewhere more reliable / use another manager to pull it in.

Not sure about LGTM issue offhand--looks like they've been doing some overhauls lately.

seberg · 2019-07-24T22:35:04Z

Added release notes.

mattip · 2019-09-12T06:44:33Z

@seberg a few last cleanups

seberg · 2019-09-12T16:08:21Z

Sorry, forgot. Fixed the last three small comments and squashed.

When the read functions reach bad data the loop is stopped. This adds a check whether the file/string was actually read to its end, and if not gives a DeprecationWarning. Closes numpygh-6076

seberg added 00 - Bug component: numpy._core 07 - Deprecation 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes labels May 22, 2019

mattip reviewed May 23, 2019

View reviewed changes

Comment thread numpy/core/src/multiarray/ctors.c Resolved

seberg force-pushed the fromfile-eof-deprecation branch from 37b5e44 to f9c3197 Compare May 23, 2019 05:22

eric-wieser reviewed May 23, 2019

View reviewed changes

Comment thread numpy/core/tests/test_multiarray.py Resolved

Comment thread numpy/core/tests/test_multiarray.py Resolved

Comment thread numpy/core/src/multiarray/ctors.c Resolved

seberg force-pushed the fromfile-eof-deprecation branch 2 times, most recently from 9227d04 to b2c5f6c Compare May 24, 2019 00:06

eric-wieser reviewed May 26, 2019

View reviewed changes

Comment thread numpy/core/src/multiarray/_multiarray_tests.c.src Resolved

eric-wieser reviewed May 26, 2019

View reviewed changes

Comment thread numpy/core/src/multiarray/ctors.c Resolved

eric-wieser reviewed May 26, 2019

View reviewed changes

Comment thread numpy/core/src/multiarray/ctors.c Resolved

seberg force-pushed the fromfile-eof-deprecation branch 2 times, most recently from facf937 to f92823b Compare June 1, 2019 00:13

eric-wieser reviewed Jun 8, 2019

View reviewed changes

Comment thread numpy/core/src/multiarray/ctors.c Resolved

seberg force-pushed the fromfile-eof-deprecation branch from f92823b to dae1540 Compare June 8, 2019 19:47

seberg commented Jun 13, 2019

View reviewed changes

Comment thread numpy/core/tests/test_deprecations.py Resolved

seberg force-pushed the fromfile-eof-deprecation branch 2 times, most recently from f947373 to 1524d06 Compare July 24, 2019 22:35

seberg removed the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label Jul 24, 2019

seberg mentioned this pull request Aug 20, 2019

Unexpected behaviour numpy.fromstring #11878

Closed

seberg force-pushed the fromfile-eof-deprecation branch from 1524d06 to eb712bd Compare August 20, 2019 15:06

zjpoh mentioned this pull request Aug 21, 2019

ENH: Parse complex number from string #14227

Merged

mattip reviewed Aug 21, 2019

View reviewed changes

Comment thread numpy/core/tests/test_deprecations.py Resolved

eric-wieser reviewed Aug 21, 2019

View reviewed changes

Comment thread numpy/core/src/multiarray/_multiarray_tests.c.src Resolved

eric-wieser reviewed Aug 21, 2019

View reviewed changes

Comment thread numpy/core/src/multiarray/ctors.c Resolved

seberg force-pushed the fromfile-eof-deprecation branch from eb712bd to 83fed9c Compare September 12, 2019 16:04

DEP: Deprecate silent ignoring of bad data in fromfile/fromstring

bf6809a

When the read functions reach bad data the loop is stopped. This adds a check whether the file/string was actually read to its end, and if not gives a DeprecationWarning. Closes numpygh-6076

seberg force-pushed the fromfile-eof-deprecation branch from 83fed9c to bf6809a Compare September 12, 2019 23:43

mattip merged commit 79ee678 into numpy:master Sep 13, 2019

seberg deleted the fromfile-eof-deprecation branch September 13, 2019 06:29

mattip mentioned this pull request Sep 24, 2019

BUG: dtype refcount cleanups #14586

Merged

Conversation

seberg commented May 22, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eric-wieser commented Jun 8, 2019

Uh oh!

seberg commented Jun 8, 2019

Uh oh!

eric-wieser commented Jun 13, 2019

Uh oh!

seberg commented Jun 13, 2019

Uh oh!

seberg commented Jun 13, 2019

Uh oh!

Uh oh!

tylerjereddy commented Jun 13, 2019

Uh oh!

seberg commented Jul 24, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mattip commented Sep 12, 2019

Uh oh!

seberg commented Sep 12, 2019

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants