Avoid AddTraceback if stringtab isn't set up #4378
Conversation
This can happen (rarely) with exceptions very early in the module init process. Fixes cython#4377 I also cleaned up a few inline functions to have return statements. I'm struggling to see how to add a test for this (so I've omitted it) but happy to take any suggestions
|
FWIW, we could try to set up an empty virtualenv and run a test with it activated |
Use a make Numpy module to make a version of "import_array" that always fails
There was a problem hiding this comment.
I think a better fix would 1) reorder the module init code (why is numpy even getting imported before initialising the stringtab????) and 2) use a second error label for early errors, even if that only ends up setting a flag to skip the traceback.
Edit: not sure if a second error label is required – a global flag that is set at some point down the code might also work. But a separate error label seems a bit cleaner and more cythonic.
Cython/Includes/numpy/__init__.pxd
Outdated
| cdef inline int import_array() except -1: | ||
| try: | ||
| __pyx_import_array() | ||
| return __pyx_import_array() |
There was a problem hiding this comment.
I don't think this is necessary. And note that the NumPy project owns this file now. We shouldn't apply changes on our side without pushing them there first.
There was a problem hiding this comment.
I've undone this - while I think it would have been clearer I don't think it's worth the effort of changing in two places
Cython/Compiler/ModuleNode.py
Outdated
| # We can run into errors before the stringtab is initialize. | ||
| # In this case it is not safe to add a traceback (because it uses the stringtab) |
There was a problem hiding this comment.
It doesn't globally use the stringtab, though, just one specific entry from it: the function name that we pass below.
We currently use char* for the function name and file path, which is admittedly suboptimal, but could simplify this code since we don't actually need the stringtab for that. I'm unsure, though, because I'd rather like to use prepared Python strings than char* in the future.
There was a problem hiding this comment.
It's actually __pyx_n_s_cline_in_traceback in __Pyx_CLineForTraceback that it uses (i.e. it's nested pretty deep in the traceback generating code, and I wouldn't like to guarantee that it's the only one).
There was a problem hiding this comment.
Good to know. Still, my general review comment still applies that there's a cleaner way to do this.
to avoid checking if entire stringtab is initialized
Module init code is now re-ordered so "Init" utiltiy code comes at the end of the function once all the Cython constants are set up.
I've gone for the global flag approach because I thought it fitted better with how the code was already organised (there was a check to see if the module dict was initialized before the traceback anyway). |
scoder
left a comment
scoder
left a comment
There was a problem hiding this comment.
The flag is fine, the test is very nice.
Cython/Compiler/Code.py
Outdated
| 'init_globals', # init_globals generates an init_globals_body insertion point | ||
| # that comes before the utility code |
There was a problem hiding this comment.
Why not use separate functions (and sections) for these? E.g. InitGlobals, InitStrings, InitConstants, or something along those lines. There isn't really a reason why they should all share the same section, except for laziness. And that now gets in the way.
There was a problem hiding this comment.
Right - I've split this into two init_globals (defines function __Pyx_InitGlobals) and init_constants (defines function __Pyx_InitConstants). The main reason I didn't split it into three with a separate InitStrings function is that there was already a __Pyx_InitStrings function, and I was struggling to come up with a clear name to distinguish them.
That's still slightly lazy, but I think the main thing is that the utility code has its own section.
da-woods
changed the title
|
Yes, that's very nice. |
commit 1461e51 Author: Stefan Behnel <[email protected]> Date: Mon Oct 25 14:29:02 2021 +0200 Clean up the NumPy integration test by moving the doctests into the functions that they test. commit 68bb716 Author: Stefan Behnel <[email protected]> Date: Mon Oct 25 14:14:24 2021 +0200 Remove dead test code. commit 4a74678 Author: Stefan Behnel <[email protected]> Date: Sun Oct 24 21:27:44 2021 +0200 Use newer test dependencies in Py3.6+. (Excluding 3.10 for now to give the projects a bit more time.) commit 9d1ffd5 Author: da-woods <[email protected]> Date: Sun Oct 24 23:13:50 2021 +0100 Initial support for Python 3.11 (cythonGH-4414) * Add a basic replacement for PyCode_New(). An optimized versions would be nice, but this is intended to work sufficiently to start testing. Also, CPython 3.11 might actually add a new C-API function to simplify setting the current code position. That might be used instead. * Disable introspection of frame object with vectorcall This feature looked to only be used for early Python versions that don't have the full vectorcall protocol (and the contents of the frame object are changed in Python 3.11). commit 346c81f Author: Stefan Behnel <[email protected]> Date: Sun Oct 24 21:18:10 2021 +0200 Make sure that version dependent special methods are correctly and completely excluded via preprocessor guards. Previously, implementing "__div__" could fail in Py3 (if the code for adapting the Python wrapper was generated) or would at least generate C compiler warnings about unused "__div__" C functions. commit 3748c3c Author: Stefan Behnel <[email protected]> Date: Sun Oct 24 13:22:34 2021 +0200 Add Py3.10 as CI test target. commit 0f84a57 Author: Max Bachmann <[email protected]> Date: Sat Oct 23 22:06:37 2021 +0200 Update incorrect version support comment for pycapsule.pxd (cythonGH-4426) commit c25c87d Author: Dobatymo <[email protected]> Date: Sat Oct 23 03:33:02 2021 +0800 Fix libcpp map/set/multiset/unordered type issues (cythonGH-4410) Fix insert return types, constness and input iterator templates. Fix typing in iterators and add constructor to allow explicit conversion from iterator to const_iterator. commit f776da0 Author: Dobatymo <[email protected]> Date: Sat Oct 23 03:28:53 2021 +0800 Add C++ multimap/unordered_multimap (cythonGH-4419) commit c83fd44 Author: 0dminnimda <[email protected]> Date: Fri Oct 22 21:44:11 2021 +0300 Introduce new shell syntax for ci-run.sh to improve Windows support (cythonGH-4400) commit c8c9a12 Merge: 174ca03 f53ac52 Author: Stefan Behnel <[email protected]> Date: Thu Oct 21 19:02:11 2021 +0200 Merge branch '0.29.x' commit f53ac52 Author: Stefan Behnel <[email protected]> Date: Tue May 25 11:20:54 2021 +0200 docs: Use the Cython + IPython lexers that come with Pygments to avoid having to maintain our own ones. commit 174ca03 Author: account-login <[email protected]> Date: Wed Oct 20 17:03:20 2021 +0800 Add some missing functions to libcpp maps and string (cythonGH-4395) * add swap() to libcpp.string * add load_factor() to libcpp.unordered_map and libcpp.unordered_set commit 42a4af2 Merge: 53b0eb2 fb5d29e Author: Stefan Behnel <[email protected]> Date: Mon Oct 18 20:38:22 2021 +0200 Merge branch '0.29.x' commit fb5d29e Author: da-woods <[email protected]> Date: Mon Oct 18 19:36:54 2021 +0100 Fix tracing after adapting it to Py3.11 (cythonGH-4420) commit 53b0eb2 Author: da-woods <[email protected]> Date: Mon Oct 18 19:36:54 2021 +0100 Fix tracing after adapting it to Py3.11 (cythonGH-4420) commit 5f820ed Author: da-woods <[email protected]> Date: Mon Oct 18 11:10:05 2021 +0100 Fix fused.__self__ tests on PyPy (cythonGH-4417) PyPy v7.3.6 looks to have added a helpful "did you mean..." to the AttributeError exception. It's currently tripping up these tests. commit 4df1103 Merge: f6eeeda cbddad2 Author: Stefan Behnel <[email protected]> Date: Mon Oct 18 12:05:10 2021 +0200 Merge branch '0.29.x' commit cbddad2 Author: Victor Stinner <[email protected]> Date: Mon Oct 18 12:03:17 2021 +0200 Make Profile.c use PyThreadState_EnterTracing() (cythonGH-4411) Instead of __Pyx_SetTracing(), Profile.c now uses PyThreadState_EnterTracing() and PyThreadState_LeaveTracing(), which were added to Python 3.11.0a2: python/cpython#28542 When these functions are used, Cython no longer sets directly PyThreadState.cframe.use_tracing. commit f6eeeda Author: da-woods <[email protected]> Date: Sun Oct 17 19:01:52 2021 +0100 Fix fused cpdef default arguments (cythonGH-4413) A couple of things were going wrong: * they're creating CloneNodes (but not requiring the contents of the clone of the clone node to be temp) * assignment from a clone node generates cleanup code (which is against the general rules of a clone node), and also loses a reference via giveref * cpdef functions cause a small memory leak (cython#4412) by assigning to their constants twice. This is unfortunately difficult to test for. With this patch we no longer leak, but still duplicate a little bit of work. commit a0571a6 Author: da-woods <[email protected]> Date: Sun Oct 17 18:51:08 2021 +0100 Import TextTestResult in test runner instead of _TextTestResult (cythonGH-4415) All the versions we currently test are new enough that the alias is no longer necessary. commit c129b15 Author: Dobatymo <[email protected]> Date: Fri Oct 15 16:31:32 2021 +0800 Fix wrong type in unordered_multiset::swap() (cythonGH-4408) commit 72c18e7 Author: 0dminnimda <[email protected]> Date: Thu Oct 7 10:56:43 2021 +0300 Improve ci-run.sh (cythonGH-4398) commit 454a498 Author: da-woods <[email protected]> Date: Wed Oct 6 07:16:08 2021 +0100 Improve "import_array" guard (cythonGH-4397) Stop using NPY_NDARRAYOBJECT_H since: a) in principle it's private b) Numpy has renamed it and use a public symbol instead. I think the existing tests are adequate - we just aren't yet testing against a new enough version of Numpy to have caught it yet. Closes cython#4396 Closes cython#4394 commit 97c05e7 Author: Stefan Behnel <[email protected]> Date: Sat Oct 2 11:08:43 2021 +0200 Make a compile test runnable. commit 8c7b0f3 Author: da-woods <[email protected]> Date: Fri Oct 1 10:29:34 2021 +0100 Handle function "outer_attrs" more consistently (cythonGH-4375) A few children of function nodes need to be consistently evaluated outside the function scope. This PR attempts to do so and thus fixes cython#4367. commit 494f517 Author: da-woods <[email protected]> Date: Fri Oct 1 10:22:36 2021 +0100 Change gcc version check in test runner to a numeric comparison (cythonGH-4359) The string comparison was reporting '11' < '4' (so OpenMP tests were being skipped on GCC 11) commit cce3693 Author: Christian Clauss <[email protected]> Date: Wed Sep 29 09:49:45 2021 +0200 Fix typo discovered by codespell (cython#4387) commit daa0a44 Author: da-woods <[email protected]> Date: Tue Sep 28 08:48:12 2021 +0100 Fix the name of attributes in the common ABI module (cythonGH-4376) Attribute names used to be fully qualified like "_cython_3_0_0a9.cython_function_or_method" instead of the plain name. Closes cython#4373 commit fa8db66 Author: da-woods <[email protected]> Date: Mon Sep 27 10:11:12 2021 +0100 Avoid AddTraceback() if stringtab isn't set up (cythonGH-4378) This can happen (rarely) with exceptions that occur very early in the module init process. Fixes cython#4377 Uses a fake Numpy module for testing to make a version of "import_array" that always fails. commit f94f26a Author: da-woods <[email protected]> Date: Mon Sep 27 09:58:20 2021 +0100 Make __Pyx_CoroutineAwaitType non-pickleable (cythonGH-4381) This is explicitly tested for: https://github.com/cython/cython/blob/aea4e6b84b38223c540266f8c57093ee2039f284/tests/run/test_coroutines_pep492.pyx#L2400 It turns out some earlier versions of Python assume that C-API classes without a dict or slot are pickleable by the class name. Currently it isn't pickleable because the class name lookup is failing but this change makes it more robust. See cython#4376 commit 7403055 Author: da-woods <[email protected]> Date: Fri Sep 24 11:05:52 2021 +0100 Avoid unnecessary binding of fused functions on class lookup (cythonGH-4370) Among other things this makes it pickleable by ensuring that it's the same object each time. commit e2a23fe Author: da-woods <[email protected]> Date: Mon Sep 20 08:34:34 2021 +0100 Remove usused "FetchCommonPointer" utility code (cythonGH-4380)
3 participants
This can happen (rarely) with exceptions very early in the
module init process.
Fixes #4377
I also cleaned up a few inline functions to have return statements.
I'm struggling to see how to add a test for this (so I'veTest is included nowomitted it) but happy to take any suggestions