PR#3: Python 3.11 support - rpms/python-greenlet

rpms / python-greenlet

#3 Python 3.11 support

Merged 3 years ago by churchyard. Opened 3 years ago by churchyard.

rpms/ churchyard/python-greenlet py3.11 into rawhide

Python 3.11 support

Miro Hrončok • 3 years ago

dcac805

306.patch

file added

+238

		`@@ -0,0 +1,238 @@`
		`+ From 31ccde2be3eeefca06277e20c3a06022e5ab2e62 Mon Sep 17 00:00:00 2001`
		`+ From: Victor Stinner <[email protected]>`
		`+ Date: Mon, 6 Jun 2022 16:14:52 +0200`
		`+ Subject: [PATCH] Closes #305: Add Python 3.11 support`
		`+ MIME-Version: 1.0`
		`+ Content-Type: text/plain; charset=UTF-8`
		`+ Content-Transfer-Encoding: 8bit`
		`+`
		`+ * Add GREENLET_PY311 macro`
		`+ * PyGreenlet structure:`
		`+`
		`+ * Add 3 members for the "data stack": 'datastack_chunk',`
		`+ 'datastack_top' and 'datastack_limit'.`
		`+ * Add 'current_frame' member.`
		`+`
		`+ * Rename CFrame to _PyCFrame`
		`+ * tox.ini: Add py311 environment.`
		`+`
		`+ Changes partially backport from the master branch:`
		`+ commit 63e1099acc3677e614532bea0fa2e1967b69125f.`
		`+`
		`+ Co-Authored-By: Miro Hrončok <[email protected]>`
		`+ ---`
		`+ src/greenlet/greenlet.c \| 61 ++++++++++++++++++++++++++++++++---------`
		`+ src/greenlet/greenlet.h \| 17 +++++++++++-`
		`+ tox.ini \| 2 +-`
		`+ 3 files changed, 65 insertions(+), 15 deletions(-)`
		`+`
		`+ diff --git a/src/greenlet/greenlet.c b/src/greenlet/greenlet.c`
		`+ index f47bbf88..2f3ad6e9 100644`
		`+ --- a/src/greenlet/greenlet.c`
		`+ +++ b/src/greenlet/greenlet.c`
		`+ @@ -170,9 +170,11 @@ green_clear_exc(PyGreenlet* g)`
		`+ {`
		`+ #if GREENLET_PY37`
		`+ g->exc_info = NULL;`
		`+ - g->exc_state.exc_type = NULL;`
		`+ g->exc_state.exc_value = NULL;`
		`+ +#if !GREENLET_PY311`
		`+ + g->exc_state.exc_type = NULL;`
		`+ g->exc_state.exc_traceback = NULL;`
		`+ +#endif`
		`+ g->exc_state.previous_item = NULL;`
		`+ #else`
		`+ g->exc_type = NULL;`
		`+ @@ -525,8 +527,13 @@ g_switchstack(void)`
		`+ { /* save state */`
		`+ PyGreenlet* current = ts_current;`
		`+ PyThreadState* tstate = PyThreadState_GET();`
		`+ +#if GREENLET_PY311`
		`+ + current->recursion_depth = (tstate->recursion_limit`
		`+ + - tstate->recursion_remaining);`
		`+ +#else`
		`+ current->recursion_depth = tstate->recursion_depth;`
		`+ current->top_frame = tstate->frame;`
		`+ +#endif`
		`+ #if GREENLET_PY37`
		`+ current->context = tstate->context;`
		`+ #endif`
		`+ @@ -551,6 +558,15 @@ g_switchstack(void)`
		`+ */`
		`+ current->cframe = tstate->cframe;`
		`+ ts__g_switchstack_use_tracing = tstate->cframe->use_tracing;`
		`+ +#if GREENLET_PY311`
		`+ + current->current_frame = tstate->cframe->current_frame;`
		`+ + current->datastack_chunk = tstate->datastack_chunk;`
		`+ + current->datastack_top = tstate->datastack_top;`
		`+ + current->datastack_limit = tstate->datastack_limit;`
		`+ + PyFrameObject *frame = PyThreadState_GetFrame(tstate);`
		`+ + Py_XDECREF(frame); /* PyThreadState_GetFrame gives us a new reference. */`
		`+ + current->top_frame = frame;`
		`+ +#endif`
		`+ #endif`
		`+ }`
		`+`
		`+ @@ -574,9 +590,6 @@ g_switchstack(void)`
		`+ PyGreenlet* target = ts_target;`
		`+ PyGreenlet* origin = ts_current;`
		`+ PyThreadState* tstate = PyThreadState_GET();`
		`+ - tstate->recursion_depth = target->recursion_depth;`
		`+ - tstate->frame = target->top_frame;`
		`+ - target->top_frame = NULL;`
		`+`
		`+ #if GREENLET_PY37`
		`+ tstate->context = target->context;`
		`+ @@ -607,7 +620,18 @@ g_switchstack(void)`
		`+ */`
		`+ tstate->cframe->use_tracing = ts__g_switchstack_use_tracing;`
		`+ #endif`
		`+ -`
		`+ +#if GREENLET_PY311`
		`+ + tstate->recursion_remaining = (tstate->recursion_limit`
		`+ + - target->recursion_depth);`
		`+ + tstate->cframe->current_frame = target->current_frame;`
		`+ + tstate->datastack_chunk = target->datastack_chunk;`
		`+ + tstate->datastack_top = target->datastack_top;`
		`+ + tstate->datastack_limit = target->datastack_limit;`
		`+ +#else`
		`+ + tstate->recursion_depth = target->recursion_depth;`
		`+ + tstate->frame = target->top_frame;`
		`+ +#endif`
		`+ + target->top_frame = NULL;`
		`+ assert(ts_origin == NULL);`
		`+ Py_INCREF(target);`
		`+ ts_current = target;`
		`+ @@ -810,7 +834,7 @@ static int GREENLET_NOINLINE(g_initialstub)(void* mark)`
		`+ We want to defer copying the state info until we're sure`
		`+ we need it and are in a stable place to do so.`
		`+ */`
		`+ - CFrame trace_info;`
		`+ + _PyCFrame trace_info;`
		`+ #endif`
		`+ /* save exception in case getattr clears it */`
		`+ PyErr_Fetch(&exc, &val, &tb);`
		`+ @@ -875,7 +899,12 @@ static int GREENLET_NOINLINE(g_initialstub)(void* mark)`
		`+ }`
		`+ self->top_frame = NULL;`
		`+ green_clear_exc(self);`
		`+ +#if GREENLET_PY311`
		`+ + self->recursion_depth = (PyThreadState_GET()->recursion_limit`
		`+ + - PyThreadState_GET()->recursion_remaining);`
		`+ +#else`
		`+ self->recursion_depth = PyThreadState_GET()->recursion_depth;`
		`+ +#endif`
		`+`
		`+ /* restore arguments in case they are clobbered */`
		`+ ts_target = self;`
		`+ @@ -1006,13 +1035,13 @@ green_new(PyTypeObject* type, PyObject* args, PyObject* kwds)`
		+ it uses the ``root_cframe`` just to have something to put there.
		`+ However, once the greenlet is actually switched to for the first`
		+ time, ``g_initialstub`` (which doesn't actually "return" while the
		`+ - greenlet is running) stores a new CFrame on its local stack, and`
		`+ + greenlet is running) stores a new _PyCFrame on its local stack, and`
		`+ copies the appropriate values from the currently running CFrame;`
		`+ - this is then made the CFrame for the newly-minted greenlet.`
		`+ + this is then made the _PyCFrame for the newly-minted greenlet.`
		+ ``g_initialstub`` then proceeds to call ``glet.run()``, which
		+ - results in ``PyEval_...`` adding the CFrame to the list. Switches
		+ + results in ``PyEval_...`` adding the _PyCFrame to the list. Switches
		`+ continue as normal. Finally, when the greenlet finishes, the call to`
		+ - ``glet.run()`` returns and the CFrame is taken out of the linked
		+ + ``glet.run()`` returns and the _PyCFrame is taken out of the linked
		`+ list and the stack value is now unused and free to expire.`
		`+ */`
		`+ ((PyGreenlet*)o)->cframe = &PyThreadState_GET()->root_cframe;`
		`+ @@ -1121,9 +1150,11 @@ green_traverse(PyGreenlet* self, visitproc visit, void* arg)`
		`+ Py_VISIT(self->context);`
		`+ #endif`
		`+ #if GREENLET_PY37`
		`+ - Py_VISIT(self->exc_state.exc_type);`
		`+ Py_VISIT(self->exc_state.exc_value);`
		`+ +#if !GREENLET_PY311`
		`+ + Py_VISIT(self->exc_state.exc_type);`
		`+ Py_VISIT(self->exc_state.exc_traceback);`
		`+ +#endif`
		`+ #else`
		`+ Py_VISIT(self->exc_type);`
		`+ Py_VISIT(self->exc_value);`
		`+ @@ -1159,9 +1190,11 @@ green_clear(PyGreenlet* self)`
		`+ Py_CLEAR(self->context);`
		`+ #endif`
		`+ #if GREENLET_PY37`
		`+ - Py_CLEAR(self->exc_state.exc_type);`
		`+ Py_CLEAR(self->exc_state.exc_value);`
		`+ +#if !GREENLET_PY311`
		`+ + Py_CLEAR(self->exc_state.exc_type);`
		`+ Py_CLEAR(self->exc_state.exc_traceback);`
		`+ +#endif`
		`+ #else`
		`+ Py_CLEAR(self->exc_type);`
		`+ Py_CLEAR(self->exc_value);`
		`+ @@ -1253,9 +1286,11 @@ green_dealloc(PyGreenlet* self)`
		`+ Py_CLEAR(self->context);`
		`+ #endif`
		`+ #if GREENLET_PY37`
		`+ - Py_CLEAR(self->exc_state.exc_type);`
		`+ Py_CLEAR(self->exc_state.exc_value);`
		`+ +#if !GREENLET_PY311`
		`+ + Py_CLEAR(self->exc_state.exc_type);`
		`+ Py_CLEAR(self->exc_state.exc_traceback);`
		`+ +#endif`
		`+ #else`
		`+ Py_CLEAR(self->exc_type);`
		`+ Py_CLEAR(self->exc_value);`
		`+ diff --git a/src/greenlet/greenlet.h b/src/greenlet/greenlet.h`
		`+ index 830bef8d..c788b2fe 100644`
		`+ --- a/src/greenlet/greenlet.h`
		`+ +++ b/src/greenlet/greenlet.h`
		`+ @@ -14,6 +14,15 @@ extern "C" {`
		`+ /* This is deprecated and undocumented. It does not change. */`
		`+ #define GREENLET_VERSION "1.0.0"`
		`+`
		`+ +#if PY_VERSION_HEX >= 0x30B00A6`
		`+ +# define GREENLET_PY311 1`
		`+ + /* _PyInterpreterFrame moved to the internal C API in Python 3.11 */`
		`+ +# include <internal/pycore_frame.h>`
		`+ +#else`
		`+ +# define GREENLET_PY311 0`
		`+ +# define _PyCFrame CFrame`
		`+ +#endif`
		`+ +`
		`+ typedef struct _greenlet {`
		`+ PyObject_HEAD`
		`+ char* stack_start;`
		`+ @@ -25,6 +34,12 @@ typedef struct _greenlet {`
		`+ PyObject* run_info;`
		`+ struct _frame* top_frame;`
		`+ int recursion_depth;`
		`+ +#if GREENLET_PY311`
		`+ + _PyInterpreterFrame *current_frame;`
		`+ + _PyStackChunk *datastack_chunk;`
		`+ + PyObject **datastack_top;`
		`+ + PyObject **datastack_limit;`
		`+ +#endif`
		`+ PyObject* weakreflist;`
		`+ #if PY_VERSION_HEX >= 0x030700A3`
		`+ _PyErr_StackItem* exc_info;`
		`+ @@ -39,7 +54,7 @@ typedef struct _greenlet {`
		`+ PyObject* context;`
		`+ #endif`
		`+ #if PY_VERSION_HEX >= 0x30A00B1`
		`+ - CFrame* cframe;`
		`+ + _PyCFrame* cframe;`
		`+ #endif`
		`+ } PyGreenlet;`
		`+`
		`+ diff --git a/tox.ini b/tox.ini`
		`+ index 21ecbbb2..efec08bc 100644`
		`+ --- a/tox.ini`
		`+ +++ b/tox.ini`
		`+ @@ -1,6 +1,6 @@`
		`+ [tox]`
		`+ envlist =`
		`+ - py27,py35,py36,py37,py38,py39,py310,docs`
		`+ + py27,py35,py36,py37,py38,py39,py310,py311,docs`
		`+`
		`+ [testenv]`
		`+ commands =`

python-greenlet.spec

file modified

+13 -1

		`@@ -2,11 +2,19 @@`

		`Name: python-%{modname}`
		`Version: 1.1.2`
		`- Release: 2%{?dist}`
		`+ Release: 3%{?dist}`
		`Summary: Lightweight in-process concurrent programming`
		`License: MIT`
		`URL: https://github.com/python-greenlet/greenlet`
		`Source0: %{url}/archive/%{version}/%{modname}-%{version}.tar.gz`
		`+`
		`+ # Python 3.11 support, backported from 2.0.0a2`
		`+ # https://github.com/python-greenlet/greenlet/commit/fd0b68ab406a0dfe3d6d0d8c9d17354356f53da0`
		`+ # https://github.com/python-greenlet/greenlet/commit/63e1099acc3677e614532bea0fa2e1967b69125f`
		`+ # https://github.com/python-greenlet/greenlet/commit/5ed467e5cb34651cc013c286158d0b6d7ff0a26a`
		`+ # Proposed upstream via https://github.com/python-greenlet/greenlet/pull/306`
		`+ Patch: https://github.com/python-greenlet/greenlet/pull/306.patch`
		`+`
		`BuildRequires: gcc-c++`

		`%global _description \`
		`@@ -59,6 +67,10 @@`
		`%{_includedir}/python%{python3_version}*/%{modname}/`

		`%changelog`
		`+ * Wed Jun 01 2022 Miro Hrončok <[email protected]> - 1.1.2-3`
		`+ - Python 3.11 support`
		`+ - Fixes: rhbz#2040186`
		`+`
		`* Fri Jan 21 2022 Fedora Release Engineering <[email protected]> - 1.1.2-2`
		`- Rebuilt for https://fedoraproject.org/wiki/Fedora_36_Mass_Rebuild`

churchyard commented 3 years ago

I made it build. Now it just crashes :)

+ /usr/bin/python3 -m unittest discover greenlet.tests
........s.....................free(): invalid pointer
/var/tmp/rpm-tmp.2vvtj5: line 42:   745 Aborted                 (core dumped)

rebased onto c049a3c

3 years ago

churchyard commented 3 years ago

The following tests now trigger a Segmentation fault.

test_send_exception
test_issue_245_reference_counting_subclass_threads

And test_dealloc_other_thread does: free(): invalid pointer

Edited 3 years ago by churchyard

churchyard commented 3 years ago

The following tests now trigger a Segmentation fault.

test_send_exception

test_issue_245_reference_counting_subclass_threads

And test_dealloc_other_thread does: free(): invalid pointer

But when the tests run in isolation, they pass :/

rebased onto d634d90

3 years ago

rebased onto e6aaaaa

3 years ago

rebased onto cea5a6d

3 years ago

rebased onto 1d6d586

3 years ago

churchyard commented 3 years ago

This no longer crashes. However, I am not happy about #include <internal/pycore_frame.h> and would like to talk to @vstinner about that. Can we get around it? I see that gevent also includes that internal header via https://github.com/gevent/gevent/pull/1872 so I guess we cannot, right?

rebased onto 021648f

3 years ago