Skip to content

monkey patching of existing RLock is broken on CPython3 #546

@antocuni

Description

@antocuni

eventlet.patcher._green_existing_locks tries to monkey-patch locks which have been created before the call to monkey patch. To do so, it relies on gc.get_objects() to find them.

This works well on CPython2.7, but it is broken on CPython3.x, as the following example demonstrate (it prints Found! on 2.7 and Not found on 3.6):

import threading
import gc
lock = threading.RLock()
for obj in gc.get_objects():
    if obj is lock:
        print('Found!')
        break
else:
    print('Not found')

This happens because in 3.6 threading.RLock() returns an object of type _thread._RLock, which is implemented in C, inside _threadmodule.c.
In particular, RLockType does not have the flag Py_TPFLAGS_HAVE_GC, and as such it is never returned by gc.get_objects().

Another more direct demonstration of the problem:

import threading
lock = threading.RLock()
print('my lock [1]:', type(lock), lock, id(lock))

import eventlet
eventlet.monkey_patch()
print('my lock [2]:', type(lock), lock, id(lock))

On my machine, with the latest eventlet HEAD (commit a915bb6), running it on python3.6 shows that id(lock) is still the same even after the monkey_patch().

The fact that it went unnoticed until now probably means that this specific functionality lacks a test.

However, I don't know how to fix the problem: it seems to me that there is not reasonable way to implement this functionality in Python3 (not counting the fact that even if we find the existing locks, monkey-patching them is hard: _fix_py3_lock looks very fragile and there are probably cases in which it doesn't work at all).

A possible, brutal "solution" would be to fail and complain loudly if you try to call monkey_patch after threading has already been imported. It probably breaks a lot of existing code, but it's still better than leaving it half-broken as it is now

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions