-
Notifications
You must be signed in to change notification settings - Fork 332
monkey patching of existing RLock is broken on CPython3 #546
Description
eventlet.patcher._green_existing_locks tries to monkey-patch locks which have been created before the call to monkey patch. To do so, it relies on gc.get_objects() to find them.
This works well on CPython2.7, but it is broken on CPython3.x, as the following example demonstrate (it prints Found! on 2.7 and Not found on 3.6):
import threading
import gc
lock = threading.RLock()
for obj in gc.get_objects():
if obj is lock:
print('Found!')
break
else:
print('Not found')This happens because in 3.6 threading.RLock() returns an object of type _thread._RLock, which is implemented in C, inside _threadmodule.c.
In particular, RLockType does not have the flag Py_TPFLAGS_HAVE_GC, and as such it is never returned by gc.get_objects().
Another more direct demonstration of the problem:
import threading
lock = threading.RLock()
print('my lock [1]:', type(lock), lock, id(lock))
import eventlet
eventlet.monkey_patch()
print('my lock [2]:', type(lock), lock, id(lock))On my machine, with the latest eventlet HEAD (commit a915bb6), running it on python3.6 shows that id(lock) is still the same even after the monkey_patch().
The fact that it went unnoticed until now probably means that this specific functionality lacks a test.
However, I don't know how to fix the problem: it seems to me that there is not reasonable way to implement this functionality in Python3 (not counting the fact that even if we find the existing locks, monkey-patching them is hard: _fix_py3_lock looks very fragile and there are probably cases in which it doesn't work at all).
A possible, brutal "solution" would be to fail and complain loudly if you try to call monkey_patch after threading has already been imported. It probably breaks a lot of existing code, but it's still better than leaving it half-broken as it is now