This is the nicest profiler I've found for Python. It's cross-platform, doesn't need me to change the code that's being profiled, and its output can be piped directly into flamegraph.pl. I just used it to pinpoint a gross misuse of SQLAlchemy at work that's run in some code at the end of each day, and now I can go home earlier.
It handles threads just fine, and doesn't handle multiprocessing done via spawning (which isn't surprising; it's a sampling frame stack profiler so it only has access to what the parent process does). It's not happy under WSL so I can't check what it does for children that fork, but maybe some kind soul can run the following on Linux and report?
import time
import multiprocessing
def fact(n):
f = 1
for i in range(1, n+1):
f *= i
return f
def do(N):
n = 1
for _ in range(N):
fact(n)
n += 1
if __name__ == '__main__':
threads = []
for _ in range(2):
t = multiprocessing.Process(target=do, args=(5000,))
t.start()
threads.append(t)
for t in threads:
t.join(timeout=5)
(Did I say an hour? Time flies...)
Running that code, it looks like the traces just find a load of processes in their popen join/wait loops. No trace of fact or do
This is the nicest profiler I've found for Python. It's cross-platform, doesn't need me to change the code that's being profiled, and its output can be piped directly into
flamegraph.pl. I just used it to pinpoint a gross misuse of SQLAlchemy at work that's run in some code at the end of each day, and now I can go home earlier.No mention in the readme of how it handles (or doesn’t) forks or threads. Would be curious to hear more!
It handles threads just fine, and doesn't handle multiprocessing done via spawning (which isn't surprising; it's a sampling frame stack profiler so it only has access to what the parent process does). It's not happy under WSL so I can't check what it does for children that fork, but maybe some kind soul can run the following on Linux and report?
(Did I say an hour? Time flies...) Running that code, it looks like the traces just find a load of processes in their popen join/wait loops. No trace of
factordoThat’s the same as for the spawning children. There’s our answer: threads ok, not-threads not ok.
Makes sense, thanks!
[Comment removed by author]