You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Traceback (most recent call last):
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 2059, in run_profiler
exit_status = profiler.profile_code(
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1861, in profile_code
did_output = Scalene.output_profile()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 971, in output_profile
did_output: bool = output.output_profiles(
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_output.py", line 637, in output_profiles
console.print(tbl)
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 1720, in print
self._buffer.extend(new_segments)
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 864, in exit
self._exit_buffer()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 822, in _exit_buffer
self._check_buffer()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 2060, in _check_buffer
self.file.write(text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-94: ordinal not in range(128)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***
Looks like this error is occurring inside the rich package. Can you add the output of pip freeze? Also, we could use a repro case and more info about your system (this is in the issue template).
Please fill out the other details of this template:
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
Go to '...'
Click on '....'
Scroll down to '....'
See error
Please include a minimum working example if at all possible.
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]
If you have not yet tried with the repository version (python3 -m pip install git+https://github.com/plasma-umass/scalene), please try that before reporting.
Additional context
Add any other context about the problem here.
repro:
import numpy as np
import cupy as cp
a= cp.ones(shape=(10,10))
u,s,v = cp.linalg.svd(a)
print(u,s,v)
you could reproduce the error on Ubuntu with Cuda version 11.6, cupy is a package for Numpy to run on GPU. I think cupy package is related to the issue.
error output:
Scalene: An exception of type UnicodeEncodeError occurred. Arguments:
('ascii', '\x1b[3m test.py: % of time = 100.00% (1.682s) out of 1.682s. \x1b[0m\n+-----------------------------------------------------------------------------------------------------------------------------+\n|\x1b[1m \x1b[0m|\x1b[1;34mTime\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;38;5;106m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\n|\x1b[1m \x1b[0m\x1b[1;2mLine\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mPython\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mnative\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34msystem\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;38;5;106mGPU\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1mtest.py \x1b[0m\x1b[1m \x1b[0m|\n|------+-------+-------+-------+-------+--------------------------------------------------------------------------------------|\n|\x1b[2m \x1b[0m\x1b[2m 1\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;0;0;128;48;2;0;0;0m#!/usr/bin/env python3\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 2\x1b[0m\x1b[2m \x1b[0m|\x1b[34m 2%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 2%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 1% \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m 23% \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;205;0;205;48;2;0;0;0mimport\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mnumpy\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;205;205;0;48;2;0;0;0mas\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mnp\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 3\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;0;0;128;48;2;0;0;0m#import math\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 4\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 5\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;0;0;128;48;2;0;0;0m# from numpy import linalg as LA\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 6\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 7\x1b[0m\x1b[2m \x1b[0m|\x1b[1;31m 8%\x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 4%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 3% \x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 60%\x1b[0m\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;205;0;205;48;2;0;0;0mimport\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcupy\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;205;205;0;48;2;0;0;0mas\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcp\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 8\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 9\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m 8%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 8% \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m 12% \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;204;204;204;48;2;0;0;0ma\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m=\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcp\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m.\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mones\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mshape\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m=\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;205;0;205;48;2;0;0;0m10\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;205;0;205;48;2;0;0;0m10\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 10\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 40%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 23% \x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 5%\x1b[0m\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;204;204;204;48;2;0;0;0mu\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0ms\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mv\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m=\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcp\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m.\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mlinalg\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m.\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0msvd\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0ma\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 11\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;205;0;205;48;2;0;0;0mprint\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mu\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0ms\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mv\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n+-----------------------------------------------------------------------------------------------------------------------------+\n', 322, 328, 'ordinal not in range(128)')
Traceback (most recent call last):
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 2059, in run_profiler
exit_status = profiler.profile_code(
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1861, in profile_code
did_output = Scalene.output_profile()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 971, in output_profile
did_output: bool = output.output_profiles(
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_output.py", line 637, in output_profiles
console.print(tbl)
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 1720, in print
self._buffer.extend(new_segments)
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 864, in exit
self._exit_buffer()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 822, in _exit_buffer
self._check_buffer()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 2060, in _check_buffer
self.file.write(text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 322-327: ordinal not in range(128)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***
Hey, I also think cupy might be at fault here. It would be nice to get this fixed (or at least some printing workaround) because it'll be very hard to seperate this from the app we're trying to profile.
I'm getting the same error in a slightly different circumstance.
Describe the bug
When scalene tries to print the fancy table to stdout (console.print(tbl)), it can cause rich text to throw a UnicodeEncodeError: 'ascii' codec can't encode characters in position XXX-YYY : ordinal not in range(128). My best guess is that it's some behavior related to cupy, a fairly common python library for placing operations on the GPU.
To Reproduce
1 - start with a computer with a NVIDIA GPU.
5 - working_example.py:
from spacy import require_cpu, load
require_cpu() #ensures that cupy is NOT used as the backend (for setting spacy ops)
nlp = load("en_core_web_sm")
6 - scalene --cli working_example.py
WORKS, scalene outputs a nice table of the profiling.
Memory usage: ▁▁▁▁▂▂▃▃▃▃▄▄▄▅▅▅▅▆▇▇▇▇█████ (max: 333.114 MB, growth rate: 100%)
working_example.py: % of time = 100.00% (24.485s) out of 24.485s.
╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷ ╷
│Time │–––––– │––––… │–––––– │Memory │–––––– │––––––––––– │Copy │
Line │Python │native │syst… │GPU │Python │peak │timeline/% │(MB/s) │working_example.py
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
1 │ 50% │ 33% │ 9% │ 88% │ 82% │ 272M │▁▂▂▃▃▃▄▄▅ 82% │ 434 │from spacy import require_cpu, load
2 │ │ │ │ │ │ │ │ │require_cpu() #ensures that cupy is NOT used as the backend (for setting spacy ops)
3 │ 6% │ │ 1% │ 12% │ 96% │ 61M │▂▂▂▂▂▂ 18% │ 48 │nlp = load("en_core_web_sm")
4 │ │ │ │ │ │ │ │ │
╵ ╵ ╵ ╵ ╵ ╵ ╵ ╵ ╵
Top AVERAGE memory consumption, by line:
(1) 1: 272 MB
Top PEAK memory consumption, by line:
(1) 1: 272 MB
(2) 3: 61 MB
Possible memory leak identified at line 1 (estimated likelihood: 95%, velocity: 11 MB/s)
7- broken_example.py
from spacy import require_gpu, load
require_gpu() #ensures that cupy IS used as the backend (for setting spacy ops)
nlp = load("en_core_web_sm")
8 - scalene --cli broken_example.py
Scalene: An exception of type UnicodeEncodeError occurred. Arguments:
('ascii', ' Memory usage: \x1b[32m▁▁▁▁▁▂▂▂▂▃▃▄▄▄▄▅▅▆█████████\x1b[0m (max: 633.836 MB, growth rate: 85%)
\n broken_example.py: % of time = 100.00% (25.644s) out of 25.644s.
\n+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\n|\x1b[1m \x1b[0m|\x1b[1;34mTime\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––…\x1b[0m\x1b[1m \x1b[0m|\x1b[1;33m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;32mMemory\x1b[0m\x1b[1m \x1b[0m|\x1b[1;32m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;32m–––––––––––\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;33mCopy\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\n|\x1b[1m \x1b[0m\x1b[1;2mLine\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mPython\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mnative\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34msyst…\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;33mGPU\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;32mPython\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;32mpeak\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;32mtimeline\x1b[0m\x1b[1;32m/%\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;33m(MB/s)\x1b[0m\x1b[1m \x1b[0m|\x1b[1mbroken_example.py \x1b[0m\x1b[1m \x1b[0m|\n|------+-------+-------+------+-------+-------+-------+---------------+-------+--------------------------------------------------------------------------------------------------------------------------------|\n|\x1b[2m \x1b[0m\x1b[2m 1\x1b[0m\x1b[2m \x1b[0m|\x1b[1;31m 48%\x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 32%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 9%\x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 89%\x1b[0m\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[32m 82% \x1b[0m\x1b[32m \x1b[0m|\x1b[32m 272M\x1b[0m\x1b[32m \x1b[0m|\x1b[1;31m▁▁▁▁▁▁▁▁▂ 37%\x1b[0m\x1b[32m \x1b[0m|\x1b[33m 415\x1b[0m\x1b[33m \x1b[0m|\x1b[35;40mfrom\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mspacy\x1b[0m\x1b[37;40m \x1b[0m\x1b[35;40mimport\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mrequire_gpu\x1b[0m\x1b[37;40m,\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mload\x1b[0m\x1b[40m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 2\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[37;40mrequire_gpu\x1b[0m\x1b[37;40m(\x1b[0m\x1b[37;40m)\x1b[0m\x1b[37;40m \x1b[0m\x1b[34;40m#ensures that cupy IS used as the backend (for setting spacy ops)\x1b[0m\x1b[40m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 3\x1b[0m\x1b[2m \x1b[0m|\x1b[34m 6%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 3%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 1%\x1b[0m\x1b[34m \x1b[0m|\x1b[33m 11% \x1b[0m\x1b[33m \x1b[0m|\x1b[32m 12% \x1b[0m\x1b[32m \x1b[0m|\x1b[32m 361M\x1b[0m\x1b[32m \x1b[0m|\x1b[32m▃▃▄▄▅▅▅▅▅ 63%\x1b[0m\x1b[32m \x1b[0m|\x1b[33m 65\x1b[0m\x1b[33m \x1b[0m|\x1b[37;40mnlp\x1b[0m\x1b[37;40m \x1b[0m\x1b[36;40m=\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mload\x1b[0m\x1b[37;40m(\x1b[0m\x1b[31;40m"\x1b[0m\x1b[31;40men_core_web_sm\x1b[0m\x1b[31;40m"\x1b[0m\x1b[37;40m)\x1b[0m\x1b[40m
\x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 4\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[40m
\x1b[0m |\n+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\n', 84, 111, 'ordinal not in range(128)')
Traceback (most recent call last):
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 2059, in run_profiler
exit_status = profiler.profile_code(
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1861, in profile_code
did_output = Scalene.output_profile()
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 971, in output_profile
did_output: bool = output.output_profiles(
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_output.py", line 637, in output_profiles
console.print(tbl)
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 1720, in print
self._buffer.extend(new_segments)
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 864, in exit
self._exit_buffer()
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 822, in _exit_buffer
self._check_buffer()
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 2060, in _check_buffer
self.file.write(text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 84-110: ordinal not in range(128)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***
Expected behavior
Scalene successfully profiles the script and outputs the nearly identical table to the one obtained successfully in step 5.
Desktop (please complete the following information):
OS: ubuntu 20.04 (docker image on a windows host)
GPU: GTX 3090
Browser: CLI
Version: 1.5.21.4
Additional context
I originally went down the rabbit hole of trying to fix the actual ut8 / ASCII codec error, but none of the fixes obtained from googling that except rich text raises worked for me. In particular:
I set the environment variable PYTHONIOENCODING to utf-8
I tried sys.stdin.reconfigure(encoding='utf-8') and sys.stdout.reconfigure(encoding='utf-8')
I set LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
I even tried setting the windows powershell terminal to utf-8 using language settings -> administrative language settings -> change system locale -> Beta: Use Unicode UTF-8 for worldwide language support. (since it's a Ubuntu docker container outputting text to a Windows powershell terminal)
So I think the actual unicode exception is either a red herring or maybe something in cupy resets the python's expectation to use the utf-8 codec?
Thanks @Louis-Paul-Bowman for this detailed report! I was able to find the issue, which indeed is CuPy related (actually NVIDIA related): NVIDIA/cuda-python#29.
Please test by running the current repo version (I will release a new version once confirmed): python3 -m pip install -U git+https://github.com/plasma-umass/scalene)
Please test by running the current repo version (I will release a new version once confirmed): python3 -m pip install -U git+https://github.com/plasma-umass/scalene)
Confirmed, fixed on my end when pulling the repo version.
Nice find! That is... cryptic behavior to say the least.
Thanks @Louis-Paul-Bowman for this detailed report! I was able to find the issue, which indeed is CuPy related (actually NVIDIA related): NVIDIA/cuda-python#29.
I can confirm that the workaround export LC_ALL="POSIX" fixed my problem highly similar to this issue, millions of thanks!
Activity
emeryberger commentedon Jun 11, 2023
Looks like this error is occurring inside the
richpackage. Can you add the output ofpip freeze? Also, we could use a repro case and more info about your system (this is in the issue template).gongyguo commentedon Jun 11, 2023
absl-py==1.4.0
alabaster @ file:///home/ktietz/src/ci/alabaster_1611921544520/work
anaconda-client==1.11.0
anaconda-navigator==2.3.1
anaconda-project @ file:///opt/conda/conda-bld/anaconda-project_1660339890420/work
anyio @ file:///tmp/build/80754af9/anyio_1644463572971/work/dist
appdirs==1.4.4
argon2-cffi @ file:///opt/conda/conda-bld/argon2-cffi_1645000214183/work
argon2-cffi-bindings @ file:///tmp/build/80754af9/argon2-cffi-bindings_1644569679365/work
arrow @ file:///opt/conda/conda-bld/arrow_1649166651673/work
astroid @ file:///tmp/abs_e5wkt48jiz/croots/recipe/astroid_1659023120113/work
astropy @ file:///opt/conda/conda-bld/astropy_1657786094003/work
astunparse==1.6.3
atomicwrites==1.4.0
attrs @ file:///opt/conda/conda-bld/attrs_1642510447205/work
Automat @ file:///tmp/build/80754af9/automat_1600298431173/work
autopep8 @ file:///opt/conda/conda-bld/autopep8_1650463822033/work
Babel @ file:///tmp/build/80754af9/babel_1620871417480/work
backcall @ file:///home/ktietz/src/ci/backcall_1611930011877/work
backports.functools-lru-cache @ file:///tmp/build/80754af9/backports.functools_lru_cache_1618170165463/work
backports.tempfile @ file:///home/linux1/recipes/ci/backports.tempfile_1610991236607/work
backports.weakref==1.0.post1
bcrypt @ file:///tmp/abs_6fpe92qzzo/croots/recipe/bcrypt_1659554336150/work
beautifulsoup4 @ file:///opt/conda/conda-bld/beautifulsoup4_1650462163268/work
binaryornot @ file:///tmp/build/80754af9/binaryornot_1617751525010/work
bitarray @ file:///opt/conda/conda-bld/bitarray_1657739645104/work
bkcharts==0.2
black @ file:///opt/conda/conda-bld/black_1660237809219/work
bleach @ file:///opt/conda/conda-bld/bleach_1641577558959/work
bokeh @ file:///tmp/abs_34854e1f-d7d3-4f22-85d9-1075588e4ecdga64o0qg/croots/recipe/bokeh_1658136654619/work
boto3 @ file:///tmp/abs_ae3c72db-af47-4298-baea-7270430e2c96scbpg1_h/croots/recipe/boto3_1657820109150/work
botocore @ file:///opt/conda/conda-bld/botocore_1657739486257/work
Bottleneck @ file:///opt/conda/conda-bld/bottleneck_1657175564434/work
brotlipy==0.7.0
cachetools==5.3.0
certifi @ file:///opt/conda/conda-bld/certifi_1663615672595/work/certifi
cffi @ file:///tmp/abs_98z5h56wf8/croots/recipe/cffi_1659598650955/work
chardet @ file:///tmp/build/80754af9/chardet_1607706775000/work
charset-normalizer @ file:///tmp/build/80754af9/charset-normalizer_1630003229654/work
click @ file:///tmp/build/80754af9/click_1646056590078/work
cloudpickle==2.2.1
clyent==1.2.2
cmake==3.26.1
colorama @ file:///opt/conda/conda-bld/colorama_1657009087971/work
colorcet @ file:///tmp/build/80754af9/colorcet_1651851439427/work
conda==22.9.0
conda-build==3.22.0
conda-content-trust @ file:///tmp/abs_5952f1c8-355c-4855-ad2e-538535021ba5h26t22e5/croots/recipe/conda-content-trust_1658126371814/work
conda-pack @ file:///tmp/build/80754af9/conda-pack_1611163042455/work
conda-package-handling @ file:///opt/conda/conda-bld/conda-package-handling_1663598473529/work
conda-repo-cli==1.0.20
conda-token @ file:///Users/paulyim/miniconda3/envs/c3i/conda-bld/conda-token_1662660369760/work
conda-verify==3.4.2
constantly==15.1.0
cookiecutter @ file:///opt/conda/conda-bld/cookiecutter_1649151442564/work
cryptography @ file:///tmp/build/80754af9/cryptography_1652101588893/work
cssselect==1.1.0
cubinlinker-cu11==0.3.0.post1
cuda-python==11.8.1
cudf-cu11==23.2.0
cuml-cu11==23.2.0
cupy-cuda116==10.6.0
cupy-cuda11x==11.6.0
cycler @ file:///tmp/build/80754af9/cycler_1637851556182/work
Cython @ file:///opt/conda/conda-bld/cython_1663692770955/work
cytoolz==0.11.0
daal4py==2021.6.0
dask==2023.1.1
dask-cuda==23.2.1
dask-cudf-cu11==23.2.0
datashader @ file:///tmp/abs_aa58dfo4_s/croots/recipe/datashader_1659349033064/work
datashape==0.5.4
debugpy @ file:///tmp/build/80754af9/debugpy_1637091799509/work
decorator @ file:///opt/conda/conda-bld/decorator_1643638310831/work
defusedxml @ file:///tmp/build/80754af9/defusedxml_1615228127516/work
diff-match-patch @ file:///Users/ktietz/demo/mc3/conda-bld/diff-match-patch_1630511840874/work
dill @ file:///tmp/build/80754af9/dill_1623919422540/work
distributed==2023.1.1
docutils @ file:///opt/conda/conda-bld/docutils_1657175430858/work
entrypoints @ file:///tmp/build/80754af9/entrypoints_1649926439650/work
et-xmlfile==1.1.0
faiss==1.7.3
fastjsonschema @ file:///opt/conda/conda-bld/python-fastjsonschema_1661371079312/work
fastrlock==0.8.1
filelock @ file:///opt/conda/conda-bld/filelock_1647002191454/work
flake8 @ file:///opt/conda/conda-bld/flake8_1648129545443/work
Flask @ file:///home/ktietz/src/ci/flask_1611932660458/work
flatbuffers==23.3.3
fonttools==4.25.0
fsspec @ file:///opt/conda/conda-bld/fsspec_1659972197723/work
future @ file:///tmp/build/80754af9/future_1607571303524/work
gast==0.4.0
gensim @ file:///tmp/build/80754af9/gensim_1646806807927/work
glob2 @ file:///home/linux1/recipes/ci/glob2_1610991677669/work
gmpy2 @ file:///tmp/build/80754af9/gmpy2_1645438755360/work
google-auth==2.16.2
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
greenlet @ file:///tmp/build/80754af9/greenlet_1628888132713/work
grpcio==1.51.3
h5py @ file:///tmp/abs_4aewd3wzey/croots/recipe/h5py_1659091371897/work
HeapDict @ file:///Users/ktietz/demo/mc3/conda-bld/heapdict_1630598515714/work
holoviews @ file:///tmp/abs_eecc808c-455e-4be4-9911-ecf8341b3a34jfwskiqe/croots/recipe/holoviews_1658171506757/work
hvplot @ file:///tmp/abs_6fcys5jcv1/croots/recipe/hvplot_1659026496554/work
hyperlink @ file:///tmp/build/80754af9/hyperlink_1610130746837/work
idna @ file:///tmp/build/80754af9/idna_1637925883363/work
imagecodecs @ file:///opt/conda/conda-bld/imagecodecs_1664561985385/work
imageio @ file:///tmp/abs_cd920173-f360-47c5-97b0-bf4d1076d5d4dvic0oys/croots/recipe/imageio_1658785036907/work
imagesize @ file:///opt/conda/conda-bld/imagesize_1657179498843/work
importlib-metadata @ file:///tmp/build/80754af9/importlib-metadata_1648544546694/work
incremental @ file:///tmp/build/80754af9/incremental_1636629750599/work
inflection==0.5.1
iniconfig @ file:///home/linux1/recipes/ci/iniconfig_1610983019677/work
intake @ file:///opt/conda/conda-bld/intake_1647436631684/work
intervaltree @ file:///Users/ktietz/demo/mc3/conda-bld/intervaltree_1630511889664/work
ipykernel @ file:///opt/conda/conda-bld/ipykernel_1662361798230/work
ipython @ file:///tmp/abs_94gruux8u8/croots/recipe/ipython_1659529858706/work
ipython-genutils @ file:///tmp/build/80754af9/ipython_genutils_1606773439826/work
ipywidgets @ file:///tmp/build/80754af9/ipywidgets_1634143127070/work
isort @ file:///tmp/build/80754af9/isort_1628603791788/work
itemadapter @ file:///tmp/build/80754af9/itemadapter_1626442940632/work
itemloaders @ file:///opt/conda/conda-bld/itemloaders_1646805235997/work
itsdangerous @ file:///tmp/build/80754af9/itsdangerous_1621432558163/work
jdcal @ file:///Users/ktietz/demo/mc3/conda-bld/jdcal_1630584345063/work
jedi @ file:///tmp/build/80754af9/jedi_1644297102865/work
jeepney @ file:///tmp/build/80754af9/jeepney_1627537048313/work
jellyfish @ file:///tmp/build/80754af9/jellyfish_1647944426575/work
Jinja2==3.1.2
jinja2-time @ file:///opt/conda/conda-bld/jinja2-time_1649251842261/work
jmespath @ file:///Users/ktietz/demo/mc3/conda-bld/jmespath_1630583964805/work
joblib @ file:///tmp/build/80754af9/joblib_1635411271373/work
json5 @ file:///tmp/build/80754af9/json5_1624432770122/work
jsonschema @ file:///opt/conda/conda-bld/jsonschema_1663375472438/work
jupyter @ file:///tmp/abs_33h4eoipez/croots/recipe/jupyter_1659349046347/work
jupyter-console @ file:///opt/conda/conda-bld/jupyter_console_1647002188872/work
jupyter-server @ file:///tmp/abs_b88b31b8-83b9-476d-a46d-e563c421f38fvsnyi1ur/croots/recipe/jupyter_server_1658754481507/work
jupyter_client @ file:///opt/conda/conda-bld/jupyter_client_1661848916004/work
jupyter_core @ file:///opt/conda/conda-bld/jupyter_core_1664917302524/work
jupyterlab @ file:///tmp/abs_12f3h01vmy/croots/recipe/jupyterlab_1658907535764/work
jupyterlab-pygments @ file:///tmp/build/80754af9/jupyterlab_pygments_1601490720602/work
jupyterlab-server @ file:///opt/conda/conda-bld/jupyterlab_server_1644500396812/work
jupyterlab-widgets @ file:///tmp/build/80754af9/jupyterlab_widgets_1609884341231/work
keras==2.11.0
keyring @ file:///tmp/build/80754af9/keyring_1638531355686/work
kiwisolver @ file:///opt/conda/conda-bld/kiwisolver_1653292039266/work
lazy-object-proxy @ file:///tmp/build/80754af9/lazy-object-proxy_1616529027849/work
libarchive-c @ file:///tmp/build/80754af9/python-libarchive-c_1617780486945/work
libclang==15.0.6.1
lit==16.0.0
llvmlite==0.39.1
locket @ file:///opt/conda/conda-bld/locket_1652903118915/work
lxml @ file:///opt/conda/conda-bld/lxml_1657545139709/work
lz4 @ file:///tmp/build/80754af9/lz4_1619516502891/work
Mako==1.2.4
Markdown @ file:///tmp/build/80754af9/markdown_1614363852612/work
markdown-it-py==2.2.0
MarkupSafe @ file:///tmp/build/80754af9/markupsafe_1621523467000/work
matplotlib @ file:///opt/conda/conda-bld/matplotlib-suite_1660167928326/work
matplotlib-inline @ file:///opt/conda/conda-bld/matplotlib-inline_1662014470464/work
mccabe @ file:///opt/conda/conda-bld/mccabe_1644221741721/work
mdurl==0.1.2
mistune @ file:///tmp/build/80754af9/mistune_1607364877025/work
mkl-fft==1.3.1
mkl-random @ file:///tmp/build/80754af9/mkl_random_1626186066731/work
mkl-service==2.4.0
mock @ file:///tmp/build/80754af9/mock_1607622725907/work
mpmath==1.2.1
msgpack @ file:///opt/conda/conda-bld/msgpack-python_1652362659880/work
multipledispatch @ file:///tmp/build/80754af9/multipledispatch_1607574243360/work
munkres==1.1.4
mypy-extensions==0.4.3
navigator-updater==0.3.0
nbclassic @ file:///opt/conda/conda-bld/nbclassic_1644943264176/work
nbclient @ file:///tmp/build/80754af9/nbclient_1650290509967/work
nbconvert @ file:///opt/conda/conda-bld/nbconvert_1649751911790/work
nbformat @ file:///opt/conda/conda-bld/nbformat_1663744952973/work
nest-asyncio @ file:///tmp/build/80754af9/nest-asyncio_1649847906199/work
networkx @ file:///opt/conda/conda-bld/networkx_1657784097507/work
nltk @ file:///opt/conda/conda-bld/nltk_1645628263994/work
nose @ file:///opt/conda/conda-bld/nose_1642704612149/work
notebook @ file:///tmp/abs_abf6xa6h6f/croots/recipe/notebook_1659083654985/work
numba==0.56.4
numexpr @ file:///opt/conda/conda-bld/numexpr_1656940300424/work
numpy @ file:///opt/conda/conda-bld/numpy_and_numpy_base_1653915516269/work
numpydoc @ file:///opt/conda/conda-bld/numpydoc_1657529872251/work
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-cupti-cu11==11.7.101
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.2.10.91
nvidia-cusolver-cu11==11.4.0.1
nvidia-cusparse-cu11==11.7.4.91
nvidia-nccl-cu11==2.14.3
nvidia-nvtx-cu11==11.7.91
nvtx==0.2.5
oauthlib==3.2.2
olefile @ file:///Users/ktietz/demo/mc3/conda-bld/olefile_1629805411829/work
openpyxl==3.0.10
opt-einsum==3.3.0
packaging @ file:///tmp/build/80754af9/packaging_1637314298585/work
pandas==1.4.4
pandocfilters @ file:///opt/conda/conda-bld/pandocfilters_1643405455980/work
panel @ file:///tmp/abs_bb3d3b2f-b3ea-41c0-a72e-8f54852d5cdfs70inytz/croots/recipe/panel_1658133826470/work
param @ file:///tmp/build/80754af9/param_1636647414893/work
parsel @ file:///tmp/build/80754af9/parsel_1646722533460/work
parso @ file:///opt/conda/conda-bld/parso_1641458642106/work
partd @ file:///opt/conda/conda-bld/partd_1647245470509/work
pathlib @ file:///Users/ktietz/demo/mc3/conda-bld/pathlib_1629713961906/work
pathspec @ file:///tmp/abs_1foqurpsov/croots/recipe/pathspec_1659627126545/work
patsy==0.5.2
pep8==1.7.1
pexpect @ file:///tmp/build/80754af9/pexpect_1605563209008/work
pickleshare @ file:///tmp/build/80754af9/pickleshare_1606932040724/work
Pillow==9.2.0
pkginfo @ file:///tmp/build/80754af9/pkginfo_1643162084911/work
platformdirs @ file:///opt/conda/conda-bld/platformdirs_1662711380096/work
plotly @ file:///tmp/abs_7afcdfad-dbbb-49d2-adea-186abf525c45jbnd8p95/croots/recipe/plotly_1658160053621/work
pluggy @ file:///tmp/build/80754af9/pluggy_1648024445381/work
ply==3.11
poyo @ file:///tmp/build/80754af9/poyo_1617751526755/work
prometheus-client @ file:///tmp/abs_d3zeliano1/croots/recipe/prometheus_client_1659455100375/work
prompt-toolkit @ file:///tmp/build/80754af9/prompt-toolkit_1633440160888/work
Protego @ file:///tmp/build/80754af9/protego_1598657180827/work
protobuf==3.20.3
psutil==5.9.5
ptxcompiler-cu11==0.7.0.post1
ptyprocess @ file:///tmp/build/80754af9/ptyprocess_1609355006118/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
py @ file:///opt/conda/conda-bld/py_1644396412707/work
pyarrow==10.0.0
pyasn1 @ file:///Users/ktietz/demo/mc3/conda-bld/pyasn1_1629708007385/work
pyasn1-modules==0.2.8
pycodestyle @ file:///tmp/build/80754af9/pycodestyle_1636635402688/work
pycosat==0.6.3
pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work
pyct @ file:///tmp/abs_68a517ee-55fb-480e-82ab-1a8adb440a58x7qfc024/croots/recipe/pyct_1658500310800/work
pycuda==2022.2.2
pycurl==7.45.1
PyDispatcher==2.0.5
pydocstyle @ file:///tmp/build/80754af9/pydocstyle_1621600989141/work
pyerfa @ file:///tmp/build/80754af9/pyerfa_1621556109336/work
pyflakes @ file:///tmp/build/80754af9/pyflakes_1636644436481/work
Pygments==2.15.1
PyHamcrest @ file:///tmp/build/80754af9/pyhamcrest_1615748656804/work
PyJWT @ file:///opt/conda/conda-bld/pyjwt_1657544592787/work
pylibraft-cu11==23.2.0
pylint @ file:///tmp/abs_6fxmc66kyk/croots/recipe/pylint_1659110350161/work
pyls-spyder==0.4.0
pymp==0.0.6
pynvml==11.4.1
pyodbc @ file:///tmp/abs_d365zrcsdp/croots/recipe/pyodbc_1659513794382/work
pyOpenSSL @ file:///opt/conda/conda-bld/pyopenssl_1643788558760/work
pyparsing @ file:///opt/conda/conda-bld/pyparsing_1661452539315/work
PyQt5-sip==12.11.0
pyrsistent @ file:///tmp/build/80754af9/pyrsistent_1636110951836/work
PySocks @ file:///tmp/build/80754af9/pysocks_1605305812635/work
pytest==7.1.2
python-dateutil @ file:///tmp/build/80754af9/python-dateutil_1626374649649/work
python-lsp-black @ file:///opt/conda/conda-bld/python-lsp-black_1661852031497/work
python-lsp-jsonrpc==1.0.0
python-lsp-server @ file:///opt/conda/conda-bld/python-lsp-server_1661813814476/work
python-slugify @ file:///tmp/build/80754af9/python-slugify_1620405669636/work
python-snappy @ file:///tmp/build/80754af9/python-snappy_1610133040135/work
pytools==2022.1.14
pytz @ file:///opt/conda/conda-bld/pytz_1654762638606/work
pyviz-comms @ file:///tmp/build/80754af9/pyviz_comms_1623747165329/work
PyWavelets @ file:///tmp/build/80754af9/pywavelets_1648710015787/work
pyxdg @ file:///tmp/build/80754af9/pyxdg_1603822279816/work
PyYAML==6.0
pyzmq @ file:///opt/conda/conda-bld/pyzmq_1657724186960/work
QDarkStyle @ file:///tmp/build/80754af9/qdarkstyle_1617386714626/work
qstylizer @ file:///tmp/build/80754af9/qstylizer_1617713584600/work/dist/qstylizer-0.1.10-py2.py3-none-any.whl
QtAwesome @ file:///tmp/build/80754af9/qtawesome_1637160816833/work
qtconsole @ file:///opt/conda/conda-bld/qtconsole_1662018252641/work
QtPy @ file:///opt/conda/conda-bld/qtpy_1662014892439/work
queuelib==1.5.0
raft-dask-cu11==23.2.0
regex @ file:///tmp/abs_41f5bce5-0a2e-45aa-b231-1fd2fbd57753gfpe6sjm/croots/recipe/regex_1658257178822/work
requests @ file:///opt/conda/conda-bld/requests_1657734628632/work
requests-file @ file:///Users/ktietz/demo/mc3/conda-bld/requests-file_1629455781986/work
requests-oauthlib==1.3.1
rich==13.4.1
rmm-cu11==23.2.0
rope @ file:///opt/conda/conda-bld/rope_1643788605236/work
rsa==4.9
Rtree @ file:///tmp/build/80754af9/rtree_1618420843093/work
ruamel-yaml-conda @ file:///tmp/build/80754af9/ruamel_yaml_1616016711199/work
s3transfer @ file:///opt/conda/conda-bld/s3transfer_1654524197066/work
scalene==1.5.21.4
scann==1.2.9
scikit-cuda==0.5.3
scikit-image @ file:///tmp/build/80754af9/scikit-image_1648214171611/work
scikit-learn @ file:///tmp/build/80754af9/scikit-learn_1642617106979/work
scikit-learn-intelex==2021.20221004.171807
scipy==1.9.1
Scrapy @ file:///tmp/abs_e3bmwi01y8/croots/recipe/scrapy_1659598696235/work
seaborn @ file:///tmp/build/80754af9/seaborn_1629307859561/work
SecretStorage @ file:///tmp/build/80754af9/secretstorage_1614022780358/work
Send2Trash @ file:///tmp/build/80754af9/send2trash_1632406701022/work
service-identity @ file:///Users/ktietz/demo/mc3/conda-bld/service_identity_1629460757137/work
sip @ file:///tmp/abs_44cd77b_pu/croots/recipe/sip_1659012365470/work
six @ file:///tmp/build/80754af9/six_1644875935023/work
smart-open @ file:///opt/conda/conda-bld/smart_open_1651563547610/work
sniffio @ file:///tmp/build/80754af9/sniffio_1614030464178/work
snowballstemmer @ file:///tmp/build/80754af9/snowballstemmer_1637937080595/work
sortedcollections @ file:///tmp/build/80754af9/sortedcollections_1611172717284/work
sortedcontainers @ file:///tmp/build/80754af9/sortedcontainers_1623949099177/work
soupsieve @ file:///tmp/build/80754af9/soupsieve_1636706018808/work
Sphinx @ file:///opt/conda/conda-bld/sphinx_1657784123546/work
sphinxcontrib-applehelp @ file:///home/ktietz/src/ci/sphinxcontrib-applehelp_1611920841464/work
sphinxcontrib-devhelp @ file:///home/ktietz/src/ci/sphinxcontrib-devhelp_1611920923094/work
sphinxcontrib-htmlhelp @ file:///tmp/build/80754af9/sphinxcontrib-htmlhelp_1623945626792/work
sphinxcontrib-jsmath @ file:///home/ktietz/src/ci/sphinxcontrib-jsmath_1611920942228/work
sphinxcontrib-qthelp @ file:///home/ktietz/src/ci/sphinxcontrib-qthelp_1611921055322/work
sphinxcontrib-serializinghtml @ file:///tmp/build/80754af9/sphinxcontrib-serializinghtml_1624451540180/work
spyder @ file:///opt/conda/conda-bld/spyder_1663056818299/work
spyder-kernels @ file:///opt/conda/conda-bld/spyder-kernels_1662457880976/work
SQLAlchemy @ file:///tmp/abs_18b3238f-9c23-4182-a392-63af30a93c1er8j_yw60/croots/recipe/sqlalchemy_1657867856580/work
statsmodels @ file:///tmp/build/80754af9/statsmodels_1648015433305/work
sympy @ file:///tmp/build/80754af9/sympy_1647853653589/work
tables @ file:///tmp/build/80754af9/pytables_1607975397488/work
tabulate @ file:///opt/conda/conda-bld/tabulate_1657784105888/work
TBB==0.2
tblib @ file:///Users/ktietz/demo/mc3/conda-bld/tblib_1629402031467/work
tenacity @ file:///tmp/abs_9ca5cd06-f53b-4ea6-8eea-954e11157fddo5mhurpp/croots/recipe/tenacity_1657899103026/work
tensorboard==2.11.2
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.11.0
tensorflow-estimator==2.11.0
tensorflow-io-gcs-filesystem==0.31.0
termcolor==2.2.0
terminado @ file:///tmp/build/80754af9/terminado_1644322582718/work
testpath @ file:///opt/conda/conda-bld/testpath_1655908557405/work
text-unidecode @ file:///Users/ktietz/demo/mc3/conda-bld/text-unidecode_1629401354553/work
textdistance @ file:///tmp/build/80754af9/textdistance_1612461398012/work
threadpoolctl @ file:///Users/ktietz/demo/mc3/conda-bld/threadpoolctl_1629802263681/work
three-merge @ file:///tmp/build/80754af9/three-merge_1607553261110/work
tifffile @ file:///tmp/build/80754af9/tifffile_1627275862826/work
tinycss @ file:///tmp/build/80754af9/tinycss_1617713798712/work
tldextract @ file:///opt/conda/conda-bld/tldextract_1646638314385/work
toml @ file:///tmp/build/80754af9/toml_1616166611790/work
tomli @ file:///opt/conda/conda-bld/tomli_1657175507142/work
tomlkit @ file:///tmp/abs_56_0lnnq5x/croots/recipe/tomlkit_1658946880479/work
toolz @ file:///tmp/build/80754af9/toolz_1636545406491/work
torch==2.0.0
tornado @ file:///tmp/build/80754af9/tornado_1606942317143/work
tqdm @ file:///opt/conda/conda-bld/tqdm_1664392687731/work
traitlets @ file:///tmp/build/80754af9/traitlets_1636710298902/work
treelite==3.1.0
treelite-runtime==3.1.0
triton==2.0.0
Twisted @ file:///tmp/abs_82802zpkox/croots/recipe/twisted_1659592759417/work
typing_extensions @ file:///tmp/abs_ben9emwtky/croots/recipe/typing_extensions_1659638822008/work
ucx-py-cu11==0.30.0
ujson @ file:///opt/conda/conda-bld/ujson_1657544923770/work
Unidecode @ file:///tmp/build/80754af9/unidecode_1614712377438/work
urllib3 @ file:///tmp/abs_5dhwnz6atv/croots/recipe/urllib3_1659110457909/work
w3lib @ file:///Users/ktietz/demo/mc3/conda-bld/w3lib_1629359764703/work
watchdog @ file:///tmp/build/80754af9/watchdog_1638367282716/work
wcwidth @ file:///Users/ktietz/demo/mc3/conda-bld/wcwidth_1629357192024/work
webencodings==0.5.1
websocket-client @ file:///tmp/build/80754af9/websocket-client_1614803975924/work
Werkzeug @ file:///opt/conda/conda-bld/werkzeug_1645628268370/work
whatthepatch @ file:///opt/conda/conda-bld/whatthepatch_1661795988879/work
widgetsnbextension @ file:///tmp/build/80754af9/widgetsnbextension_1644992802045/work
wrapt @ file:///tmp/abs_c335821b-6e43-4504-9816-b1a52d3d3e1eel6uae8l/croots/recipe/wrapt_1657814400492/work
wurlitzer @ file:///tmp/build/80754af9/wurlitzer_1638368168359/work
xarray @ file:///opt/conda/conda-bld/xarray_1639166117697/work
xlrd @ file:///tmp/build/80754af9/xlrd_1608072521494/work
XlsxWriter @ file:///opt/conda/conda-bld/xlsxwriter_1649073856329/work
yapf @ file:///tmp/build/80754af9/yapf_1615749224965/work
zict==2.1.0
zipp @ file:///opt/conda/conda-bld/zipp_1652341764480/work
zope.interface @ file:///tmp/build/80754af9/zope.interface_1625036153595/work
emeryberger commentedon Jun 11, 2023
Please fill out the other details of this template:
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
Please include a minimum working example if at all possible.
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
If you have not yet tried with the repository version (
python3 -m pip install git+https://github.com/plasma-umass/scalene), please try that before reporting.Additional context
Add any other context about the problem here.
gongyguo commentedon Jun 11, 2023
repro:
import numpy as np
import cupy as cp
a= cp.ones(shape=(10,10))
u,s,v = cp.linalg.svd(a)
print(u,s,v)
you could reproduce the error on Ubuntu with Cuda version 11.6, cupy is a package for Numpy to run on GPU. I think cupy package is related to the issue.
error output:
Scalene: An exception of type UnicodeEncodeError occurred. Arguments:
('ascii', '\x1b[3m test.py: % of time = 100.00% (1.682s) out of 1.682s. \x1b[0m\n+-----------------------------------------------------------------------------------------------------------------------------+\n|\x1b[1m \x1b[0m|\x1b[1;34mTime\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;38;5;106m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\n|\x1b[1m \x1b[0m\x1b[1;2mLine\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mPython\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mnative\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34msystem\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;38;5;106mGPU\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1mtest.py \x1b[0m\x1b[1m \x1b[0m|\n|------+-------+-------+-------+-------+--------------------------------------------------------------------------------------|\n|\x1b[2m \x1b[0m\x1b[2m 1\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;0;0;128;48;2;0;0;0m#!/usr/bin/env python3\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 2\x1b[0m\x1b[2m \x1b[0m|\x1b[34m 2%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 2%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 1% \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m 23% \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;205;0;205;48;2;0;0;0mimport\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mnumpy\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;205;205;0;48;2;0;0;0mas\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mnp\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 3\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;0;0;128;48;2;0;0;0m#import math\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 4\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 5\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;0;0;128;48;2;0;0;0m# from numpy import linalg as LA\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 6\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 7\x1b[0m\x1b[2m \x1b[0m|\x1b[1;31m 8%\x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 4%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 3% \x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 60%\x1b[0m\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;205;0;205;48;2;0;0;0mimport\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcupy\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;205;205;0;48;2;0;0;0mas\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcp\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 8\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 9\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m 8%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 8% \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m 12% \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;204;204;204;48;2;0;0;0ma\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m=\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcp\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m.\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mones\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mshape\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m=\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;205;0;205;48;2;0;0;0m10\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;205;0;205;48;2;0;0;0m10\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 10\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 40%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 23% \x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 5%\x1b[0m\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;204;204;204;48;2;0;0;0mu\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0ms\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mv\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m=\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcp\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m.\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mlinalg\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m.\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0msvd\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0ma\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 11\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;205;0;205;48;2;0;0;0mprint\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mu\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0ms\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mv\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n+-----------------------------------------------------------------------------------------------------------------------------+\n', 322, 328, 'ordinal not in range(128)')
Traceback (most recent call last):
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 2059, in run_profiler
exit_status = profiler.profile_code(
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1861, in profile_code
did_output = Scalene.output_profile()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 971, in output_profile
did_output: bool = output.output_profiles(
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_output.py", line 637, in output_profiles
console.print(tbl)
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 1720, in print
self._buffer.extend(new_segments)
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 864, in exit
self._exit_buffer()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 822, in _exit_buffer
self._check_buffer()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 2060, in _check_buffer
self.file.write(text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 322-327: ordinal not in range(128)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***
Louis-Paul-Bowman commentedon Jul 14, 2023
Hey, I also think cupy might be at fault here. It would be nice to get this fixed (or at least some printing workaround) because it'll be very hard to seperate this from the app we're trying to profile.
I'm getting the same error in a slightly different circumstance.
Describe the bug
When scalene tries to print the fancy table to stdout (console.print(tbl)), it can cause rich text to throw a UnicodeEncodeError: 'ascii' codec can't encode characters in position XXX-YYY : ordinal not in range(128). My best guess is that it's some behavior related to cupy, a fairly common python library for placing operations on the GPU.
To Reproduce
1 - start with a computer with a NVIDIA GPU.
2 - requirements.txt:
scalene
cupy-cuda11x
spacy[cuda11x]==3.5
--extra-index-url https://download.pytorch.org/whl/cu118
torch
3 - pip install -r requirements.txt
4 - python -m spacy download "en_core_web_sm"
5 - working_example.py:
from spacy import require_cpu, load
require_cpu() #ensures that cupy is NOT used as the backend (for setting spacy ops)
nlp = load("en_core_web_sm")
6 - scalene --cli working_example.py
WORKS, scalene outputs a nice table of the profiling.
Line │Python │native │syst… │GPU │Python │peak │timeline/% │(MB/s) │working_example.py
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
1 │ 50% │ 33% │ 9% │ 88% │ 82% │ 272M │▁▂▂▃▃▃▄▄▅ 82% │ 434 │from spacy import require_cpu, load
2 │ │ │ │ │ │ │ │ │require_cpu() #ensures that cupy is NOT used as the backend (for setting spacy ops)
3 │ 6% │ │ 1% │ 12% │ 96% │ 61M │▂▂▂▂▂▂ 18% │ 48 │nlp = load("en_core_web_sm")
4 │ │ │ │ │ │ │ │ │
╵ ╵ ╵ ╵ ╵ ╵ ╵ ╵ ╵
Top AVERAGE memory consumption, by line:
(1) 1: 272 MB
Top PEAK memory consumption, by line:
(1) 1: 272 MB
(2) 3: 61 MB
Possible memory leak identified at line 1 (estimated likelihood: 95%, velocity: 11 MB/s)
7- broken_example.py
from spacy import require_gpu, load
require_gpu() #ensures that cupy IS used as the backend (for setting spacy ops)
nlp = load("en_core_web_sm")
8 - scalene --cli broken_example.py
Scalene: An exception of type UnicodeEncodeError occurred. Arguments:
('ascii', ' Memory usage: \x1b[32m▁▁▁▁▁▂▂▂▂▃▃▄▄▄▄▅▅▆█████████\x1b[0m (max: 633.836 MB, growth rate: 85%)
\n broken_example.py: % of time = 100.00% (25.644s) out of 25.644s.
\n+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\n|\x1b[1m \x1b[0m|\x1b[1;34mTime\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––…\x1b[0m\x1b[1m \x1b[0m|\x1b[1;33m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;32mMemory\x1b[0m\x1b[1m \x1b[0m|\x1b[1;32m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;32m–––––––––––\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;33mCopy\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\n|\x1b[1m \x1b[0m\x1b[1;2mLine\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mPython\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mnative\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34msyst…\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;33mGPU\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;32mPython\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;32mpeak\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;32mtimeline\x1b[0m\x1b[1;32m/%\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;33m(MB/s)\x1b[0m\x1b[1m \x1b[0m|\x1b[1mbroken_example.py \x1b[0m\x1b[1m \x1b[0m|\n|------+-------+-------+------+-------+-------+-------+---------------+-------+--------------------------------------------------------------------------------------------------------------------------------|\n|\x1b[2m \x1b[0m\x1b[2m 1\x1b[0m\x1b[2m \x1b[0m|\x1b[1;31m 48%\x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 32%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 9%\x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 89%\x1b[0m\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[32m 82% \x1b[0m\x1b[32m \x1b[0m|\x1b[32m 272M\x1b[0m\x1b[32m \x1b[0m|\x1b[1;31m▁▁▁▁▁▁▁▁▂ 37%\x1b[0m\x1b[32m \x1b[0m|\x1b[33m 415\x1b[0m\x1b[33m \x1b[0m|\x1b[35;40mfrom\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mspacy\x1b[0m\x1b[37;40m \x1b[0m\x1b[35;40mimport\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mrequire_gpu\x1b[0m\x1b[37;40m,\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mload\x1b[0m\x1b[40m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 2\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[37;40mrequire_gpu\x1b[0m\x1b[37;40m(\x1b[0m\x1b[37;40m)\x1b[0m\x1b[37;40m \x1b[0m\x1b[34;40m#ensures that cupy IS used as the backend (for setting spacy ops)\x1b[0m\x1b[40m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 3\x1b[0m\x1b[2m \x1b[0m|\x1b[34m 6%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 3%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 1%\x1b[0m\x1b[34m \x1b[0m|\x1b[33m 11% \x1b[0m\x1b[33m \x1b[0m|\x1b[32m 12% \x1b[0m\x1b[32m \x1b[0m|\x1b[32m 361M\x1b[0m\x1b[32m \x1b[0m|\x1b[32m▃▃▄▄▅▅▅▅▅ 63%\x1b[0m\x1b[32m \x1b[0m|\x1b[33m 65\x1b[0m\x1b[33m \x1b[0m|\x1b[37;40mnlp\x1b[0m\x1b[37;40m \x1b[0m\x1b[36;40m=\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mload\x1b[0m\x1b[37;40m(\x1b[0m\x1b[31;40m"\x1b[0m\x1b[31;40men_core_web_sm\x1b[0m\x1b[31;40m"\x1b[0m\x1b[37;40m)\x1b[0m\x1b[40m
\x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 4\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[40m
\x1b[0m |\n+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\n', 84, 111, 'ordinal not in range(128)')
Traceback (most recent call last):
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 2059, in run_profiler
exit_status = profiler.profile_code(
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1861, in profile_code
did_output = Scalene.output_profile()
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 971, in output_profile
did_output: bool = output.output_profiles(
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_output.py", line 637, in output_profiles
console.print(tbl)
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 1720, in print
self._buffer.extend(new_segments)
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 864, in exit
self._exit_buffer()
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 822, in _exit_buffer
self._check_buffer()
File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 2060, in _check_buffer
self.file.write(text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 84-110: ordinal not in range(128)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***
Expected behavior
Scalene successfully profiles the script and outputs the nearly identical table to the one obtained successfully in step 5.
Desktop (please complete the following information):
Additional context
I originally went down the rabbit hole of trying to fix the actual ut8 / ASCII codec error, but none of the fixes obtained from googling that except rich text raises worked for me. In particular:
So I think the actual unicode exception is either a red herring or maybe something in cupy resets the python's expectation to use the utf-8 codec?
Pip list
Package Version
absl-py 1.4.0
aiodns 3.0.0
aiohttp 3.8.4
aiosignal 1.3.1
alembic 1.11.1
amqp 5.1.1
aniso8601 9.0.1
asttokens 2.2.1
astunparse 1.6.3
async-timeout 4.0.2
asyncio 3.4.3
attrs 23.1.0
backcall 0.2.0
beautifulsoup4 4.12.2
billiard 4.1.0
blinker 1.4
blis 0.7.9
Brotli 1.0.9
cachetools 5.3.1
catalogue 2.0.8
cchardet 2.1.7
celery 5.3.1
certifi 2023.5.7
cffi 1.15.1
charset-normalizer 3.2.0
cli-formatter 1.2.0
click 8.1.5
click-didyoumean 0.3.0
click-plugins 1.1.1
click-repl 0.3.0
cloudpickle 2.2.1
cmake 3.26.4
comm 0.1.3
confection 0.1.0
contourpy 1.1.0
cryptography 41.0.2
cupy-cuda11x 11.6.0
cycler 0.11.0
cymem 2.0.7
debugpy 1.6.7
decorator 5.1.1
dnspython 2.3.0
email-validator 1.1.3
en-core-web-sm 3.5.0
en-core-web-trf 3.5.0
et-xmlfile 1.1.0
executing 1.2.0
fastrlock 0.8.1
filelock 3.12.2
Flask 2.0.2
Flask-Dance 5.1.0
Flask-Login 0.5.0
Flask-Migrate 3.1.0
Flask-Minify 0.37
flask-restx 0.5.1
Flask-SQLAlchemy 2.5.1
Flask-WTF 1.0.0
flatbuffers 23.5.26
fonttools 4.41.0
frozenlist 1.4.0
fsspec 2023.6.0
gast 0.4.0
google-auth 2.22.0
google-auth-oauthlib 0.4.6
google-pasta 0.2.0
greenlet 2.0.2
grpcio 1.56.0
gunicorn 20.1.0
h5py 3.9.0
htmlmin 0.1.12
huggingface-hub 0.16.4
idna 3.4
importlib-metadata 6.8.0
importlib-resources 6.0.0
inflate64 0.3.1
ipykernel 6.24.0
ipython 8.14.0
ipywidgets 8.0.7
itsdangerous 2.1.2
jedi 0.18.2
Jinja2 3.0.3
joblib 1.3.1
jsmin 3.0.1
jsonschema 4.18.2
jsonschema-specifications 2023.6.1
jupyter_client 8.3.0
jupyter_core 5.3.1
jupyterlab-widgets 3.0.8
keras 2.10.0
Keras-Preprocessing 1.1.2
kiwisolver 1.4.4
kombu 5.3.1
langcodes 3.3.0
lesscpy 0.15.1
libclang 16.0.0
lit 16.0.6
llvmlite 0.40.1
lxml 4.9.3
Mako 1.2.4
Markdown 3.4.3
markdown-it-py 3.0.0
MarkupSafe 2.1.3
matplotlib 3.7.2
matplotlib-inline 0.1.6
mdurl 0.1.2
minio 7.1.15
mpmath 1.3.0
multidict 6.0.4
multivolumefile 0.2.3
murmurhash 1.0.9
nest-asyncio 1.5.6
networkx 3.1
nltk 3.8.1
numba 0.57.1
numpy 1.24.4
nvidia-ml-py3 7.352.0
oauthlib 3.2.2
openpyxl 3.1.2
opt-einsum 3.3.0
packaging 23.1
pandas 2.0.3
parso 0.8.3
pathy 0.10.2
pdfminer.six 20221105
pexpect 4.8.0
pickleshare 0.7.5
Pillow 10.0.0
pip 23.1.2
platformdirs 3.8.1
ply 3.11
preshed 3.0.8
prompt-toolkit 3.0.39
protobuf 3.19.6
psutil 5.9.5
ptyprocess 0.7.0
pure-eval 0.2.2
py7zr 0.20.5
pyasn1 0.5.0
pyasn1-modules 0.3.0
pybcj 1.0.1
pycares 4.3.0
pycparser 2.21
pycryptodomex 3.18.0
pydantic 1.10.11
Pygments 2.15.1
pynvml 11.4.1
pyOpenSSL 23.2.0
pyparsing 3.0.9
PyPDF2 3.0.1
pyppmd 1.0.0
python-dateutil 2.8.2
python-docx 0.8.11
python-dotenv 0.19.2
pytz 2023.3
PyYAML 6.0
pyzmq 25.1.0
pyzstd 0.15.9
rapidfuzz 3.1.1
rcssmin 1.1.1
referencing 0.29.1
regex 2023.6.3
requests 2.31.0
requests-file 1.5.1
requests-oauthlib 1.3.1
rich 13.4.2
rpds-py 0.8.10
rsa 4.9
safetensors 0.3.1
scalene 1.5.21.4
scikit-learn 1.3.0
scipy 1.11.1
sense2vec 2.0.2
setuptools 58.1.0
six 1.16.0
smart-open 6.3.0
snakeviz 2.2.0
soupsieve 2.4.1
spacy 3.5.0
spacy-alignments 0.9.0
spacy-legacy 3.0.12
spacy-loggers 1.0.4
spacy-transformers 1.2.5
SQLAlchemy 1.4.29
srsly 2.4.6
stack-data 0.6.2
striprtf 0.0.25
sympy 1.12
tensorboard 2.10.1
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorflow 2.10.1
tensorflow-estimator 2.10.0
tensorflow-io-gcs-filesystem 0.32.0
termcolor 2.3.0
texttable 1.6.7
thinc 8.1.10
threadpoolctl 3.2.0
tldextract 3.4.4
tokenizers 0.13.3
torch 2.0.1+cu118
tornado 6.3.2
tqdm 4.65.0
traitlets 5.9.0
transformers 4.30.2
triton 2.0.0
typer 0.7.0
typing_extensions 4.7.1
tzdata 2023.3
urllib3 1.26.16
URLObject 2.4.3
uuid 1.30
validators 0.20.0
vine 5.0.0
wasabi 1.1.2
wcwidth 0.2.6
Werkzeug 2.0.3
wheel 0.40.0
widgetsnbextension 4.0.8
wrapt 1.15.0
WTForms 3.0.1
xxhash 3.2.0
yarl 1.9.2
zipp 3.16.1
emeryberger commentedon Jul 15, 2023
Thanks @Louis-Paul-Bowman for this detailed report! I was able to find the issue, which indeed is CuPy related (actually NVIDIA related): NVIDIA/cuda-python#29.
emeryberger commentedon Jul 15, 2023
Please test by running the current repo version (I will release a new version once confirmed):
python3 -m pip install -U git+https://github.com/plasma-umass/scalene)emeryberger commentedon Jul 15, 2023
@gongyguo I have successfully run your repro case with the patched version. Please test. Thanks!
Louis-Paul-Bowman commentedon Jul 15, 2023
Confirmed, fixed on my end when pulling the repo version.
Nice find! That is... cryptic behavior to say the least.
mwdotzom commentedon Jul 17, 2023
I can confirm that the workaround
export LC_ALL="POSIX"fixed my problem highly similar to this issue, millions of thanks!emeryberger commentedon Jul 17, 2023
New version released, thanks again for everyone's help in tracking this one down!
Fixes plasma-umass#625, which is due to a bug caused by NVIDIA (NVIDI…