Skip to content

how to fix the problem of encoding #625

Closed

Description

@gongyguo

Traceback (most recent call last):
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 2059, in run_profiler
exit_status = profiler.profile_code(
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1861, in profile_code
did_output = Scalene.output_profile()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 971, in output_profile
did_output: bool = output.output_profiles(
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_output.py", line 637, in output_profiles
console.print(tbl)
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 1720, in print
self._buffer.extend(new_segments)
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 864, in exit
self._exit_buffer()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 822, in _exit_buffer
self._check_buffer()
File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 2060, in _check_buffer
self.file.write(text)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-94: ordinal not in range(128)
*** You may need to add PYTHONIOENCODING=utf-8 to your environment ***

Activity

  1. emeryberger commented on Jun 11, 2023

    @emeryberger
    Member

    Looks like this error is occurring inside the rich package. Can you add the output of pip freeze? Also, we could use a repro case and more info about your system (this is in the issue template).

  2. gongyguo commented on Jun 11, 2023

    @gongyguo
    Author

    absl-py==1.4.0
    alabaster @ file:///home/ktietz/src/ci/alabaster_1611921544520/work
    anaconda-client==1.11.0
    anaconda-navigator==2.3.1
    anaconda-project @ file:///opt/conda/conda-bld/anaconda-project_1660339890420/work
    anyio @ file:///tmp/build/80754af9/anyio_1644463572971/work/dist
    appdirs==1.4.4
    argon2-cffi @ file:///opt/conda/conda-bld/argon2-cffi_1645000214183/work
    argon2-cffi-bindings @ file:///tmp/build/80754af9/argon2-cffi-bindings_1644569679365/work
    arrow @ file:///opt/conda/conda-bld/arrow_1649166651673/work
    astroid @ file:///tmp/abs_e5wkt48jiz/croots/recipe/astroid_1659023120113/work
    astropy @ file:///opt/conda/conda-bld/astropy_1657786094003/work
    astunparse==1.6.3
    atomicwrites==1.4.0
    attrs @ file:///opt/conda/conda-bld/attrs_1642510447205/work
    Automat @ file:///tmp/build/80754af9/automat_1600298431173/work
    autopep8 @ file:///opt/conda/conda-bld/autopep8_1650463822033/work
    Babel @ file:///tmp/build/80754af9/babel_1620871417480/work
    backcall @ file:///home/ktietz/src/ci/backcall_1611930011877/work
    backports.functools-lru-cache @ file:///tmp/build/80754af9/backports.functools_lru_cache_1618170165463/work
    backports.tempfile @ file:///home/linux1/recipes/ci/backports.tempfile_1610991236607/work
    backports.weakref==1.0.post1
    bcrypt @ file:///tmp/abs_6fpe92qzzo/croots/recipe/bcrypt_1659554336150/work
    beautifulsoup4 @ file:///opt/conda/conda-bld/beautifulsoup4_1650462163268/work
    binaryornot @ file:///tmp/build/80754af9/binaryornot_1617751525010/work
    bitarray @ file:///opt/conda/conda-bld/bitarray_1657739645104/work
    bkcharts==0.2
    black @ file:///opt/conda/conda-bld/black_1660237809219/work
    bleach @ file:///opt/conda/conda-bld/bleach_1641577558959/work
    bokeh @ file:///tmp/abs_34854e1f-d7d3-4f22-85d9-1075588e4ecdga64o0qg/croots/recipe/bokeh_1658136654619/work
    boto3 @ file:///tmp/abs_ae3c72db-af47-4298-baea-7270430e2c96scbpg1_h/croots/recipe/boto3_1657820109150/work
    botocore @ file:///opt/conda/conda-bld/botocore_1657739486257/work
    Bottleneck @ file:///opt/conda/conda-bld/bottleneck_1657175564434/work
    brotlipy==0.7.0
    cachetools==5.3.0
    certifi @ file:///opt/conda/conda-bld/certifi_1663615672595/work/certifi
    cffi @ file:///tmp/abs_98z5h56wf8/croots/recipe/cffi_1659598650955/work
    chardet @ file:///tmp/build/80754af9/chardet_1607706775000/work
    charset-normalizer @ file:///tmp/build/80754af9/charset-normalizer_1630003229654/work
    click @ file:///tmp/build/80754af9/click_1646056590078/work
    cloudpickle==2.2.1
    clyent==1.2.2
    cmake==3.26.1
    colorama @ file:///opt/conda/conda-bld/colorama_1657009087971/work
    colorcet @ file:///tmp/build/80754af9/colorcet_1651851439427/work
    conda==22.9.0
    conda-build==3.22.0
    conda-content-trust @ file:///tmp/abs_5952f1c8-355c-4855-ad2e-538535021ba5h26t22e5/croots/recipe/conda-content-trust_1658126371814/work
    conda-pack @ file:///tmp/build/80754af9/conda-pack_1611163042455/work
    conda-package-handling @ file:///opt/conda/conda-bld/conda-package-handling_1663598473529/work
    conda-repo-cli==1.0.20
    conda-token @ file:///Users/paulyim/miniconda3/envs/c3i/conda-bld/conda-token_1662660369760/work
    conda-verify==3.4.2
    constantly==15.1.0
    cookiecutter @ file:///opt/conda/conda-bld/cookiecutter_1649151442564/work
    cryptography @ file:///tmp/build/80754af9/cryptography_1652101588893/work
    cssselect==1.1.0
    cubinlinker-cu11==0.3.0.post1
    cuda-python==11.8.1
    cudf-cu11==23.2.0
    cuml-cu11==23.2.0
    cupy-cuda116==10.6.0
    cupy-cuda11x==11.6.0
    cycler @ file:///tmp/build/80754af9/cycler_1637851556182/work
    Cython @ file:///opt/conda/conda-bld/cython_1663692770955/work
    cytoolz==0.11.0
    daal4py==2021.6.0
    dask==2023.1.1
    dask-cuda==23.2.1
    dask-cudf-cu11==23.2.0
    datashader @ file:///tmp/abs_aa58dfo4_s/croots/recipe/datashader_1659349033064/work
    datashape==0.5.4
    debugpy @ file:///tmp/build/80754af9/debugpy_1637091799509/work
    decorator @ file:///opt/conda/conda-bld/decorator_1643638310831/work
    defusedxml @ file:///tmp/build/80754af9/defusedxml_1615228127516/work
    diff-match-patch @ file:///Users/ktietz/demo/mc3/conda-bld/diff-match-patch_1630511840874/work
    dill @ file:///tmp/build/80754af9/dill_1623919422540/work
    distributed==2023.1.1
    docutils @ file:///opt/conda/conda-bld/docutils_1657175430858/work
    entrypoints @ file:///tmp/build/80754af9/entrypoints_1649926439650/work
    et-xmlfile==1.1.0
    faiss==1.7.3
    fastjsonschema @ file:///opt/conda/conda-bld/python-fastjsonschema_1661371079312/work
    fastrlock==0.8.1
    filelock @ file:///opt/conda/conda-bld/filelock_1647002191454/work
    flake8 @ file:///opt/conda/conda-bld/flake8_1648129545443/work
    Flask @ file:///home/ktietz/src/ci/flask_1611932660458/work
    flatbuffers==23.3.3
    fonttools==4.25.0
    fsspec @ file:///opt/conda/conda-bld/fsspec_1659972197723/work
    future @ file:///tmp/build/80754af9/future_1607571303524/work
    gast==0.4.0
    gensim @ file:///tmp/build/80754af9/gensim_1646806807927/work
    glob2 @ file:///home/linux1/recipes/ci/glob2_1610991677669/work
    gmpy2 @ file:///tmp/build/80754af9/gmpy2_1645438755360/work
    google-auth==2.16.2
    google-auth-oauthlib==0.4.6
    google-pasta==0.2.0
    greenlet @ file:///tmp/build/80754af9/greenlet_1628888132713/work
    grpcio==1.51.3
    h5py @ file:///tmp/abs_4aewd3wzey/croots/recipe/h5py_1659091371897/work
    HeapDict @ file:///Users/ktietz/demo/mc3/conda-bld/heapdict_1630598515714/work
    holoviews @ file:///tmp/abs_eecc808c-455e-4be4-9911-ecf8341b3a34jfwskiqe/croots/recipe/holoviews_1658171506757/work
    hvplot @ file:///tmp/abs_6fcys5jcv1/croots/recipe/hvplot_1659026496554/work
    hyperlink @ file:///tmp/build/80754af9/hyperlink_1610130746837/work
    idna @ file:///tmp/build/80754af9/idna_1637925883363/work
    imagecodecs @ file:///opt/conda/conda-bld/imagecodecs_1664561985385/work
    imageio @ file:///tmp/abs_cd920173-f360-47c5-97b0-bf4d1076d5d4dvic0oys/croots/recipe/imageio_1658785036907/work
    imagesize @ file:///opt/conda/conda-bld/imagesize_1657179498843/work
    importlib-metadata @ file:///tmp/build/80754af9/importlib-metadata_1648544546694/work
    incremental @ file:///tmp/build/80754af9/incremental_1636629750599/work
    inflection==0.5.1
    iniconfig @ file:///home/linux1/recipes/ci/iniconfig_1610983019677/work
    intake @ file:///opt/conda/conda-bld/intake_1647436631684/work
    intervaltree @ file:///Users/ktietz/demo/mc3/conda-bld/intervaltree_1630511889664/work
    ipykernel @ file:///opt/conda/conda-bld/ipykernel_1662361798230/work
    ipython @ file:///tmp/abs_94gruux8u8/croots/recipe/ipython_1659529858706/work
    ipython-genutils @ file:///tmp/build/80754af9/ipython_genutils_1606773439826/work
    ipywidgets @ file:///tmp/build/80754af9/ipywidgets_1634143127070/work
    isort @ file:///tmp/build/80754af9/isort_1628603791788/work
    itemadapter @ file:///tmp/build/80754af9/itemadapter_1626442940632/work
    itemloaders @ file:///opt/conda/conda-bld/itemloaders_1646805235997/work
    itsdangerous @ file:///tmp/build/80754af9/itsdangerous_1621432558163/work
    jdcal @ file:///Users/ktietz/demo/mc3/conda-bld/jdcal_1630584345063/work
    jedi @ file:///tmp/build/80754af9/jedi_1644297102865/work
    jeepney @ file:///tmp/build/80754af9/jeepney_1627537048313/work
    jellyfish @ file:///tmp/build/80754af9/jellyfish_1647944426575/work
    Jinja2==3.1.2
    jinja2-time @ file:///opt/conda/conda-bld/jinja2-time_1649251842261/work
    jmespath @ file:///Users/ktietz/demo/mc3/conda-bld/jmespath_1630583964805/work
    joblib @ file:///tmp/build/80754af9/joblib_1635411271373/work
    json5 @ file:///tmp/build/80754af9/json5_1624432770122/work
    jsonschema @ file:///opt/conda/conda-bld/jsonschema_1663375472438/work
    jupyter @ file:///tmp/abs_33h4eoipez/croots/recipe/jupyter_1659349046347/work
    jupyter-console @ file:///opt/conda/conda-bld/jupyter_console_1647002188872/work
    jupyter-server @ file:///tmp/abs_b88b31b8-83b9-476d-a46d-e563c421f38fvsnyi1ur/croots/recipe/jupyter_server_1658754481507/work
    jupyter_client @ file:///opt/conda/conda-bld/jupyter_client_1661848916004/work
    jupyter_core @ file:///opt/conda/conda-bld/jupyter_core_1664917302524/work
    jupyterlab @ file:///tmp/abs_12f3h01vmy/croots/recipe/jupyterlab_1658907535764/work
    jupyterlab-pygments @ file:///tmp/build/80754af9/jupyterlab_pygments_1601490720602/work
    jupyterlab-server @ file:///opt/conda/conda-bld/jupyterlab_server_1644500396812/work
    jupyterlab-widgets @ file:///tmp/build/80754af9/jupyterlab_widgets_1609884341231/work
    keras==2.11.0
    keyring @ file:///tmp/build/80754af9/keyring_1638531355686/work
    kiwisolver @ file:///opt/conda/conda-bld/kiwisolver_1653292039266/work
    lazy-object-proxy @ file:///tmp/build/80754af9/lazy-object-proxy_1616529027849/work
    libarchive-c @ file:///tmp/build/80754af9/python-libarchive-c_1617780486945/work
    libclang==15.0.6.1
    lit==16.0.0
    llvmlite==0.39.1
    locket @ file:///opt/conda/conda-bld/locket_1652903118915/work
    lxml @ file:///opt/conda/conda-bld/lxml_1657545139709/work
    lz4 @ file:///tmp/build/80754af9/lz4_1619516502891/work
    Mako==1.2.4
    Markdown @ file:///tmp/build/80754af9/markdown_1614363852612/work
    markdown-it-py==2.2.0
    MarkupSafe @ file:///tmp/build/80754af9/markupsafe_1621523467000/work
    matplotlib @ file:///opt/conda/conda-bld/matplotlib-suite_1660167928326/work
    matplotlib-inline @ file:///opt/conda/conda-bld/matplotlib-inline_1662014470464/work
    mccabe @ file:///opt/conda/conda-bld/mccabe_1644221741721/work
    mdurl==0.1.2
    mistune @ file:///tmp/build/80754af9/mistune_1607364877025/work
    mkl-fft==1.3.1
    mkl-random @ file:///tmp/build/80754af9/mkl_random_1626186066731/work
    mkl-service==2.4.0
    mock @ file:///tmp/build/80754af9/mock_1607622725907/work
    mpmath==1.2.1
    msgpack @ file:///opt/conda/conda-bld/msgpack-python_1652362659880/work
    multipledispatch @ file:///tmp/build/80754af9/multipledispatch_1607574243360/work
    munkres==1.1.4
    mypy-extensions==0.4.3
    navigator-updater==0.3.0
    nbclassic @ file:///opt/conda/conda-bld/nbclassic_1644943264176/work
    nbclient @ file:///tmp/build/80754af9/nbclient_1650290509967/work
    nbconvert @ file:///opt/conda/conda-bld/nbconvert_1649751911790/work
    nbformat @ file:///opt/conda/conda-bld/nbformat_1663744952973/work
    nest-asyncio @ file:///tmp/build/80754af9/nest-asyncio_1649847906199/work
    networkx @ file:///opt/conda/conda-bld/networkx_1657784097507/work
    nltk @ file:///opt/conda/conda-bld/nltk_1645628263994/work
    nose @ file:///opt/conda/conda-bld/nose_1642704612149/work
    notebook @ file:///tmp/abs_abf6xa6h6f/croots/recipe/notebook_1659083654985/work
    numba==0.56.4
    numexpr @ file:///opt/conda/conda-bld/numexpr_1656940300424/work
    numpy @ file:///opt/conda/conda-bld/numpy_and_numpy_base_1653915516269/work
    numpydoc @ file:///opt/conda/conda-bld/numpydoc_1657529872251/work
    nvidia-cublas-cu11==11.10.3.66
    nvidia-cuda-cupti-cu11==11.7.101
    nvidia-cuda-nvrtc-cu11==11.7.99
    nvidia-cuda-runtime-cu11==11.7.99
    nvidia-cudnn-cu11==8.5.0.96
    nvidia-cufft-cu11==10.9.0.58
    nvidia-curand-cu11==10.2.10.91
    nvidia-cusolver-cu11==11.4.0.1
    nvidia-cusparse-cu11==11.7.4.91
    nvidia-nccl-cu11==2.14.3
    nvidia-nvtx-cu11==11.7.91
    nvtx==0.2.5
    oauthlib==3.2.2
    olefile @ file:///Users/ktietz/demo/mc3/conda-bld/olefile_1629805411829/work
    openpyxl==3.0.10
    opt-einsum==3.3.0
    packaging @ file:///tmp/build/80754af9/packaging_1637314298585/work
    pandas==1.4.4
    pandocfilters @ file:///opt/conda/conda-bld/pandocfilters_1643405455980/work
    panel @ file:///tmp/abs_bb3d3b2f-b3ea-41c0-a72e-8f54852d5cdfs70inytz/croots/recipe/panel_1658133826470/work
    param @ file:///tmp/build/80754af9/param_1636647414893/work
    parsel @ file:///tmp/build/80754af9/parsel_1646722533460/work
    parso @ file:///opt/conda/conda-bld/parso_1641458642106/work
    partd @ file:///opt/conda/conda-bld/partd_1647245470509/work
    pathlib @ file:///Users/ktietz/demo/mc3/conda-bld/pathlib_1629713961906/work
    pathspec @ file:///tmp/abs_1foqurpsov/croots/recipe/pathspec_1659627126545/work
    patsy==0.5.2
    pep8==1.7.1
    pexpect @ file:///tmp/build/80754af9/pexpect_1605563209008/work
    pickleshare @ file:///tmp/build/80754af9/pickleshare_1606932040724/work
    Pillow==9.2.0
    pkginfo @ file:///tmp/build/80754af9/pkginfo_1643162084911/work
    platformdirs @ file:///opt/conda/conda-bld/platformdirs_1662711380096/work
    plotly @ file:///tmp/abs_7afcdfad-dbbb-49d2-adea-186abf525c45jbnd8p95/croots/recipe/plotly_1658160053621/work
    pluggy @ file:///tmp/build/80754af9/pluggy_1648024445381/work
    ply==3.11
    poyo @ file:///tmp/build/80754af9/poyo_1617751526755/work
    prometheus-client @ file:///tmp/abs_d3zeliano1/croots/recipe/prometheus_client_1659455100375/work
    prompt-toolkit @ file:///tmp/build/80754af9/prompt-toolkit_1633440160888/work
    Protego @ file:///tmp/build/80754af9/protego_1598657180827/work
    protobuf==3.20.3
    psutil==5.9.5
    ptxcompiler-cu11==0.7.0.post1
    ptyprocess @ file:///tmp/build/80754af9/ptyprocess_1609355006118/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
    py @ file:///opt/conda/conda-bld/py_1644396412707/work
    pyarrow==10.0.0
    pyasn1 @ file:///Users/ktietz/demo/mc3/conda-bld/pyasn1_1629708007385/work
    pyasn1-modules==0.2.8
    pycodestyle @ file:///tmp/build/80754af9/pycodestyle_1636635402688/work
    pycosat==0.6.3
    pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work
    pyct @ file:///tmp/abs_68a517ee-55fb-480e-82ab-1a8adb440a58x7qfc024/croots/recipe/pyct_1658500310800/work
    pycuda==2022.2.2
    pycurl==7.45.1
    PyDispatcher==2.0.5
    pydocstyle @ file:///tmp/build/80754af9/pydocstyle_1621600989141/work
    pyerfa @ file:///tmp/build/80754af9/pyerfa_1621556109336/work
    pyflakes @ file:///tmp/build/80754af9/pyflakes_1636644436481/work
    Pygments==2.15.1
    PyHamcrest @ file:///tmp/build/80754af9/pyhamcrest_1615748656804/work
    PyJWT @ file:///opt/conda/conda-bld/pyjwt_1657544592787/work
    pylibraft-cu11==23.2.0
    pylint @ file:///tmp/abs_6fxmc66kyk/croots/recipe/pylint_1659110350161/work
    pyls-spyder==0.4.0
    pymp==0.0.6
    pynvml==11.4.1
    pyodbc @ file:///tmp/abs_d365zrcsdp/croots/recipe/pyodbc_1659513794382/work
    pyOpenSSL @ file:///opt/conda/conda-bld/pyopenssl_1643788558760/work
    pyparsing @ file:///opt/conda/conda-bld/pyparsing_1661452539315/work
    PyQt5-sip==12.11.0
    pyrsistent @ file:///tmp/build/80754af9/pyrsistent_1636110951836/work
    PySocks @ file:///tmp/build/80754af9/pysocks_1605305812635/work
    pytest==7.1.2
    python-dateutil @ file:///tmp/build/80754af9/python-dateutil_1626374649649/work
    python-lsp-black @ file:///opt/conda/conda-bld/python-lsp-black_1661852031497/work
    python-lsp-jsonrpc==1.0.0
    python-lsp-server @ file:///opt/conda/conda-bld/python-lsp-server_1661813814476/work
    python-slugify @ file:///tmp/build/80754af9/python-slugify_1620405669636/work
    python-snappy @ file:///tmp/build/80754af9/python-snappy_1610133040135/work
    pytools==2022.1.14
    pytz @ file:///opt/conda/conda-bld/pytz_1654762638606/work
    pyviz-comms @ file:///tmp/build/80754af9/pyviz_comms_1623747165329/work
    PyWavelets @ file:///tmp/build/80754af9/pywavelets_1648710015787/work
    pyxdg @ file:///tmp/build/80754af9/pyxdg_1603822279816/work
    PyYAML==6.0
    pyzmq @ file:///opt/conda/conda-bld/pyzmq_1657724186960/work
    QDarkStyle @ file:///tmp/build/80754af9/qdarkstyle_1617386714626/work
    qstylizer @ file:///tmp/build/80754af9/qstylizer_1617713584600/work/dist/qstylizer-0.1.10-py2.py3-none-any.whl
    QtAwesome @ file:///tmp/build/80754af9/qtawesome_1637160816833/work
    qtconsole @ file:///opt/conda/conda-bld/qtconsole_1662018252641/work
    QtPy @ file:///opt/conda/conda-bld/qtpy_1662014892439/work
    queuelib==1.5.0
    raft-dask-cu11==23.2.0
    regex @ file:///tmp/abs_41f5bce5-0a2e-45aa-b231-1fd2fbd57753gfpe6sjm/croots/recipe/regex_1658257178822/work
    requests @ file:///opt/conda/conda-bld/requests_1657734628632/work
    requests-file @ file:///Users/ktietz/demo/mc3/conda-bld/requests-file_1629455781986/work
    requests-oauthlib==1.3.1
    rich==13.4.1
    rmm-cu11==23.2.0
    rope @ file:///opt/conda/conda-bld/rope_1643788605236/work
    rsa==4.9
    Rtree @ file:///tmp/build/80754af9/rtree_1618420843093/work
    ruamel-yaml-conda @ file:///tmp/build/80754af9/ruamel_yaml_1616016711199/work
    s3transfer @ file:///opt/conda/conda-bld/s3transfer_1654524197066/work
    scalene==1.5.21.4
    scann==1.2.9
    scikit-cuda==0.5.3
    scikit-image @ file:///tmp/build/80754af9/scikit-image_1648214171611/work
    scikit-learn @ file:///tmp/build/80754af9/scikit-learn_1642617106979/work
    scikit-learn-intelex==2021.20221004.171807
    scipy==1.9.1
    Scrapy @ file:///tmp/abs_e3bmwi01y8/croots/recipe/scrapy_1659598696235/work
    seaborn @ file:///tmp/build/80754af9/seaborn_1629307859561/work
    SecretStorage @ file:///tmp/build/80754af9/secretstorage_1614022780358/work
    Send2Trash @ file:///tmp/build/80754af9/send2trash_1632406701022/work
    service-identity @ file:///Users/ktietz/demo/mc3/conda-bld/service_identity_1629460757137/work
    sip @ file:///tmp/abs_44cd77b_pu/croots/recipe/sip_1659012365470/work
    six @ file:///tmp/build/80754af9/six_1644875935023/work
    smart-open @ file:///opt/conda/conda-bld/smart_open_1651563547610/work
    sniffio @ file:///tmp/build/80754af9/sniffio_1614030464178/work
    snowballstemmer @ file:///tmp/build/80754af9/snowballstemmer_1637937080595/work
    sortedcollections @ file:///tmp/build/80754af9/sortedcollections_1611172717284/work
    sortedcontainers @ file:///tmp/build/80754af9/sortedcontainers_1623949099177/work
    soupsieve @ file:///tmp/build/80754af9/soupsieve_1636706018808/work
    Sphinx @ file:///opt/conda/conda-bld/sphinx_1657784123546/work
    sphinxcontrib-applehelp @ file:///home/ktietz/src/ci/sphinxcontrib-applehelp_1611920841464/work
    sphinxcontrib-devhelp @ file:///home/ktietz/src/ci/sphinxcontrib-devhelp_1611920923094/work
    sphinxcontrib-htmlhelp @ file:///tmp/build/80754af9/sphinxcontrib-htmlhelp_1623945626792/work
    sphinxcontrib-jsmath @ file:///home/ktietz/src/ci/sphinxcontrib-jsmath_1611920942228/work
    sphinxcontrib-qthelp @ file:///home/ktietz/src/ci/sphinxcontrib-qthelp_1611921055322/work
    sphinxcontrib-serializinghtml @ file:///tmp/build/80754af9/sphinxcontrib-serializinghtml_1624451540180/work
    spyder @ file:///opt/conda/conda-bld/spyder_1663056818299/work
    spyder-kernels @ file:///opt/conda/conda-bld/spyder-kernels_1662457880976/work
    SQLAlchemy @ file:///tmp/abs_18b3238f-9c23-4182-a392-63af30a93c1er8j_yw60/croots/recipe/sqlalchemy_1657867856580/work
    statsmodels @ file:///tmp/build/80754af9/statsmodels_1648015433305/work
    sympy @ file:///tmp/build/80754af9/sympy_1647853653589/work
    tables @ file:///tmp/build/80754af9/pytables_1607975397488/work
    tabulate @ file:///opt/conda/conda-bld/tabulate_1657784105888/work
    TBB==0.2
    tblib @ file:///Users/ktietz/demo/mc3/conda-bld/tblib_1629402031467/work
    tenacity @ file:///tmp/abs_9ca5cd06-f53b-4ea6-8eea-954e11157fddo5mhurpp/croots/recipe/tenacity_1657899103026/work
    tensorboard==2.11.2
    tensorboard-data-server==0.6.1
    tensorboard-plugin-wit==1.8.1
    tensorflow==2.11.0
    tensorflow-estimator==2.11.0
    tensorflow-io-gcs-filesystem==0.31.0
    termcolor==2.2.0
    terminado @ file:///tmp/build/80754af9/terminado_1644322582718/work
    testpath @ file:///opt/conda/conda-bld/testpath_1655908557405/work
    text-unidecode @ file:///Users/ktietz/demo/mc3/conda-bld/text-unidecode_1629401354553/work
    textdistance @ file:///tmp/build/80754af9/textdistance_1612461398012/work
    threadpoolctl @ file:///Users/ktietz/demo/mc3/conda-bld/threadpoolctl_1629802263681/work
    three-merge @ file:///tmp/build/80754af9/three-merge_1607553261110/work
    tifffile @ file:///tmp/build/80754af9/tifffile_1627275862826/work
    tinycss @ file:///tmp/build/80754af9/tinycss_1617713798712/work
    tldextract @ file:///opt/conda/conda-bld/tldextract_1646638314385/work
    toml @ file:///tmp/build/80754af9/toml_1616166611790/work
    tomli @ file:///opt/conda/conda-bld/tomli_1657175507142/work
    tomlkit @ file:///tmp/abs_56_0lnnq5x/croots/recipe/tomlkit_1658946880479/work
    toolz @ file:///tmp/build/80754af9/toolz_1636545406491/work
    torch==2.0.0
    tornado @ file:///tmp/build/80754af9/tornado_1606942317143/work
    tqdm @ file:///opt/conda/conda-bld/tqdm_1664392687731/work
    traitlets @ file:///tmp/build/80754af9/traitlets_1636710298902/work
    treelite==3.1.0
    treelite-runtime==3.1.0
    triton==2.0.0
    Twisted @ file:///tmp/abs_82802zpkox/croots/recipe/twisted_1659592759417/work
    typing_extensions @ file:///tmp/abs_ben9emwtky/croots/recipe/typing_extensions_1659638822008/work
    ucx-py-cu11==0.30.0
    ujson @ file:///opt/conda/conda-bld/ujson_1657544923770/work
    Unidecode @ file:///tmp/build/80754af9/unidecode_1614712377438/work
    urllib3 @ file:///tmp/abs_5dhwnz6atv/croots/recipe/urllib3_1659110457909/work
    w3lib @ file:///Users/ktietz/demo/mc3/conda-bld/w3lib_1629359764703/work
    watchdog @ file:///tmp/build/80754af9/watchdog_1638367282716/work
    wcwidth @ file:///Users/ktietz/demo/mc3/conda-bld/wcwidth_1629357192024/work
    webencodings==0.5.1
    websocket-client @ file:///tmp/build/80754af9/websocket-client_1614803975924/work
    Werkzeug @ file:///opt/conda/conda-bld/werkzeug_1645628268370/work
    whatthepatch @ file:///opt/conda/conda-bld/whatthepatch_1661795988879/work
    widgetsnbextension @ file:///tmp/build/80754af9/widgetsnbextension_1644992802045/work
    wrapt @ file:///tmp/abs_c335821b-6e43-4504-9816-b1a52d3d3e1eel6uae8l/croots/recipe/wrapt_1657814400492/work
    wurlitzer @ file:///tmp/build/80754af9/wurlitzer_1638368168359/work
    xarray @ file:///opt/conda/conda-bld/xarray_1639166117697/work
    xlrd @ file:///tmp/build/80754af9/xlrd_1608072521494/work
    XlsxWriter @ file:///opt/conda/conda-bld/xlsxwriter_1649073856329/work
    yapf @ file:///tmp/build/80754af9/yapf_1615749224965/work
    zict==2.1.0
    zipp @ file:///opt/conda/conda-bld/zipp_1652341764480/work
    zope.interface @ file:///tmp/build/80754af9/zope.interface_1625036153595/work

  3. emeryberger commented on Jun 11, 2023

    @emeryberger
    Member

    Please fill out the other details of this template:

    Describe the bug
    A clear and concise description of what the bug is.

    To Reproduce
    Steps to reproduce the behavior:

    1. Go to '...'
    2. Click on '....'
    3. Scroll down to '....'
    4. See error

    Please include a minimum working example if at all possible.

    Expected behavior
    A clear and concise description of what you expected to happen.

    Screenshots
    If applicable, add screenshots to help explain your problem.

    Desktop (please complete the following information):

    • OS: [e.g. iOS]
    • Browser [e.g. chrome, safari]
    • Version [e.g. 22]

    If you have not yet tried with the repository version (python3 -m pip install git+https://github.com/plasma-umass/scalene), please try that before reporting.

    Additional context
    Add any other context about the problem here.

  4. gongyguo commented on Jun 11, 2023

    @gongyguo
    Author

    repro:
    import numpy as np
    import cupy as cp
    a= cp.ones(shape=(10,10))
    u,s,v = cp.linalg.svd(a)
    print(u,s,v)

    you could reproduce the error on Ubuntu with Cuda version 11.6, cupy is a package for Numpy to run on GPU. I think cupy package is related to the issue.

    error output:
    Scalene: An exception of type UnicodeEncodeError occurred. Arguments:
    ('ascii', '\x1b[3m test.py: % of time = 100.00% (1.682s) out of 1.682s. \x1b[0m\n+-----------------------------------------------------------------------------------------------------------------------------+\n|\x1b[1m \x1b[0m|\x1b[1;34mTime\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;38;5;106m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\n|\x1b[1m \x1b[0m\x1b[1;2mLine\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mPython\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mnative\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34msystem\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;38;5;106mGPU\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1mtest.py \x1b[0m\x1b[1m \x1b[0m|\n|------+-------+-------+-------+-------+--------------------------------------------------------------------------------------|\n|\x1b[2m \x1b[0m\x1b[2m 1\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;0;0;128;48;2;0;0;0m#!/usr/bin/env python3\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 2\x1b[0m\x1b[2m \x1b[0m|\x1b[34m 2%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 2%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 1% \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m 23% \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;205;0;205;48;2;0;0;0mimport\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mnumpy\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;205;205;0;48;2;0;0;0mas\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mnp\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 3\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;0;0;128;48;2;0;0;0m#import math\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 4\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 5\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;0;0;128;48;2;0;0;0m# from numpy import linalg as LA\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 6\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 7\x1b[0m\x1b[2m \x1b[0m|\x1b[1;31m 8%\x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 4%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 3% \x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 60%\x1b[0m\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;205;0;205;48;2;0;0;0mimport\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcupy\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;205;205;0;48;2;0;0;0mas\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcp\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 8\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 9\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m 8%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 8% \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m 12% \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;204;204;204;48;2;0;0;0ma\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m=\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcp\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m.\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mones\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mshape\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m=\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;205;0;205;48;2;0;0;0m10\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;205;0;205;48;2;0;0;0m10\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 10\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 40%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 23% \x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 5%\x1b[0m\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;204;204;204;48;2;0;0;0mu\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0ms\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mv\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m=\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m \x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mcp\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m.\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mlinalg\x1b[0m\x1b[38;2;51;153;204;48;2;0;0;0m.\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0msvd\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0ma\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 11\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[38;5;106m \x1b[0m\x1b[38;5;106m \x1b[0m|\x1b[38;2;205;0;205;48;2;0;0;0mprint\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m(\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mu\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0ms\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m,\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0mv\x1b[0m\x1b[38;2;204;204;204;48;2;0;0;0m)\x1b[0m\x1b[48;2;0;0;0m \x1b[0m |\n+-----------------------------------------------------------------------------------------------------------------------------+\n', 322, 328, 'ordinal not in range(128)')
    Traceback (most recent call last):
    File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 2059, in run_profiler
    exit_status = profiler.profile_code(
    File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1861, in profile_code
    did_output = Scalene.output_profile()
    File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 971, in output_profile
    did_output: bool = output.output_profiles(
    File "/home/gongyao/anaconda3/lib/python3.9/site-packages/scalene/scalene_output.py", line 637, in output_profiles
    console.print(tbl)
    File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 1720, in print
    self._buffer.extend(new_segments)
    File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 864, in exit
    self._exit_buffer()
    File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 822, in _exit_buffer
    self._check_buffer()
    File "/home/gongyao/anaconda3/lib/python3.9/site-packages/rich/console.py", line 2060, in _check_buffer
    self.file.write(text)
    UnicodeEncodeError: 'ascii' codec can't encode characters in position 322-327: ordinal not in range(128)
    *** You may need to add PYTHONIOENCODING=utf-8 to your environment ***

  5. Louis-Paul-Bowman commented on Jul 14, 2023

    @Louis-Paul-Bowman

    Hey, I also think cupy might be at fault here. It would be nice to get this fixed (or at least some printing workaround) because it'll be very hard to seperate this from the app we're trying to profile.
    I'm getting the same error in a slightly different circumstance.

    Describe the bug
    When scalene tries to print the fancy table to stdout (console.print(tbl)), it can cause rich text to throw a UnicodeEncodeError: 'ascii' codec can't encode characters in position XXX-YYY : ordinal not in range(128). My best guess is that it's some behavior related to cupy, a fairly common python library for placing operations on the GPU.

    To Reproduce
    1 - start with a computer with a NVIDIA GPU.

    2 - requirements.txt:
    scalene
    cupy-cuda11x
    spacy[cuda11x]==3.5
    --extra-index-url https://download.pytorch.org/whl/cu118
    torch

    3 - pip install -r requirements.txt

    4 - python -m spacy download "en_core_web_sm"

    5 - working_example.py:
    from spacy import require_cpu, load
    require_cpu() #ensures that cupy is NOT used as the backend (for setting spacy ops)
    nlp = load("en_core_web_sm")

    6 - scalene --cli working_example.py
    WORKS, scalene outputs a nice table of the profiling.

                                                                 Memory usage: ▁▁▁▁▂▂▃▃▃▃▄▄▄▅▅▅▅▆▇▇▇▇█████ (max: 333.114 MB, growth rate: 100%)
                                                                       working_example.py: % of time = 100.00% (24.485s) out of 24.485s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
    

    Line │Python │native │syst… │GPU │Python │peak │timeline/% │(MB/s) │working_example.py
    ╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
    1 │ 50% │ 33% │ 9% │ 88% │ 82% │ 272M │▁▂▂▃▃▃▄▄▅ 82% │ 434 │from spacy import require_cpu, load
    2 │ │ │ │ │ │ │ │ │require_cpu() #ensures that cupy is NOT used as the backend (for setting spacy ops)
    3 │ 6% │ │ 1% │ 12% │ 96% │ 61M │▂▂▂▂▂▂ 18% │ 48 │nlp = load("en_core_web_sm")
    4 │ │ │ │ │ │ │ │ │
    ╵ ╵ ╵ ╵ ╵ ╵ ╵ ╵ ╵
    Top AVERAGE memory consumption, by line:
    (1) 1: 272 MB
    Top PEAK memory consumption, by line:
    (1) 1: 272 MB
    (2) 3: 61 MB
    Possible memory leak identified at line 1 (estimated likelihood: 95%, velocity: 11 MB/s)

    7- broken_example.py
    from spacy import require_gpu, load
    require_gpu() #ensures that cupy IS used as the backend (for setting spacy ops)
    nlp = load("en_core_web_sm")

    8 - scalene --cli broken_example.py

    Scalene: An exception of type UnicodeEncodeError occurred. Arguments:
    ('ascii', ' Memory usage: \x1b[32m▁▁▁▁▁▂▂▂▂▃▃▄▄▄▄▅▅▆█████████\x1b[0m (max: 633.836 MB, growth rate: 85%)
    \n broken_example.py: % of time = 100.00% (25.644s) out of 25.644s.
    \n+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\n|\x1b[1m \x1b[0m|\x1b[1;34mTime\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;34m––––…\x1b[0m\x1b[1m \x1b[0m|\x1b[1;33m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;32mMemory\x1b[0m\x1b[1m \x1b[0m|\x1b[1;32m––––––\x1b[0m\x1b[1m \x1b[0m|\x1b[1;32m–––––––––––\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;33mCopy\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\n|\x1b[1m \x1b[0m\x1b[1;2mLine\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mPython\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34mnative\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;34msyst…\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;33mGPU\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;32mPython\x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;32mpeak\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;32mtimeline\x1b[0m\x1b[1;32m/%\x1b[0m\x1b[1m \x1b[0m\x1b[1m \x1b[0m|\x1b[1;3;33m(MB/s)\x1b[0m\x1b[1m \x1b[0m|\x1b[1mbroken_example.py \x1b[0m\x1b[1m \x1b[0m|\n|------+-------+-------+------+-------+-------+-------+---------------+-------+--------------------------------------------------------------------------------------------------------------------------------|\n|\x1b[2m \x1b[0m\x1b[2m 1\x1b[0m\x1b[2m \x1b[0m|\x1b[1;31m 48%\x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 32%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 9%\x1b[0m\x1b[34m \x1b[0m|\x1b[1;31m 89%\x1b[0m\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[32m 82% \x1b[0m\x1b[32m \x1b[0m|\x1b[32m 272M\x1b[0m\x1b[32m \x1b[0m|\x1b[1;31m▁▁▁▁▁▁▁▁▂ 37%\x1b[0m\x1b[32m \x1b[0m|\x1b[33m 415\x1b[0m\x1b[33m \x1b[0m|\x1b[35;40mfrom\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mspacy\x1b[0m\x1b[37;40m \x1b[0m\x1b[35;40mimport\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mrequire_gpu\x1b[0m\x1b[37;40m,\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mload\x1b[0m\x1b[40m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 2\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[37;40mrequire_gpu\x1b[0m\x1b[37;40m(\x1b[0m\x1b[37;40m)\x1b[0m\x1b[37;40m \x1b[0m\x1b[34;40m#ensures that cupy IS used as the backend (for setting spacy ops)\x1b[0m\x1b[40m \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 3\x1b[0m\x1b[2m \x1b[0m|\x1b[34m 6%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 3%\x1b[0m\x1b[34m \x1b[0m|\x1b[34m 1%\x1b[0m\x1b[34m \x1b[0m|\x1b[33m 11% \x1b[0m\x1b[33m \x1b[0m|\x1b[32m 12% \x1b[0m\x1b[32m \x1b[0m|\x1b[32m 361M\x1b[0m\x1b[32m \x1b[0m|\x1b[32m▃▃▄▄▅▅▅▅▅ 63%\x1b[0m\x1b[32m \x1b[0m|\x1b[33m 65\x1b[0m\x1b[33m \x1b[0m|\x1b[37;40mnlp\x1b[0m\x1b[37;40m \x1b[0m\x1b[36;40m=\x1b[0m\x1b[37;40m \x1b[0m\x1b[37;40mload\x1b[0m\x1b[37;40m(\x1b[0m\x1b[31;40m"\x1b[0m\x1b[31;40men_core_web_sm\x1b[0m\x1b[31;40m"\x1b[0m\x1b[37;40m)\x1b[0m\x1b[40m
    \x1b[0m |\n|\x1b[2m \x1b[0m\x1b[2m 4\x1b[0m\x1b[2m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[34m \x1b[0m\x1b[34m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[32m \x1b[0m\x1b[32m \x1b[0m|\x1b[33m \x1b[0m\x1b[33m \x1b[0m|\x1b[40m
    \x1b[0m |\n+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+\n', 84, 111, 'ordinal not in range(128)')
    Traceback (most recent call last):
    File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 2059, in run_profiler
    exit_status = profiler.profile_code(
    File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1861, in profile_code
    did_output = Scalene.output_profile()
    File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 971, in output_profile
    did_output: bool = output.output_profiles(
    File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/scalene/scalene_output.py", line 637, in output_profiles
    console.print(tbl)
    File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 1720, in print
    self._buffer.extend(new_segments)
    File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 864, in exit
    self._exit_buffer()
    File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 822, in _exit_buffer
    self._check_buffer()
    File "/root/.pyenv/versions/3.9.16/lib/python3.9/site-packages/rich/console.py", line 2060, in _check_buffer
    self.file.write(text)
    UnicodeEncodeError: 'ascii' codec can't encode characters in position 84-110: ordinal not in range(128)
    *** You may need to add PYTHONIOENCODING=utf-8 to your environment ***

    Expected behavior
    Scalene successfully profiles the script and outputs the nearly identical table to the one obtained successfully in step 5.

    Desktop (please complete the following information):

    • OS: ubuntu 20.04 (docker image on a windows host)
    • GPU: GTX 3090
    • Browser: CLI
    • Version: 1.5.21.4

    Additional context
    I originally went down the rabbit hole of trying to fix the actual ut8 / ASCII codec error, but none of the fixes obtained from googling that except rich text raises worked for me. In particular:

    • I set the environment variable PYTHONIOENCODING to utf-8
    • I tried sys.stdin.reconfigure(encoding='utf-8') and sys.stdout.reconfigure(encoding='utf-8')
    • I set LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
    • I even tried setting the windows powershell terminal to utf-8 using language settings -> administrative language settings -> change system locale -> Beta: Use Unicode UTF-8 for worldwide language support. (since it's a Ubuntu docker container outputting text to a Windows powershell terminal)

    So I think the actual unicode exception is either a red herring or maybe something in cupy resets the python's expectation to use the utf-8 codec?

    Pip list
    Package Version


    absl-py 1.4.0
    aiodns 3.0.0
    aiohttp 3.8.4
    aiosignal 1.3.1
    alembic 1.11.1
    amqp 5.1.1
    aniso8601 9.0.1
    asttokens 2.2.1
    astunparse 1.6.3
    async-timeout 4.0.2
    asyncio 3.4.3
    attrs 23.1.0
    backcall 0.2.0
    beautifulsoup4 4.12.2
    billiard 4.1.0
    blinker 1.4
    blis 0.7.9
    Brotli 1.0.9
    cachetools 5.3.1
    catalogue 2.0.8
    cchardet 2.1.7
    celery 5.3.1
    certifi 2023.5.7
    cffi 1.15.1
    charset-normalizer 3.2.0
    cli-formatter 1.2.0
    click 8.1.5
    click-didyoumean 0.3.0
    click-plugins 1.1.1
    click-repl 0.3.0
    cloudpickle 2.2.1
    cmake 3.26.4
    comm 0.1.3
    confection 0.1.0
    contourpy 1.1.0
    cryptography 41.0.2
    cupy-cuda11x 11.6.0
    cycler 0.11.0
    cymem 2.0.7
    debugpy 1.6.7
    decorator 5.1.1
    dnspython 2.3.0
    email-validator 1.1.3
    en-core-web-sm 3.5.0
    en-core-web-trf 3.5.0
    et-xmlfile 1.1.0
    executing 1.2.0
    fastrlock 0.8.1
    filelock 3.12.2
    Flask 2.0.2
    Flask-Dance 5.1.0
    Flask-Login 0.5.0
    Flask-Migrate 3.1.0
    Flask-Minify 0.37
    flask-restx 0.5.1
    Flask-SQLAlchemy 2.5.1
    Flask-WTF 1.0.0
    flatbuffers 23.5.26
    fonttools 4.41.0
    frozenlist 1.4.0
    fsspec 2023.6.0
    gast 0.4.0
    google-auth 2.22.0
    google-auth-oauthlib 0.4.6
    google-pasta 0.2.0
    greenlet 2.0.2
    grpcio 1.56.0
    gunicorn 20.1.0
    h5py 3.9.0
    htmlmin 0.1.12
    huggingface-hub 0.16.4
    idna 3.4
    importlib-metadata 6.8.0
    importlib-resources 6.0.0
    inflate64 0.3.1
    ipykernel 6.24.0
    ipython 8.14.0
    ipywidgets 8.0.7
    itsdangerous 2.1.2
    jedi 0.18.2
    Jinja2 3.0.3
    joblib 1.3.1
    jsmin 3.0.1
    jsonschema 4.18.2
    jsonschema-specifications 2023.6.1
    jupyter_client 8.3.0
    jupyter_core 5.3.1
    jupyterlab-widgets 3.0.8
    keras 2.10.0
    Keras-Preprocessing 1.1.2
    kiwisolver 1.4.4
    kombu 5.3.1
    langcodes 3.3.0
    lesscpy 0.15.1
    libclang 16.0.0
    lit 16.0.6
    llvmlite 0.40.1
    lxml 4.9.3
    Mako 1.2.4
    Markdown 3.4.3
    markdown-it-py 3.0.0
    MarkupSafe 2.1.3
    matplotlib 3.7.2
    matplotlib-inline 0.1.6
    mdurl 0.1.2
    minio 7.1.15
    mpmath 1.3.0
    multidict 6.0.4
    multivolumefile 0.2.3
    murmurhash 1.0.9
    nest-asyncio 1.5.6
    networkx 3.1
    nltk 3.8.1
    numba 0.57.1
    numpy 1.24.4
    nvidia-ml-py3 7.352.0
    oauthlib 3.2.2
    openpyxl 3.1.2
    opt-einsum 3.3.0
    packaging 23.1
    pandas 2.0.3
    parso 0.8.3
    pathy 0.10.2
    pdfminer.six 20221105
    pexpect 4.8.0
    pickleshare 0.7.5
    Pillow 10.0.0
    pip 23.1.2
    platformdirs 3.8.1
    ply 3.11
    preshed 3.0.8
    prompt-toolkit 3.0.39
    protobuf 3.19.6
    psutil 5.9.5
    ptyprocess 0.7.0
    pure-eval 0.2.2
    py7zr 0.20.5
    pyasn1 0.5.0
    pyasn1-modules 0.3.0
    pybcj 1.0.1
    pycares 4.3.0
    pycparser 2.21
    pycryptodomex 3.18.0
    pydantic 1.10.11
    Pygments 2.15.1
    pynvml 11.4.1
    pyOpenSSL 23.2.0
    pyparsing 3.0.9
    PyPDF2 3.0.1
    pyppmd 1.0.0
    python-dateutil 2.8.2
    python-docx 0.8.11
    python-dotenv 0.19.2
    pytz 2023.3
    PyYAML 6.0
    pyzmq 25.1.0
    pyzstd 0.15.9
    rapidfuzz 3.1.1
    rcssmin 1.1.1
    referencing 0.29.1
    regex 2023.6.3
    requests 2.31.0
    requests-file 1.5.1
    requests-oauthlib 1.3.1
    rich 13.4.2
    rpds-py 0.8.10
    rsa 4.9
    safetensors 0.3.1
    scalene 1.5.21.4
    scikit-learn 1.3.0
    scipy 1.11.1
    sense2vec 2.0.2
    setuptools 58.1.0
    six 1.16.0
    smart-open 6.3.0
    snakeviz 2.2.0
    soupsieve 2.4.1
    spacy 3.5.0
    spacy-alignments 0.9.0
    spacy-legacy 3.0.12
    spacy-loggers 1.0.4
    spacy-transformers 1.2.5
    SQLAlchemy 1.4.29
    srsly 2.4.6
    stack-data 0.6.2
    striprtf 0.0.25
    sympy 1.12
    tensorboard 2.10.1
    tensorboard-data-server 0.6.1
    tensorboard-plugin-wit 1.8.1
    tensorflow 2.10.1
    tensorflow-estimator 2.10.0
    tensorflow-io-gcs-filesystem 0.32.0
    termcolor 2.3.0
    texttable 1.6.7
    thinc 8.1.10
    threadpoolctl 3.2.0
    tldextract 3.4.4
    tokenizers 0.13.3
    torch 2.0.1+cu118
    tornado 6.3.2
    tqdm 4.65.0
    traitlets 5.9.0
    transformers 4.30.2
    triton 2.0.0
    typer 0.7.0
    typing_extensions 4.7.1
    tzdata 2023.3
    urllib3 1.26.16
    URLObject 2.4.3
    uuid 1.30
    validators 0.20.0
    vine 5.0.0
    wasabi 1.1.2
    wcwidth 0.2.6
    Werkzeug 2.0.3
    wheel 0.40.0
    widgetsnbextension 4.0.8
    wrapt 1.15.0
    WTForms 3.0.1
    xxhash 3.2.0
    yarl 1.9.2
    zipp 3.16.1

  6. emeryberger commented on Jul 15, 2023

    @emeryberger
    Member

    Thanks @Louis-Paul-Bowman for this detailed report! I was able to find the issue, which indeed is CuPy related (actually NVIDIA related): NVIDIA/cuda-python#29.

  7. emeryberger commented on Jul 15, 2023

    @emeryberger
    Member

    Please test by running the current repo version (I will release a new version once confirmed): python3 -m pip install -U git+https://github.com/plasma-umass/scalene)

  8. emeryberger commented on Jul 15, 2023

    @emeryberger
    Member

    @gongyguo I have successfully run your repro case with the patched version. Please test. Thanks!

  9. Louis-Paul-Bowman commented on Jul 15, 2023

    @Louis-Paul-Bowman

    Please test by running the current repo version (I will release a new version once confirmed): python3 -m pip install -U git+https://github.com/plasma-umass/scalene)

    Confirmed, fixed on my end when pulling the repo version.
    Nice find! That is... cryptic behavior to say the least.

  10. mwdotzom commented on Jul 17, 2023

    @mwdotzom

    Thanks @Louis-Paul-Bowman for this detailed report! I was able to find the issue, which indeed is CuPy related (actually NVIDIA related): NVIDIA/cuda-python#29.

    I can confirm that the workaround export LC_ALL="POSIX" fixed my problem highly similar to this issue, millions of thanks!

  11. emeryberger commented on Jul 17, 2023

    @emeryberger
    Member

    New version released, thanks again for everyone's help in tracking this one down!

  12. added a commit that references this issue on Apr 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions