Modifying tox_root does not propagate to dependent fields. #2933
Description
Issue
Changing the value of tox_root does not propagate to fields whose default value also depends on tox_root (i.e work_dir, temp_dir, env_dir, etc...).
This reproducible in tox 4.4.6 and in the latest commit (which is 177a227 at time of writing) when trying to modify tox_root using the CLI interface (e.g tox --root ~).
I assume this would also be the case when using a plugin to modify tox_root, but the following (using an approach based on this discussion) doesn't even change the value of tox_root, so I assume that tox_root might be getting cached before this hook can run:
@impl
def tox_add_core_config(core_conf: CoreConfigSet, state: State):
core_conf.loaders.insert(0, MemoryLoader(tox_root=Path.home())Environment
Provide at least:
- OS:
Distributor ID: Ubuntu
Description: Ubuntu 22.04.1 LTS
Release: 22.04
Codename: jammypip listof the host Python wheretoxis installed:
Package Version
--------------------- -------
cachetools 5.3.0
chardet 5.1.0
colorama 0.4.6
distlib 0.3.6
docstring-to-markdown 0.11
filelock 3.9.0
jedi 0.18.2
packaging 23.0
parso 0.8.3
pip 22.0.2
platformdirs 3.0.0
pluggy 1.0.0
pylsp-rope 0.1.11
pyproject_api 1.5.0
python-lsp-jsonrpc 1.0.0
python-lsp-server 1.7.1
pytoolconfig 1.2.5
rope 1.7.0
setuptools 59.6.0
tomli 2.0.1
tox 4.4.6
ujson 5.7.0
virtualenv 20.20.0
Output of running tox
Provide the output of tox -rvv:
py: 603 W remove tox env folder /home/kd/Documents/tox/repro/.tox/py [tox/tox_env/api.py:322]
py: 683 I find interpreter for spec PythonSpec(path=/home/kd/Documents/tox/venv/bin/python3) [virtualenv/discovery/builtin.py:56]
py: 683 D discover exe for PythonInfo(spec=CPython3.10.6.final.0-64, exe=/home/kd/Documents/tox/venv/bin/python3, platform=linux, version='3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) in /usr [virtualenv/discovery/py_info.py:437]
py: 685 D filesystem is case-sensitive [virtualenv/info.py:24]
py: 687 D got python info of /usr/bin/python3.10 from /home/kd/.local/share/virtualenv/py_info/1/8a94588eda9d64d9e9a351ab8144e55b1fabf5113b54e67dd26a8c27df0381b3.json [virtualenv/app_data/via_disk_folder.py:129]
py: 688 I proposed PythonInfo(spec=CPython3.10.6.final.0-64, system=/usr/bin/python3.10, exe=/home/kd/Documents/tox/venv/bin/python3, platform=linux, version='3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:63]
py: 689 D accepted PythonInfo(spec=CPython3.10.6.final.0-64, system=/usr/bin/python3.10, exe=/home/kd/Documents/tox/venv/bin/python3, platform=linux, version='3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:65]
py: 762 I create virtual environment via CPython3Posix(dest=/home/kd/Documents/tox/repro/.tox/py, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:48]
py: 763 D create folder /home/kd/Documents/tox/repro/.tox/py/bin [virtualenv/util/path/_sync.py:9]
py: 764 D create folder /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages [virtualenv/util/path/_sync.py:9]
py: 766 D write /home/kd/Documents/tox/repro/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
py: 767 D home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
py: 767 D implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
py: 767 D version_info = 3.10.6.final.0 [virtualenv/create/pyenv_cfg.py:34]
py: 768 D virtualenv = 20.20.0 [virtualenv/create/pyenv_cfg.py:34]
py: 768 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
py: 768 D base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
py: 768 D base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
py: 768 D base-executable = /usr/bin/python3.10 [virtualenv/create/pyenv_cfg.py:34]
py: 769 D symlink /usr/bin/python3.10 to /home/kd/Documents/tox/repro/.tox/py/bin/python [virtualenv/util/path/_sync.py:28]
py: 770 D create virtualenv import hook file /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:89]
py: 771 D create /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:92]
py: 772 D ============================== target debug ============================== [virtualenv/run/session.py:50]
py: 772 D debug via /home/kd/Documents/tox/repro/.tox/py/bin/python /home/kd/Documents/tox/venv/lib/python3.10/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:193]
py: 772 D {
"sys": {
"executable": "/home/kd/Documents/tox/repro/.tox/py/bin/python",
"_base_executable": "/home/kd/Documents/tox/repro/.tox/py/bin/python",
"prefix": "/home/kd/Documents/tox/repro/.tox/py",
"base_prefix": "/usr",
"real_prefix": null,
"exec_prefix": "/home/kd/Documents/tox/repro/.tox/py",
"base_exec_prefix": "/usr",
"path": [
"/usr/lib/python310.zip",
"/usr/lib/python3.10",
"/usr/lib/python3.10/lib-dynload",
"/home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages"
],
"meta_path": [
"<class '_virtualenv._Finder'>",
"<class '_frozen_importlib.BuiltinImporter'>",
"<class '_frozen_importlib.FrozenImporter'>",
"<class '_frozen_importlib_external.PathFinder'>"
],
"fs_encoding": "utf-8",
"io_encoding": "utf-8"
},
"version": "3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]",
"makefile_filename": "/usr/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile",
"os": "<module 'os' from '/usr/lib/python3.10/os.py'>",
"site": "<module 'site' from '/usr/lib/python3.10/site.py'>",
"datetime": "<module 'datetime' from '/usr/lib/python3.10/datetime.py'>",
"math": "<module 'math' (built-in)>",
"json": "<module 'json' from '/usr/lib/python3.10/json/__init__.py'>"
} [virtualenv/run/session.py:51]
py: 847 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/kd/.local/share/virtualenv) [virtualenv/run/session.py:55]
py: 863 D got embed update of distribution wheel from /home/kd/.local/share/virtualenv/wheel/3.10/embed/3/wheel.json [virtualenv/app_data/via_disk_folder.py:129]
py: 865 D got embed update of distribution pip from /home/kd/.local/share/virtualenv/wheel/3.10/embed/3/pip.json [virtualenv/app_data/via_disk_folder.py:129]
py: 876 D got embed update of distribution setuptools from /home/kd/.local/share/virtualenv/wheel/3.10/embed/3/setuptools.json [virtualenv/app_data/via_disk_folder.py:129]
py: 878 D install pip from wheel /home/kd/Documents/tox/venv/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/pip-23.0.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py: 880 D install wheel from wheel /home/kd/Documents/tox/venv/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/wheel-0.38.4-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py: 884 D install setuptools from wheel /home/kd/Documents/tox/venv/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/setuptools-67.4.0-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py: 891 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/pip [virtualenv/util/path/_sync.py:36]
py: 903 D copy /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/setuptools-67.4.0.virtualenv to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/setuptools-67.4.0.virtualenv [virtualenv/util/path/_sync.py:36]
py: 904 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/wheel [virtualenv/util/path/_sync.py:36]
py: 909 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/_distutils_hack to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:36]
py: 927 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/setuptools-67.4.0.dist-info to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/setuptools-67.4.0.dist-info [virtualenv/util/path/_sync.py:36]
py: 966 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/setuptools to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/setuptools [virtualenv/util/path/_sync.py:36]
py: 994 D copy /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.virtualenv to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/wheel-0.38.4.virtualenv [virtualenv/util/path/_sync.py:36]
py: 998 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.dist-info to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/wheel-0.38.4.dist-info [virtualenv/util/path/_sync.py:36]
py: 1047 D generated console scripts wheel3 wheel3.10 wheel-3.10 wheel [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py: 1360 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/pkg_resources to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/pkg_resources [virtualenv/util/path/_sync.py:36]
py: 1480 D copy /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.4.0-py3-none-any/distutils-precedence.pth to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:36]
py: 1484 D generated console scripts [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py: 1656 D copy directory /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip-23.0.1.dist-info to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/pip-23.0.1.dist-info [virtualenv/util/path/_sync.py:36]
py: 1670 D copy /home/kd/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip-23.0.1.virtualenv to /home/kd/Documents/tox/repro/.tox/py/lib/python3.10/site-packages/pip-23.0.1.virtualenv [virtualenv/util/path/_sync.py:36]
py: 1674 D generated console scripts pip3 pip-3.10 pip3.10 pip [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py: 1675 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:61]
py: 1682 D write /home/kd/Documents/tox/repro/.tox/py/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
py: 1682 D home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
py: 1683 D implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
py: 1683 D version_info = 3.10.6.final.0 [virtualenv/create/pyenv_cfg.py:34]
py: 1683 D virtualenv = 20.20.0 [virtualenv/create/pyenv_cfg.py:34]
py: 1684 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
py: 1684 D base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
py: 1684 D base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
py: 1685 D base-executable = /usr/bin/python3.10 [virtualenv/create/pyenv_cfg.py:34]
py: 1702 W commands[0]> python -c 'print('"'"'\n'"'"'.join(['"'"'tox_root:\t/home/kd/Documents/tox/repro'"'"', '"'"'work_dir:\t/home/kd/Documents/tox/repro/.tox'"'"', '"'"'temp_dir:\t/home/kd/Documents/tox/repro/.tox/.tmp'"'"', '"'"'env_dir:\t/home/kd/Documents/tox/repro/.tox/py'"'"', '"'"'env_tmp_dir:\t/home/kd/Documents/tox/repro/.tox/py/tmp'"'"', '"'"'env_log_dir:\t/home/kd/Documents/tox/repro/.tox/py/log'"'"']))' [tox/tox_env/api.py:428]
tox_root: /home/kd/Documents/tox/repro
work_dir: /home/kd/Documents/tox/repro/.tox
temp_dir: /home/kd/Documents/tox/repro/.tox/.tmp
env_dir: /home/kd/Documents/tox/repro/.tox/py
env_tmp_dir: /home/kd/Documents/tox/repro/.tox/py/tmp
env_log_dir: /home/kd/Documents/tox/repro/.tox/py/log
py: 1809 I exit 0 (0.10 seconds) /home/kd/Documents/tox/repro> python -c 'print('"'"'\n'"'"'.join(['"'"'tox_root:\t/home/kd/Documents/tox/repro'"'"', '"'"'work_dir:\t/home/kd/Documents/tox/repro/.tox'"'"', '"'"'temp_dir:\t/home/kd/Documents/tox/repro/.tox/.tmp'"'"', '"'"'env_dir:\t/home/kd/Documents/tox/repro/.tox/py'"'"', '"'"'env_tmp_dir:\t/home/kd/Documents/tox/repro/.tox/py/tmp'"'"', '"'"'env_log_dir:\t/home/kd/Documents/tox/repro/.tox/py/log'"'"']))' pid=3033 [tox/execute/api.py:275]
py: OK (1.21=setup[1.10]+cmd[0.10] seconds)
congratulations :) (1.41 seconds)
Minimal example
If possible, provide a minimal reproducer for the issue:
-
Reproduce minimal tox project
$ tree . ├── foo └── tox.ini
Where
# tox.ini [testenv] commands= python -c "print('\n'.join(\[\ 'tox_root:\t{tox_root}', \ 'work_dir:\t{work_dir}', \ 'env_dir:\t{env_dir}', \ 'env_tmp_dir:\t{env_tmp_dir}', \ 'env_log_dir:\t{env_log_dir}'\]))"
-
Run
tox -qq --root fooExpected Output:
tox_root: /home/kd/Documents/tox/repro/foo work_dir: /home/kd/Documents/tox/repro/foo/.tox temp_dir: /home/kd/Documents/tox/repro/foo/.tox/.tmp env_dir: /home/kd/Documents/tox/repro/foo/.tox/py env_tmp_dir: /home/kd/Documents/tox/repro/foo/.tox/py/tmp env_log_dir: /home/kd/Documents/tox/repro/foo/.tox/py/log
Actual Output:
tox_root: /home/kd/Documents/tox/repro/foo work_dir: /home/kd/Documents/tox/repro/.tox temp_dir: /home/kd/Documents/tox/repro/.tox/.tmp env_dir: /home/kd/Documents/tox/repro/.tox/py env_tmp_dir: /home/kd/Documents/tox/repro/.tox/py/tmp env_log_dir: /home/kd/Documents/tox/repro/.tox/py/log
Miscelleanous
-
--workdirfunctions as expected. With the same setup as above runningtox -qq --workdir foo/.toxdoes correctly propagate the changed workdir to dependent config values:tox_root: /home/kd/Documents/tox/repro work_dir: /home/kd/Documents/tox/repro/foo/.tox temp_dir: /home/kd/Documents/tox/repro/foo/.tox/.tmp env_dir: /home/kd/Documents/tox/repro/foo/.tox/py env_tmp_dir: /home/kd/Documents/tox/repro/foo/.tox/py/tmp env_log_dir: /home/kd/Documents/tox/repro/foo/.tox/py/log