Skip to content

Commit 4169ac7

Browse files
jjjermiahjjjermiah
committed
docs: Enhance contributing guide with Pixi setup and testing instructions
1 parent 6a0de2d commit 4169ac7

File tree

1 file changed

+112
-65
lines changed

1 file changed

+112
-65
lines changed

‎docs/project_info/contributing.rst‎

Lines changed: 112 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,43 @@ If you want to create more pull requests, first run :code:`git checkout main` an
8585

8686
Feel free to ask questions about this if you want to contribute to Snakemake :)
8787

88+
.. _pixi-getting_started:
89+
90+
Development with ``pixi``
91+
=======================
92+
93+
`pixi <https://pixi.sh/>`_ is a tool that is designed to help you manage
94+
your development environment.It acts as a drop-in replacement for
95+
`conda <https://docs.conda.io/en/latest/>`_, offering:
96+
97+
- **Easy installation & Updating**: `install pixi <https://pixi.sh/latest/#installation>`_
98+
through many methods and for different shells.
99+
Updating ``pixi`` is as simple as ``pixi self-update``
100+
101+
- **Ease of Use**: A streamlined CLI (similar to Yarn or Cargo) for quick
102+
environment creation and management. Try commands like ``pixi init``,
103+
``pixi add <package>``, or ``pixi run`` to see how intuitive it is.
104+
105+
- **Multiple Environments**: Define and switch between multiple sets of
106+
dependencies under one project.
107+
Pixi uses a ``feature`` system to compose an ``environment``.
108+
Think of ``features`` as a way to group dependencies and settings together.
109+
and an environment is a collection of features.
110+
This allows easy management of different environments for multiple use.
111+
See the ``pyproject.toml`` file for an example of how the ``test`` feature
112+
is used to define the ``dev``, ``py311`` and ``py312`` environments.
113+
114+
- **Cross-Platform Solving**: Target Linux, macOS, and Windows from a single
115+
config. Pixi resolves the correct packages for each platform and captures
116+
them in a lockfile for reproducible setups—no Docker needed.
117+
118+
- **Speed & Conda Compatibility**: Written in Rust, Pixi downloads and solves
119+
packages in parallel for faster operations. It uses the Conda ecosystem
120+
and channels, so you get the same packages with improved performance. In
121+
many cases, Pixi can outperform both Conda and Mamba.
122+
123+
To learn more, visit the `Pixi docs <https://pixi.sh>`__ or check out helpful
124+
guides on `prefix.dev <https://prefix.dev/>`__.
88125

89126
Testing Guidelines
90127
==================
@@ -103,57 +140,74 @@ Setup to run the test suite locally
103140

104141
Unit tests and regression tests are written to be run by `pytest <https://docs.pytest.org/en/stable/>`_.
105142

106-
Assuming you are on a Linux system, and have a working Conda installation, the standard way to run the tests is explained in the following.
107-
With sufficient experience, it is however possible to run the tests in different setups, given the expected dependencies are installed.
108143

109-
1. Ensure your Conda version is at least 24.7.1 (Snakemake's minimum required Conda version).
110-
Check the output of ``conda --version`` and update conda if necessary.
111-
2. Activate strict channel priorities (this is always a good idea when using Conda, see `here <https://conda-forge.org/docs/user/tipsandtricks/#using-multiple-channels>`__ for the rationale), by running ``conda config --set channel_priority strict``.
112-
3. After checking out the branch you want to test, run these commands:
144+
.. _pixi-test-guide:
145+
146+
Testing Guide using ``pixi``
147+
=========================
148+
149+
**Prerequisites**: Make sure you have ``pixi`` installed: See :ref:`pixi-getting_started`.
150+
151+
**Activate your environment**:
152+
--------------------------------
153+
154+
There are a few environments you can use to run the tests.
155+
156+
The ``dev`` environment is most useful for overall development.
157+
This environment will also install the ``docs`` and ``style`` features
158+
which will allow you to also build documentation and run ``black``.
113159

114-
.. code-block:: console
160+
.. code-block:: console
161+
162+
$ pixi shell -e dev
115163
116-
$ conda env create -f test-environment.yml -n snakemake-testing
117-
$ conda activate snakemake-testing
118-
$ pip install -e .
164+
The ``py311`` and ``py312`` environments are what are used in the
165+
CI tests which isolate the Python version and the `test` dependencies.
166+
Use this if you want to test your code against the same environment
167+
as any failing CI tests.
168+
169+
.. code-block:: console
119170
120-
You may want to set a specific Python version by editing the constraint in ``test-environment.yml`` before doing this.
121-
Use of the ``-e``/``--editable`` option to ``pip`` will make your development version of Snakemake the one called when running Snakemake and all the unit tests. You only need to run the ``pip`` command once, not after each time you make code changes.
171+
$ pixi shell -e py311
122172
123-
4. From the base Snakemake folder you may now run any specific test:
173+
**Run a comprehensive, simple, or single test**:
174+
The test suite defines two types of tests via ``pixi tasks`` that you can run:
175+
176+
**test-all**: This task runs the comprehensive test suite, which includes
177+
*most* of the tests in the ``tests/`` directory.
178+
179+
.. code-block:: console
124180
125-
.. code-block:: console
181+
$ pixi run test-all
126182
127-
$ pytest tests/tests.py::test_log_input
183+
**test-simple**: This task runs the main tests located in ``tests/tests.py``.
128184

129-
You can also use the ``-k`` flag to select tests by substring match, rather than by the full name, and the ``--co`` option to preview which tests will be run. Try, for example:
185+
.. code-block:: console
130186
131-
.. code-block:: console
187+
$ pixi run test-simple
132188
133-
$ pytest --co tests/tests.py -k test_modules_all
189+
**Single test**: You can also run a single test by using ``pytest``
190+
directly with the test file and the test name.
134191

135-
Running the full test suite
136-
---------------------------
192+
.. code-block:: console
137193
138-
If you simply run ``pytest`` in the top level directory it will scan for and attempt to run every test in the directory, but you will almost certainly get errors as not all tests are working and current.
194+
$ pixi run pytest tests/tests.py::test_log_input
139195
140-
The core test suite is run as a `GitHub action <https://docs.github.com/en/actions/about-github-actions/understanding-github-actions>`_ by the code under ``.github/workflows/main.yml``, so you should look in this file for the list of tests actually expected to pass in a regular test environment.
196+
.. tip::
197+
This test suite is quite long, and can be run in parts similar to the
198+
CI/CD tests which run it in 1/10 parts.
141199

142-
At the time of writing this text, the suite is:
200+
To do so, you can use the ``--splits`` and ``--group`` flags to run
201+
a subset of the tests. For example, to run the first group of tests
202+
in a 10 part split:
143203

144-
.. code-block::
204+
.. code-block:: console
145205
146-
tests/tests.py
147-
tests/tests_using_conda.py
148-
tests/test_expand.py
149-
tests/test_io.py
150-
tests/test_schema.py
151-
tests/test_linting.py
152-
tests/test_executor_test_suite.py
153-
tests/test_api.py
154-
tests/test_internals.py
206+
$ pixi run test-simple \
207+
--splits 10 \
208+
--group 1 \
209+
--splitting-algorithm=least_duration
155210
156-
Other tests in the directory may or may not work.
157211
158212
Warnings and oddities
159213
---------------------
@@ -168,9 +222,6 @@ This is likely due to system security profiles that conda, being a non-root appl
168222
The Debian/Ubuntu ``singularity-container`` DEB package, which must be installed by the system administrator, does work.
169223
The equivalent RPM package should also work on RedHat-type systems.
170224

171-
Tests in ``tests/test_api.py`` require a working ``git``.
172-
This is not included in ``test-environment.yml`` as it's assumed you must have GIT installed to be working on the source code, but installing git into the conda environment should work if need be.
173-
174225
Depending on how the Snakemake code was downloaded and installed in the test environment, Snakemake may not be able to determine its own version and may think that it is version 0.
175226
The existing unit tests should all cope with this, and in general you should avoid writing tests that rely on explicit version checks.
176227

@@ -217,46 +268,42 @@ For the documentation, please adhere to the following guidelines:
217268
Documentation Setup
218269
-------------------
219270

220-
For building the documentation, you have to install the Sphinx.
221-
If you have already installed Conda, all you need to do is to create a
222-
Snakemake development environment via
271+
The documentation is built using `Sphinx <https://www.sphinx-doc.org>`_.
223272

224-
.. code-block:: console
225-
226-
$ git clone [email protected]:snakemake/snakemake.git
227-
$ cd snakemake
228-
$ conda env create -f doc-environment.yml -n snakemake_docs
229-
$ conda activate snakemake_docs
230-
231-
You will also need to install your development version of Snakemake for the docs to be built correctly
273+
To get started, make sure you have ``pixi`` installed:
274+
See :ref:`pixi-getting_started`.
275+
We use ``pixi`` to manage the docs environment and tasks to streamline
276+
the developer experience.
232277

233278
.. code-block:: console
234279
235-
$ pip install -e .
280+
$ ➜ pixi task list --environment docs
236281
237-
Then, the docs can be built with
282+
Tasks that can run on this machine:
283+
-----------------------------------
284+
build-docs, docs
238285
239-
.. code-block:: console
286+
- build-docs Build the documentation in the docs/ directory
287+
- docs Serve the documentation on http://localhost:8000 with live reload
240288
241-
$ cd docs
242-
$ make html
243-
$ make clean && make html # force rebuild
244289
245-
Alternatively, you can use virtualenv.
246-
The following assumes you have a working Python 3 setup.
290+
**Test if the docs build**:
291+
To only build the documentation, you can use the ``build-docs`` task.
247292

248293
.. code-block:: console
249294
250-
$ git clone [email protected]:snakemake/snakemake.git
251-
$ cd snakemake/docs
252-
$ virtualenv -p python3 .venv
253-
$ source .venv/bin/activate
254-
$ pip install --upgrade -r requirements.txt
295+
$ pixi run build-docs
255296
256-
Afterwards, the docs can be built with
297+
**Live server with auto-reload**:
298+
To serve the documentation on a local server with live reload,
299+
use the ``docs`` task.
257300

258301
.. code-block:: console
259302
260-
$ source .venv/bin/activate
261-
$ make html # rebuild for changed files only
262-
$ make clean && make html # force rebuild
303+
$ pixi run docs
304+
[sphinx-autobuild] Starting initial build
305+
[sphinx-autobuild] > python -m sphinx build docs/ docs/_build/html
306+
...
307+
The HTML pages are in docs/_build/html.
308+
[sphinx-autobuild] Serving on http://0.0.0.0:8000
309+
[sphinx-autobuild] Waiting to detect changes...

0 commit comments

Comments
 (0)