Skip to content

Commit e250ec0

Browse files
authored
Includes: support git includes (#51191)
This PR allows the use of git options, like those supported for `repos.yaml`, in `include.yaml`. This has been tested manually using configuration files from `https://github.com/spack/spack-configs/tree/main/USC/config`. Example (from the updated docs): ``` include: - git: https://github.com/spack/spack-configs branch: main when: os == "centos7" paths: - USC/config/config.yaml - USC/config/packages.yaml ``` - [x] Add unit tests (at least of `git`) options (scopes) --------- Signed-off-by: tldahlgren <[email protected]>
1 parent 52954c2 commit e250ec0

File tree

5 files changed

+616
-75
lines changed

5 files changed

+616
-75
lines changed

lib/spack/docs/include_yaml.rst

Lines changed: 49 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -14,9 +14,25 @@ Include Settings (include.yaml)
1414

1515
Spack allows you to include configuration files through ``include.yaml``.
1616
Using the ``include:`` heading results in pulling in external configuration information to be used by any Spack command.
17+
Paths to configuration files may reside on the local file system, be a URL to a remote file, or be paths associated with a ``git`` repository.
1718

18-
Included configuration files are required *unless* they are explicitly optional or the entry's condition evaluates to ``false``.
19+
Included configuration files are required *unless* they are explicitly optional or the entry's condition evaluates to ``False``.
1920
Optional includes are specified with the ``optional`` clause and conditional ones with the ``when`` clause.
21+
22+
.. hint::
23+
24+
The same conditions and variables in :ref:`Spec List References <spec-list-references>` can be used for conditional activation in the ``when`` clauses.
25+
26+
.. warning::
27+
28+
Recursive includes are not currently processed in a breadth-first manner, so the value of a configuration option that is altered by multiple included files may not be what you expect.
29+
This will be addressed in a future update.
30+
31+
Local paths
32+
~~~~~~~~~~~
33+
34+
Local configuration files can be specified by path or by their parent directory.
35+
Paths may be absolute, relative (to the configuration file including the path), or specified as URLs.
2036
For example,
2137

2238
.. code-block:: yaml
@@ -28,30 +44,51 @@ For example,
2844
- path: /path/to/os-specific/config-dir
2945
when: os == "ventura"
3046
31-
shows all three.
32-
The first entry, ``/path/to/a/required/config.yaml``, indicates that the included ``config.yaml`` file is required (so must exist).
47+
illustrates required, optional, and conditional includes, respectively.
48+
The first entry only provides a local path, ``/path/to/a/required/config.yaml``, meaning that the file is required (so must exist).
3349
Use of ``optional: true`` for ``/path/to/$os/$target/config`` means the path is only included if it exists.
3450
The condition ``os == "ventura"`` in the ``when`` clause for ``/path/to/os-specific/config-dir`` means the path is only included when the operating system (``os``) is ``ventura``.
3551

36-
The same conditions and variables in :ref:`Spec List References <spec-list-references>` can be used for conditional activation in the ``when`` clauses.
52+
Remote file URLs
53+
~~~~~~~~~~~~~~~~
3754

38-
Included files can be specified by path or by their parent directory.
39-
Paths may be absolute, relative to the including configuration file, or specified as URLs.
40-
Only the ``file``, ``ftp``, ``http``, and ``https`` schemes are supported.
55+
Only the ``file``, ``ftp``, ``http``, and ``https`` protocols (or schemes) are supported for remote file URLs.
4156
Spack-specific, environment, and user path variables can be used.
4257
(See :ref:`config-file-variables` for more information.)
4358

44-
A ``sha256`` is required for remote file URLs and must be specified as follows:
59+
A ``sha256`` is required and must be specified as follows:
4560

4661
.. code-block:: yaml
4762
4863
include:
49-
- path: https://github.com/path/to/raw/config/compilers.yaml
64+
- path: https://github.com/path/to/raw/config/config.yaml
5065
sha256: 26e871804a92cd07bb3d611b31b4156ae93d35b6a6d6e0ef3a67871fcb1d258b
5166
52-
Additionally, remote file URLs must link to the **raw** form of the file's contents (e.g., `GitHub <https://docs.github.com/en/repositories/working-with-files/using-files/viewing-and-understanding-files#viewing-or-copying-the-raw-file-content>`_ or `GitLab <https://docs.gitlab.com/ee/api/repository_files.html#get-raw-file-from-repository>`_).
67+
The ``config.yaml`` file would be cached locally to a special include location and its contents included in Spack's configuration.
5368

5469
.. warning::
5570

56-
Recursive includes are not currently processed in a breadth-first manner, so the value of a configuration option that is altered by multiple included files may not be what you expect.
57-
This will be addressed in a future update.
71+
Remote file URLs must link to the **raw** form of the file's contents (e.g., `GitHub <https://docs.github.com/en/repositories/working-with-files/using-files/viewing-and-understanding-files#viewing-or-copying-the-raw-file-content>`_ or `GitLab <https://docs.gitlab.com/ee/api/repository_files.html#get-raw-file-from-repository>`_).
72+
73+
``git`` repository files
74+
~~~~~~~~~~~~~~~~~~~~~~~~
75+
76+
You can also include configuration files from a ``git`` repository.
77+
The `branch`, `commit`, or `tag` to be checked out is required.
78+
A list of relative paths in which to find the configuration files is also required.
79+
Inclusion of the repository (and its paths) can be optional or conditional.
80+
81+
For example, suppose we only want to include the ``config.yaml`` and ``packages.yaml`` files from the `spack/spack-configs <https://github.com/spack/spack-configs>`_ repository's ``USC/config`` directory when using the ``centos7`` operating system.
82+
We would then configure the ``include.yaml`` file as follows:
83+
84+
.. code-block:: yaml
85+
86+
include:
87+
- git: https://github.com/spack/spack-configs
88+
branch: main
89+
when: os == "centos7"
90+
paths:
91+
- USC/config/config.yaml
92+
- USC/config/packages.yaml
93+
94+
If the condition is satisfied, then the ``main`` branch of the repository will be cloned and the settings for the two files integrated into Spack's configuration.

0 commit comments

Comments
 (0)