Skip to content

Commit ab2609c

Browse files
committed
Merge reviewed Lychee tooling updates from PR 3118
Signed-off-by: Ralf W. Grosse-Kunstleve <[email protected]>
2 parents 8349c46 + cc6fa67 commit ab2609c

10 files changed

Lines changed: 257 additions & 153 deletions

File tree

‎.github/actions/check-doc-links/action.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ runs:
2525
shell: bash --noprofile --norc -euo pipefail {0}
2626
env:
2727
KIND: ${{ inputs.kind }}
28-
LYCHEE_VERSION: v0.24.2 # Must match the pinned rev in .pre-commit-config.yaml
28+
LYCHEE_VERSION: v0.24.2 # Must match lychee-bin in .pre-commit-config.yaml
2929
POLICY_HASH: ${{ hashFiles('lychee.toml', '.github/actions/check-doc-links/action.yml', 'ci/tools/prepare_lychee_inputs.py', 'ci/tools/retry_lychee.py') }}
3030
# The first pass and all retries use the same inputs and cache policy.
3131
LYCHEE_ARGS: >-

‎.github/workflows/pre-commit.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ jobs:
4343
env:
4444
# Full-CI docs builds and nightly check links. The installation reminder
4545
# is for developer machines; CI does not install a Git commit hook.
46+
# A separate bounded check below exercises Lychee with local fixtures.
4647
SKIP: lychee,check-precommit-installed
4748
# Minor version used by setup-python and as the cache restore-key prefix.
4849
PYTHON_VERSION: "3.14"
@@ -88,6 +89,12 @@ jobs:
8889
- name: Run pre-commit
8990
run: pre-commit run --all-files --show-diff-on-failure
9091

92+
- name: Exercise the configured Lychee hook with local fixtures
93+
env:
94+
SKIP: check-precommit-installed
95+
PYTHONUTF8: "1"
96+
run: python ci/tools/test_lychee_precommit.py
97+
9198
- name: Save pre-commit cache
9299
# Only the default branch saves, so per-PR merge refs don't store their
93100
# own duplicate copies. !cancelled() keeps a push that fails lint from

‎.pre-commit-config.yaml‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -109,13 +109,17 @@ repos:
109109
- Cython==3.2.9
110110

111111
# Link checking for authored documentation files
112-
- repo: https://github.com/lycheeverse/lychee
113-
rev: 2bba271688c1abb1503097a064e6c3bc1d1b6a9b # frozen: lychee-v0.24.2
112+
- repo: local
114113
hooks:
115114
- id: lychee
115+
name: lychee
116+
entry: python ./ci/tools/run_lychee.py
117+
language: python
118+
additional_dependencies:
119+
- lychee-bin==0.24.2 # Must match check-doc-links/action.yml
120+
types: [text]
121+
require_serial: true
116122
args:
117-
- LYCHEE_VERSION=0.24.2 # Must match the pinned rev above
118-
- --preprocess=./ci/tools/preprocess_repo_links.py
119123
- --cache
120124
- --max-concurrency=4
121125
- --max-retries=3

‎CONTRIBUTING.md‎

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Thank you for your interest in contributing to CUDA Python! Based on the type of
2525
- [Symptoms of a bad clone](#symptoms-of-a-bad-clone)
2626
- [Development on Windows](#development-on-windows)
2727
- [Enabling git symlinks](#enabling-git-symlinks)
28-
- [Pre-commit lychee workaround](#pre-commit-lychee-workaround)
28+
- [Lychee on Windows](#lychee-on-windows)
2929
- [Type stubs for cuda.core](#type-stubs-for-cudacore)
3030
- [Pre-commit](#pre-commit)
3131
- [Pixi lockfiles](#pixi-lockfiles)
@@ -184,17 +184,18 @@ Repeat the last two commands for every other tracked symlink. Re-cloning after
184184
enabling Developer Mode and setting the global option is simpler if you want
185185
all links restored.
186186

187-
### Pre-commit lychee workaround
187+
### Lychee on Windows
188188

189-
For development on Windows (not WSL), the `lychee` pre-commit task will not
190-
work when running `pre-commit run --all-files`. This problem does not occur
191-
if you install the pre-commit hook and run it automatically as part of your
192-
`git commit` workflow. To resolve this, you can either:
189+
The `lychee` pre-commit hook runs through Python and installs the native
190+
`lychee-bin==0.24.2` wheel in its own environment. It has no Git Bash
191+
dependency; use the same `pre-commit run --all-files` command from PowerShell,
192+
cmd, or Git Bash.
193193

194-
1. Run `pre-commit` in Git Bash, rather than directly in PowerShell or cmd
195-
196-
2. Skip it by setting the environment variable `SKIP` to `lychee`. This would
197-
be `$env:SKIP = "lychee"` in PowerShell or `set SKIP=lychee` in cmd.
194+
The Linux and Windows CI jobs exercise the installed hook against local
195+
fixtures, including paths containing spaces and missing targets. The full
196+
documentation link scan runs separately in documentation CI. See
197+
[Repository links during main development](ci/README.md#repository-links-during-main-development)
198+
for how repository source links are checked before merge.
198199

199200

200201
## Type stubs for cuda.core
@@ -239,7 +240,7 @@ between commits, leaving stale headers or out-of-date stubs in the history.
239240
If the hook isn't installed, `pre-commit run` will print a visible warning
240241
reminding you to run `pre-commit install`.
241242

242-
Windows contributors: see [Pre-commit lychee workaround](#pre-commit-lychee-workaround) under Development on Windows.
243+
Windows contributors: see [Lychee on Windows](#lychee-on-windows) under Development on Windows.
243244

244245
## Pixi lockfiles
245246

‎ci/README.md‎

Lines changed: 31 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -24,23 +24,43 @@ The pinned checkout action's
2424
[`README`](https://github.com/actions/checkout/blob/3d3c42e5aac5ba805825da76410c181273ba90b1/README.md#fetch-all-history-for-all-tags-and-branches)
2525
documents `fetch-depth: 0` for fetching all history, branches, and tags.
2626

27-
## Repository Links Before Merge
28-
29-
Keep links to the latest repository sources on `main`, even when their targets
30-
are added or moved by the same pull request. Lychee remaps canonical
31-
`https://github.com/NVIDIA/cuda-python/blob/main/` and `tree/main/` links
32-
for validation only; source files and published documentation keep their URLs.
33-
Links to tags, commit SHAs, other branches, and other repositories are checked
34-
as written.
27+
## Repository Links During Main Development
28+
29+
For development targeting `main`, keep links to the latest repository sources
30+
on `main`, even when their targets are added or moved by the same pull request.
31+
The link checker treats canonical
32+
`https://github.com/NVIDIA/cuda-python/blob/main/` and `tree/main/` URLs as
33+
references to the proposed main tree. Lychee remaps them for validation only;
34+
source files and published documentation keep their URLs. The remaps match only
35+
at the start of a URL, leaving URLs embedded in another URL unchanged. Relative
36+
links and links to tags, commit SHAs, other branches, and other repositories
37+
are checked as written.
3538

3639
CI remaps these links to the CI repository at the actual checked-out commit
3740
using `--remap`, retaining GitHub's file/directory views and fragment behavior.
3841
The initial check and all retries use the same remaps. Missing targets in that
3942
commit still fail.
4043

41-
Local pre-commit uses `ci/tools/preprocess_repo_links.py` through `--preprocess`
42-
to check these paths in the working tree, including uncommitted and unpushed
43-
targets. Local checks retain their existing policy of not checking fragments.
44+
Local pre-commit runs the Python hook `ci/tools/run_lychee.py`, which computes
45+
the checkout's file URI and passes anchored native `--remap` rules to Lychee.
46+
This checks paths in the working tree, including uncommitted and unpushed
47+
targets. The hook installs the official PyPI `lychee-bin==0.24.2` package in
48+
its own Python environment and uses the platform's native wheel. Local checks
49+
retain their existing policy of not checking fragments.
50+
51+
The Linux and Windows pre-commit CI jobs run a bounded smoke check of the
52+
installed hook against local Markdown and reST fixtures. They check valid
53+
files and directories and missing targets in paths containing spaces, and
54+
verify that the source files are unchanged. This smoke check does not scan all
55+
external documentation links; full authored and rendered checks run in
56+
documentation CI.
57+
58+
The `check-doc-links` action is not currently present on `12.9.x`, `13.4.x`,
59+
`12.2.x`, or `11.8.x`; merging changes into `main` does not update those
60+
branches' workflows. A maintenance branch's `/main/` URLs may intentionally
61+
refer to a different tree from its checkout. Before adopting this hook or
62+
action there, choose an explicit validation policy for those links instead of
63+
assuming that they refer to the maintenance checkout.
4464

4565
## Temporary Pixi Lockfile Maintenance Pause
4666

‎ci/tools/preprocess_repo_links.py‎

Lines changed: 0 additions & 33 deletions
This file was deleted.

‎ci/tools/run_lychee.py‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
"""Run Lychee with this checkout's main-branch source links mapped to local paths."""
5+
6+
from __future__ import annotations
7+
8+
import shutil
9+
import subprocess
10+
import sys
11+
from pathlib import Path
12+
13+
ROOT = Path(__file__).resolve().parents[2]
14+
15+
16+
def repo_link_remaps(root: Path = ROOT) -> list[str]:
17+
"""Apply the same URL-prefix boundaries as the dedicated CI link check."""
18+
local_prefix = root.resolve().as_uri() + "/"
19+
arguments = []
20+
for view in ("blob", "tree"):
21+
arguments.extend(["--remap", f"^https://github[.]com/NVIDIA/cuda-python/{view}/main/ {local_prefix}"])
22+
return arguments
23+
24+
25+
def main(arguments: list[str] | None = None) -> int:
26+
executable = shutil.which("lychee")
27+
if executable is None:
28+
raise SystemExit("lychee was not found; run this script through the configured pre-commit hook")
29+
arguments = sys.argv[1:] if arguments is None else arguments
30+
# Let Python launch the native binary on every platform, including Windows.
31+
return subprocess.run([executable, *repo_link_remaps(), *arguments], check=False).returncode # noqa: S603
32+
33+
34+
if __name__ == "__main__":
35+
raise SystemExit(main())

‎ci/tools/test_lychee_precommit.py‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
"""Exercise the configured Lychee pre-commit hook without scanning external links."""
5+
6+
from __future__ import annotations
7+
8+
import os
9+
import subprocess
10+
import sys
11+
from pathlib import Path
12+
from tempfile import TemporaryDirectory
13+
from urllib.parse import quote
14+
15+
ROOT = Path(__file__).resolve().parents[2]
16+
17+
18+
def run_hook(sources: dict[Path, bytes], missing: Path | None = None) -> None:
19+
result = subprocess.run( # noqa: S603
20+
[
21+
sys.executable,
22+
"-m",
23+
"pre_commit",
24+
"run",
25+
"lychee",
26+
"--verbose",
27+
"--color",
28+
"never",
29+
"--files",
30+
*(source.relative_to(ROOT).as_posix() for source in sources),
31+
],
32+
cwd=ROOT,
33+
env=dict(os.environ, SKIP=""),
34+
capture_output=True,
35+
encoding="utf-8",
36+
timeout=120,
37+
)
38+
output = result.stdout + result.stderr
39+
print(output, end="", flush=True)
40+
for source, original in sources.items():
41+
if source.read_bytes() != original:
42+
raise RuntimeError(f"The Lychee hook changed {source}")
43+
if "- hook id: lychee" not in output or "Skipped" in output:
44+
raise RuntimeError("The configured Lychee hook did not run")
45+
if missing is None:
46+
if result.returncode != 0:
47+
raise RuntimeError("The Lychee hook rejected existing checkout targets")
48+
elif (
49+
result.returncode == 0
50+
or "File not found" not in output
51+
or missing.as_uri() not in output
52+
or "- exit code: 2" not in output
53+
):
54+
raise RuntimeError(f"Expected Lychee to report the missing local target {missing}")
55+
56+
57+
def main() -> None:
58+
# Generate sources inside the checkout so pre-commit applies the real file filters.
59+
with TemporaryDirectory(prefix="lychee hook smoke ", dir=ROOT) as temporary:
60+
fixture = Path(temporary)
61+
directory = fixture / "new directory"
62+
directory.mkdir()
63+
target = fixture / "new source #$.py"
64+
target.write_text("value = 1\n", encoding="utf-8")
65+
relative = fixture / "relative guide.rst"
66+
relative.write_text("Relative target\n", encoding="utf-8")
67+
prefix = "https://github.com/NVIDIA/cuda-python"
68+
file_url = f"{prefix}/blob/main/{quote(target.relative_to(ROOT).as_posix())}"
69+
directory_url = f"{prefix}/tree/main/{quote(directory.relative_to(ROOT).as_posix())}/"
70+
sources = {
71+
fixture / "source guide.md": (
72+
f"[File]({file_url})\r\n[Directory]({directory_url})\r\n[Relative](relative%20guide.rst)\r\n"
73+
).encode(),
74+
fixture / "source guide.rst": (
75+
f"`File <{file_url}>`_\n`Directory <{directory_url}>`_\n`Relative <relative%20guide.rst>`_\n"
76+
).encode(),
77+
}
78+
for source, content in sources.items():
79+
source.write_bytes(content)
80+
81+
print("Checking existing local file, directory, and relative targets", flush=True)
82+
run_hook(sources)
83+
target.unlink()
84+
print("Checking that a previously valid file is rejected after deletion", flush=True)
85+
run_hook(sources, missing=target)
86+
target.write_text("value = 1\n", encoding="utf-8")
87+
directory.rmdir()
88+
print("Checking that a previously valid directory is rejected after deletion", flush=True)
89+
run_hook(sources, missing=directory)
90+
91+
print("Configured Lychee hook checks passed; documentation sources were unchanged.")
92+
93+
94+
if __name__ == "__main__":
95+
main()

0 commit comments

Comments
 (0)