Skip to content

Commit b1e2788

Browse files
authored
ci: add a maintainer override for Lychee HTTP 429 failures (#3155)
* Add maintainer label override for Lychee HTTP 429 failures Read current PR labels for authored and rendered checks. The explicit override accepts only HTTP 429 failures after retries, retaining reports and all other blocking checks. Normal policies remain unchanged. Signed-off-by: Ralf W. Grosse-Kunstleve <[email protected]> * test: remove obsolete Lychee rerun environment cleanup LYCHEE_ARGS belonged to the removed rerun path and is no longer read. Keep this cleanup separate from the maintainer HTTP 429 override, as requested in review. Signed-off-by: Ralf W. Grosse-Kunstleve <[email protected]> * ci: retry Lychee label lookups and fall back to the default policy Read current PR labels once in the shared composite action. Retry failed or malformed lookups three times with 5/10-second delays, then warn and retain the normal link policy. An unconfirmed label never enables the override. Expose the early label decision to the authored pre-commit job so other hooks still run after an ordinary labeled link failure. Nightly and default-branch calls do not query PR labels. Signed-off-by: Ralf W. Grosse-Kunstleve <[email protected]> --------- Signed-off-by: Ralf W. Grosse-Kunstleve <[email protected]>
1 parent b1c2d2f commit b1e2788

8 files changed

Lines changed: 139 additions & 19 deletions

File tree

‎.github/actions/lychee-links/action.yml‎

Lines changed: 43 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,17 +10,58 @@ inputs:
1010
description: "Which hook to run: authored or rendered"
1111
required: true
1212
policy:
13-
description: "check accepts up to 10 rate-limited URLs; cache-warm reports every unresolved link as a warning"
13+
description: >-
14+
check accepts up to 10 rate-limited URLs; cache-warm warns for any unresolved
15+
link; allow-429 accepts only HTTP 429 failures under the maintainer override
1416
default: check
1517
refresh:
1618
description: "Check links afresh and publish the successful results as the new baseline"
1719
default: "false"
20+
pr-number:
21+
description: "PR whose current maintainer override label should be read; empty skips the lookup"
22+
default: ""
23+
24+
outputs:
25+
allow-429:
26+
description: "Whether the current PR label explicitly enables the HTTP 429 override"
27+
value: ${{ steps.lychee-policy.outputs.allow-429 }}
1828

1929
# Callers provide Python and the checked-out repository; the build of any
2030
# rendered documentation must already be complete.
2131
runs:
2232
using: composite
2333
steps:
34+
- name: Read the PR's Lychee rate-limit override
35+
id: lychee-policy
36+
shell: bash --noprofile --norc -euo pipefail {0}
37+
env:
38+
GH_TOKEN: ${{ github.token }}
39+
GH_REPO: ${{ github.repository }}
40+
PR_NUMBER: ${{ inputs.pr-number }}
41+
run: |
42+
allow_429=false
43+
if [[ -n "$PR_NUMBER" ]]; then
44+
# Reruns keep the original event's labels, so query the current decision.
45+
# Only an exact boolean response can enable the maintainer override.
46+
lookup_ok=false
47+
for attempt in 1 2 3; do
48+
if result="$(gh pr view "$PR_NUMBER" --json labels --jq '.labels | any(.name == "ci: lychee-non-blocking")')"; then
49+
case "$result" in
50+
true|false) allow_429="$result"; lookup_ok=true; break ;;
51+
esac
52+
fi
53+
if (( attempt < 3 )); then
54+
sleep $((attempt * 5))
55+
fi
56+
done
57+
if [[ "$lookup_ok" != true ]]; then
58+
echo "::warning::Could not read a valid PR label decision; using the default link policy. Rerun to apply ci: lychee-non-blocking."
59+
elif [[ "$allow_429" == true ]]; then
60+
echo "::notice::ci: lychee-non-blocking permits only HTTP 429 failures after retries."
61+
fi
62+
fi
63+
echo "allow-429=$allow_429" >> "$GITHUB_OUTPUT"
64+
2465
- name: Describe the link cache
2566
id: cache
2667
shell: bash --noprofile --norc -euo pipefail {0}
@@ -56,7 +97,7 @@ runs:
5697
shell: bash --noprofile --norc -euo pipefail {0}
5798
env:
5899
KIND: ${{ inputs.kind }}
59-
POLICY: ${{ inputs.policy }}
100+
POLICY: ${{ steps.lychee-policy.outputs.allow-429 == 'true' && 'allow-429' || inputs.policy }}
60101
GITHUB_TOKEN: ${{ github.token }}
61102
run: |
62103
# Install into the interpreter that runs the driver, whatever shell

‎.github/workflows/build-docs.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -301,6 +301,8 @@ jobs:
301301
uses: ./.github/actions/lychee-links
302302
with:
303303
kind: rendered
304+
# Full CI's pull-request/N mirror supplies the PR number.
305+
pr-number: ${{ env.PR_NUMBER }}
304306

305307
- name: Upload docs GitHub Pages artifact
306308
if: ${{ inputs.deploy-docs }}

‎.github/workflows/pre-commit.yml‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ concurrency:
2222

2323
permissions:
2424
contents: read
25+
pull-requests: read
2526

2627
defaults:
2728
run:
@@ -142,16 +143,25 @@ jobs:
142143
# Warm only Lychee from the shared nightly baseline, retaining successful
143144
# checks between up to three attempts. The full suite below runs once:
144145
# formatter changes must remain failures. A missing baseline is supported;
145-
# the first pass then checks afresh.
146+
# the first pass then checks afresh. With the maintainer label, this is
147+
# also the final 429-only link gate.
146148
- name: Warm the Lychee cache
149+
id: lychee-check
147150
uses: ./.github/actions/lychee-links
148151
with:
149152
kind: authored
150153
policy: cache-warm
154+
pr-number: ${{ github.event_name == 'pull_request' && github.event.pull_request.number || '' }}
151155

152156
- name: Run pre-commit
157+
# Run the remaining hooks even if the override's link gate fails.
158+
# That earlier failure still makes the job fail.
159+
if: ${{ !cancelled() && (success() || (steps.lychee-check.outputs.allow-429 == 'true' && steps.lychee-check.outcome == 'failure')) }}
153160
env:
154161
GITHUB_TOKEN: ${{ github.token }}
162+
# The override uses the reported link result above. All other hooks
163+
# still run once; normal runs also invoke Lychee in the full suite.
164+
SKIP: ${{ steps.lychee-check.outputs.allow-429 == 'true' && 'check-precommit-installed,lychee' || 'check-precommit-installed' }}
155165
run: pre-commit run --all-files --show-diff-on-failure
156166

157167
- name: Save pre-commit cache

‎ci/README.md‎

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,41 @@ fail explicitly. Per-attempt reports and a summary are kept for seven days.
4848
| Nightly (`lychee.yml`) | Afresh authored and rendered checks | Unresolved links are warnings; setup, execution, and report errors fail |
4949

5050
The warm-up only fills the cache so the full suite, whose formatter changes must
51-
stay failures, mostly reuses successful checks.
51+
stay failures, mostly reuses successful checks. With the maintainer override,
52+
the driver's report-based result becomes the final link gate, and the full
53+
suite skips its duplicate Lychee invocation. Every other hook still runs once.
54+
55+
### Maintainer Override For HTTP 429
56+
57+
When rate limiting persists, maintainers can apply `ci: lychee-non-blocking`
58+
to the PR and rerun the affected Linux/Windows pre-commit jobs or rendered
59+
documentation job. Applying or removing the label does not itself start a
60+
new workflow run. The shared link-check action queries the PR's current labels,
61+
so a rerun sees the current decision rather than the original event's snapshot.
62+
The lookup makes up to three attempts, waiting 5 and 10 seconds between them.
63+
If every attempt fails or returns an invalid response, the action warns and
64+
keeps the caller's default policy without enabling the override: authored
65+
checks remain strict in the full pre-commit suite, and rendered checks keep
66+
their allowance for at most ten distinct HTTP 429 URLs. Rerun the affected job
67+
to retry the lookup and apply the label; other pre-commit hooks still run.
68+
69+
The label selects the driver's `allow-429` policy for both authored and
70+
rendered checks. Lychee still runs and retries normally; only a final report
71+
whose remaining failures are all HTTP 429 responses from external HTTP(S)
72+
URLs can pass with a warning. This explicit override has no URL-count limit.
73+
The warning and retained reports identify the URLs that remain unverified.
74+
Successful checks retain their usual cache behavior; failures are never cached.
75+
76+
Any other failure still blocks, including a mixture of HTTP 429s and other
77+
errors, broken local files or fragments, timeouts, and HTTP 403, 404, or 5xx
78+
responses. Hook setup, invalid or missing reports, and the offline smoke
79+
checks remain blocking. Other exceptional failures require
80+
the normal admin-merge decision.
81+
82+
The label applies only to PR checks. Local pre-commit runs, default-branch
83+
checks, and nightly cache production keep their existing policies. Without
84+
the label, authored checks remain strict and rendered checks retain their
85+
existing allowance for at most ten distinct HTTP 429 URLs.
5286

5387
### Shared URL Caches
5488

‎ci/tools/retry_lychee.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,10 @@ def finish_link_failures(report: Path, data: Report, exit_codes: list[int], mess
157157
warning = True
158158
elif all(failure.rate_limited for failure in data.failures):
159159
message += f" {distinct_urls} distinct HTTP 429 URLs remain unverified"
160-
if distinct_urls <= MAX_RATE_LIMITED_URLS:
160+
if policy == "allow-429":
161+
message += "; the maintainer override accepts only these rate limits with a warning."
162+
warning = True
163+
elif distinct_urls <= MAX_RATE_LIMITED_URLS:
161164
message += f" (limit: {MAX_RATE_LIMITED_URLS}); accepting these rate limits with a warning."
162165
warning = True
163166
else:
@@ -184,7 +187,7 @@ def retry_lychee(
184187
require_link_check_exit(initial_exit_code, 1)
185188
if max_attempts < 1:
186189
raise ValueError("max_attempts must be positive")
187-
if policy not in ("check", "cache-warm"):
190+
if policy not in ("check", "cache-warm", "allow-429"):
188191
raise ValueError(f"Unknown lychee failure policy: {policy}")
189192
exit_codes = [initial_exit_code]
190193

‎ci/tools/run_lychee_precommit.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
22
# SPDX-License-Identifier: Apache-2.0
33

4-
"""Warm link caches or check rendered HTML through the configured pre-commit hooks."""
4+
"""Check documentation links or warm caches through the configured pre-commit hooks."""
55

66
from __future__ import annotations
77

@@ -89,7 +89,7 @@ def run_checks(kind: str, report: Path, max_attempts: int = 3, *, policy: str =
8989
def main(arguments: list[str] | None = None) -> int:
9090
parser = argparse.ArgumentParser(description=__doc__)
9191
parser.add_argument("kind", choices=("authored", "rendered"))
92-
parser.add_argument("--policy", choices=("cache-warm", "check"), default="check")
92+
parser.add_argument("--policy", choices=("cache-warm", "check", "allow-429"), default="check")
9393
parser.add_argument("--max-attempts", type=int, default=3)
9494
parser.add_argument("--report", type=Path, required=True)
9595
args = parser.parse_args(arguments)

‎ci/tools/tests/test_retry_lychee.py‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -350,10 +350,10 @@ def test_one_attempt_limit_keeps_transient_failure_without_retry(tmp_path, monke
350350
assert retry_lychee.retry_lychee(2, report, max_attempts=1, run_attempt=unexpected_retry) == 2
351351

352352

353-
@pytest.mark.parametrize("url_count,expected", [(10, 0), (11, 2)])
353+
@pytest.mark.parametrize("policy,url_count,expected", [("check", 10, 0), ("check", 11, 2), ("allow-429", 11, 0)])
354354
@pytest.mark.agent_authored(model="gpt-6")
355355
def test_rate_limit_allowance_counts_distinct_urls_after_all_attempts(
356-
tmp_path, monkeypatch, capsys, url_count, expected
356+
tmp_path, monkeypatch, capsys, policy, url_count, expected
357357
):
358358
report = tmp_path / "lychee.json"
359359
data = report_data(errors=[response(f"https://example.test/{number}") for number in range(url_count)], total=100)
@@ -365,13 +365,16 @@ def run(output):
365365
write_report(output, data)
366366
return 2
367367

368-
assert retry_lychee.retry_lychee(2, report, run_attempt=run) == expected
368+
assert retry_lychee.retry_lychee(2, report, policy=policy, run_attempt=run) == expected
369369
assert delays == [60, 120]
370370
assert json.loads(report.read_text(encoding="utf-8")) == data
371371
text = report.with_suffix(".md").read_text(encoding="utf-8")
372372
assert f"{url_count} distinct HTTP 429 URLs remain unverified" in text
373373
assert "| 1 | 2 |\n| 2 | 2 |\n| 3 | 2 |" in text
374374
assert ("::warning::" in capsys.readouterr().out) == (expected == 0)
375+
if policy == "allow-429":
376+
assert "maintainer" in text.lower()
377+
assert len(list(tmp_path.glob("lychee-attempt-*.json"))) == 3
375378
for number in range(url_count):
376379
assert f"https://example.test/{number}" in text
377380

@@ -393,23 +396,27 @@ def test_rate_limit_allowance_deduplicates_exact_urls_across_sources(tmp_path, c
393396
"other_failure,timed_out",
394397
[
395398
(response(code=404), False),
399+
(response(code=403), False),
396400
(response(code=503), False),
397401
(response(code=None), True),
398402
(response(code=429), True),
399403
(response("mailto:[email protected]"), False),
400404
(response("https:missing-host"), False),
401405
(response("https://[malformed"), False),
406+
(response("file:///docs/guide.html", code=429), False),
407+
(response("file:///docs/guide.html#missing", code=None, details="Anchor not found"), False),
402408
],
403409
)
410+
@pytest.mark.parametrize("policy", ["check", "allow-429"])
404411
@pytest.mark.agent_authored(model="gpt-6")
405-
def test_429_allowance_never_accepts_other_failures(tmp_path, capsys, other_failure, timed_out):
412+
def test_429_allowance_never_accepts_other_failures(tmp_path, capsys, other_failure, timed_out, policy):
406413
report = tmp_path / "lychee.json"
407414
data = report_data(
408415
errors=[response(), *([] if timed_out else [other_failure])], timeouts=[other_failure] if timed_out else []
409416
)
410417
write_report(report, data)
411418

412-
assert retry_lychee.retry_lychee(2, report, max_attempts=1, run_attempt=unexpected_retry) == 2
419+
assert retry_lychee.retry_lychee(2, report, max_attempts=1, policy=policy, run_attempt=unexpected_retry) == 2
413420
assert "::warning::" not in capsys.readouterr().out
414421

415422

‎ci/tools/tests/test_run_lychee_precommit.py‎

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -128,26 +128,49 @@ def run_hook(kind, output):
128128

129129
monkeypatch.setattr(run_lychee_precommit, "run_hook", run_hook)
130130
monkeypatch.setattr(retry_lychee.time, "sleep", delays.append)
131-
monkeypatch.delenv("LYCHEE_ARGS", raising=False)
132131
assert run_lychee_precommit.run_checks("authored", report, policy="cache-warm") == 0
133132
assert len(attempts) == 2
134133
assert delays == [60]
135134
assert "passed on attempt 2/3" in report.with_suffix(".md").read_text()
136135
assert "example.test" not in report.with_suffix(".md").read_text()
137136

138137

138+
@pytest.mark.parametrize("policy", ["cache-warm", "allow-429"])
139139
@pytest.mark.agent_authored(model="gpt-6.1-sol")
140-
def test_missing_native_status_is_fatal_even_for_nightly_cache_warming(tmp_path, monkeypatch, capsys):
140+
def test_missing_native_status_is_fatal_even_under_relaxed_link_policies(tmp_path, monkeypatch, capsys, policy):
141141
monkeypatch.setattr(
142142
run_lychee_precommit.subprocess, "run", lambda *_args, **_kwargs: subprocess.CompletedProcess([], 1)
143143
)
144-
assert (
145-
run_lychee_precommit.main(["authored", "--policy", "cache-warm", "--report", str(tmp_path / "report.json")])
146-
== 1
147-
)
144+
assert run_lychee_precommit.main(["authored", "--policy", policy, "--report", str(tmp_path / "report.json")]) == 1
148145
assert "no native status" in capsys.readouterr().out
149146

150147

148+
@pytest.mark.parametrize("kind", ["authored", "rendered"])
149+
@pytest.mark.agent_authored(model="gpt-6")
150+
def test_cli_maintainer_override_accepts_only_rate_limits_after_retrying(tmp_path, monkeypatch, kind):
151+
report = tmp_path / "report.json"
152+
calls = []
153+
154+
def run_hook(selected_kind, output):
155+
calls.append(selected_kind)
156+
data = native_report(code=2, total=20, status=429)
157+
data["errors"] = 11
158+
data["error_map"]["guide.md"] = [
159+
{"url": f"https://example.test/{number}", "status": {"text": "Rate limited", "code": 429}}
160+
for number in range(11)
161+
]
162+
output.write_text(json.dumps(data))
163+
return 2
164+
165+
monkeypatch.setattr(run_lychee_precommit, "run_hook", run_hook)
166+
monkeypatch.setattr(retry_lychee.time, "sleep", lambda _: None)
167+
168+
assert run_lychee_precommit.main([kind, "--policy", "allow-429", "--report", str(report)]) == 0
169+
assert calls == [kind] * 3
170+
assert len(list(tmp_path.glob("report-attempt-*.json"))) == 3
171+
assert "maintainer" in report.with_suffix(".md").read_text().lower()
172+
173+
151174
@pytest.mark.agent_authored(model="gpt-6.1-sol")
152175
def test_zero_attempts_rejected_before_running_precommit(tmp_path, monkeypatch):
153176
monkeypatch.setattr(run_lychee_precommit, "run_hook", lambda *_args: pytest.fail("Unexpected invocation"))

0 commit comments

Comments
 (0)