fix: improve link checker reliability against GitHub rate limiting#95
Merged
zeitlinger merged 8 commits intomainfrom Mar 9, 2026
Merged
fix: improve link checker reliability against GitHub rate limiting#95zeitlinger merged 8 commits intomainfrom
zeitlinger merged 8 commits intomainfrom
Conversation
The config change detection in links.sh treated any mise.toml modification as a config change, triggering a full link check. This caused unnecessary 429 rate limiting on Renovate branches that only bump tool versions. Now mise.toml is checked content-aware: only lychee-related lines (version or task config) trigger a full check. Also lower max_concurrency from 4 to 1 to reduce GitHub rate limiting on legitimate full checks. Signed-off-by: Gregor Zeitlinger <[email protected]>
martincostello
approved these changes
Mar 9, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes false-positive full link checks on Renovate PRs that only bump non-lychee tool versions in mise.toml. Previously, any change to mise.toml triggered a full lychee check; now only lines that mention "lychee" do. It also lowers max_concurrency from 4 to 1 in flint's own lychee.toml to reduce GitHub 429 rate limiting during full checks.
Changes:
is_config_modified()inlinks.shnow applies content-aware filtering tomise.tomldiffs, triggering a full check only when lychee-related lines are changedmax_concurrencyreduced from 4 to 1 in.github/config/lychee.toml- README updated to document the new content-aware
mise.tomldetection and the updated default value ofLYCHEE_CONFIG_CHANGE_PATTERN
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
tasks/lint/links.sh |
Adds content-aware mise.toml diff check to is_config_modified() |
.github/config/lychee.toml |
Reduces max_concurrency from 4 to 1 to limit GitHub 429 errors |
README.md |
Documents new mise.toml content-aware behavior and updated default pattern |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Gregor Zeitlinger <[email protected]>
Signed-off-by: Gregor Zeitlinger <[email protected]>
Signed-off-by: Gregor Zeitlinger <[email protected]>
- Skip mise.toml content check when LYCHEE_CONFIG_CHANGE_PATTERN is overridden (consumer has taken full control) - Document recommended max_concurrency=2 in per-repo config section Signed-off-by: Gregor Zeitlinger <[email protected]>
Signed-off-by: Gregor Zeitlinger <[email protected]>
With max_retries=6, each 429'd link generates 6 more requests that themselves count against the rate limit, causing more 429s — a retry storm. Lower to 2 retries with 10s wait to break the cycle. Signed-off-by: Gregor Zeitlinger <[email protected]>
Remap all github.com/blob/ URLs to raw.githubusercontent.com to avoid GitHub API rate limiting (429). raw.githubusercontent.com serves files without rate limiting and returns 404 for missing files, so link validity is still verified. Fragment checking also works better since raw serves actual file content. Tree URLs stay on github.com since raw doesn't serve directories. Signed-off-by: Gregor Zeitlinger <[email protected]>
This was referenced Mar 10, 2026
zeitlinger
pushed a commit
that referenced
this pull request
Mar 11, 2026
🤖 I have created a release *beep* *boop* --- ## [0.8.0](v0.7.1...v0.8.0) (2026-03-11) ### Features * add native linting mode and version mapping infrastructure ([#93](#93)) ([24b06da](24b06da)) ### Bug Fixes * add 'mise run fix' hint to lint failure output ([#90](#90)) ([5b4ad5d](5b4ad5d)) * improve link checker reliability against GitHub rate limiting ([#95](#95)) ([7a5282d](7a5282d)) * remap same-repo GitHub URLs to local file paths ([#100](#100)) ([b4feadd](b4feadd)) --- > [!IMPORTANT] > Close and reopen this PR to trigger CI checks. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
zeitlinger
pushed a commit
that referenced
this pull request
Apr 13, 2026
## flint v0.20.0 — Rust rewrite flint has been rewritten from scratch in Rust. The bash + Docker stack is replaced by a fast, cross-platform native binary with no container dependency. ### Highlights - **Native binary** — no Docker, no bash wrappers; runs anywhere mise can install a Rust crate - **Parallel linting** — all checks run concurrently - **Diff-aware** — only checks files changed since the merge base by default (`--full` for everything) - **`flint update`** — migrates obsolete `mise.toml` tool keys automatically (e.g. `ubi:` → `github:`, `npm:markdownlint-cli` → `npm:markdownlint-cli2`) - **`flint hook install`** — installs a pre-commit hook without any mise task knowledge - **New linters**: `gofmt`, `google-java-format`, `ktlint`, `dotnet-format`, `markdownlint-cli2`, `xmllint` (via `cargo:xmloxide`), `license-header` (pure-Rust, no binary) - **Windows support** — ktlint self-executing JAR handled via explicit `java -jar` invocation ### Migration See [AGENTS-V2.md](https://github.com/grafana/flint/blob/main/AGENTS-V2.md) and [README.md](https://github.com/grafana/flint/blob/main/README.md) for full setup instructions. Quick reference for existing consumers: ```toml [tools] # While installing from source (pre-crates.io release): "cargo:https://github.com/grafana/flint" = "branch:main" [env] FLINT_CONFIG_DIR = ".github/config" [tasks.lint] run = "flint run" [tasks."lint:fix"] run = "flint run --fix" ``` Remove: `lint:super-linter`, `lint:links`, `lint:renovate-deps`, `setup:native-lint-tools`, `setup:pre-commit-hook` tasks. Run `flint update` to auto-migrate any obsolete tool keys. > [!NOTE] > The changelog below includes entries from the legacy v1 bash era — see [README-V1.md](https://github.com/grafana/flint/blob/main/README-V1.md) for v1 docs. --- ## [0.20.0](flint-v0.19.0...flint-v0.20.0) (2026-04-13) ### Features * add flint v2 Rust binary ([#139](#139)) ([19f2b25](19f2b25)) * add native linting mode and version mapping infrastructure ([#93](#93)) ([24b06da](24b06da)) * add Renovate shareable preset for consuming repos ([#17](#17)) ([8a06590](8a06590)) * consolidate link checking and add autofix flags ([#7](#7)) ([086a5e9](086a5e9)) * flint update command, explicit JAR flag, v0.20.0 ([#146](#146)) ([b43bf52](b43bf52)) * handle line-number anchors and issue comments globally ([#56](#56)) ([cf751df](cf751df)) * **links:** add GitHub URL remaps for line-number and fragment anchors ([#28](#28)) ([5b59065](5b59065)) * **links:** auto-remap base-branch GitHub URLs to PR branch ([#18](#18)) ([dd6cc61](dd6cc61)) * **renovate:** support SHA-pinned URLs in Renovate preset ([#21](#21)) ([4fd1f28](4fd1f28)) * **super-linter:** default to slim image ([#24](#24)) ([c8eeab8](c8eeab8)) * support NATIVE env var for container-free linting ([#107](#107)) ([0a8193d](0a8193d)) ### Bug Fixes * activate mise environment in native lint mode ([#123](#123)) ([d0fec45](d0fec45)) * add 'mise run fix' hint to lint failure output ([#90](#90)) ([5b4ad5d](5b4ad5d)) * decouple version mapping generation from pinned super-linter version ([#112](#112)) ([5370e77](5370e77)) * **deps:** update rust crate crossterm to 0.29 ([#156](#156)) ([c59ae3e](c59ae3e)) * **deps:** update rust crate similar to v3 ([#160](#160)) ([684be4e](684be4e)) * **deps:** update rust crate toml to v1 ([#161](#161)) ([3aae614](3aae614)) * **deps:** update rust crate toml_edit to 0.25 ([#158](#158)) ([42d9efd](42d9efd)) * exclude GitHub compare links from lychee checks ([#10](#10)) ([e714608](e714608)) * fail native lint when enabled tools are missing ([#111](#111)) ([163bb6b](163bb6b)) * improve link checker reliability against GitHub rate limiting ([#95](#95)) ([7a5282d](7a5282d)) * include staged files in native lint file list ([#135](#135)) ([34412d6](34412d6)) * **links:** add regex anchors to remap patterns ([#19](#19)) ([2e17348](2e17348)) * native lint in worktrees, trust toml, use ec binary, drop isort ([#134](#134)) ([8594bba](8594bba)) * **release-please:** fix footer not appearing on release PRs ([#40](#40)) ([d7a55e4](d7a55e4)) * remap same-repo GitHub URLs to local file paths ([#100](#100)) ([b4feadd](b4feadd)) * **renovate-deps:** forward GITHUB_TOKEN as GITHUB_COM_TOKEN ([#132](#132)) ([4d6510b](4d6510b)) * replace broken release-please PR comment with docs ([#12](#12)) ([817b37d](817b37d)) * run shellcheck on .bats files in native mode ([#137](#137)) ([a4fd3f8](a4fd3f8)) * strip Scroll to Text Fragment anchors in link checks ([#86](#86)) ([b630cdf](b630cdf)) * tighten markdownlint config for native mode ([#106](#106)) ([6ef25b2](6ef25b2)) * use remap instead of exclude for issue comment anchors ([#58](#58)) ([656f355](656f355)) --- > [!IMPORTANT] > Close and reopen this PR to trigger CI checks. Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This was referenced Apr 16, 2026
Closed
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
is_config_modified()inlinks.shno longer treats everymise.tomlchange as a config change — only lychee-related diff lines (version or task config) trigger a full link checkgithub.com/blob/URLs are remapped toraw.githubusercontent.com, which serves files without rate limiting and still returns 404 for missing filesmax_retriesfrom 6 to 2 and increaseretry_wait_timefrom 5s to 10s to avoid compounding 429 rate limiting (each retry generates more requests that trigger more 429s)Context
Renovate branches that only bump tool versions (e.g.
npm-renovate) were falsely triggering full link checks because the old logic matchedmise.tomlby filename alone. Combined with aggressive retries and GitHub rate limiting, this caused CI failures — see grafana-opentelemetry-java CI failure.Consumer action needed
After this release, consuming repos should update their
.github/config/lychee.toml:max_retries = 2(was 6)retry_wait_time = 10(was 5)The blob→raw remap and config detection fixes are picked up automatically via the remote task script.
Test plan
mise.toml— should NOT trigger full checkmise.toml— should trigger full check