Skip to content

fix: improve link checker reliability against GitHub rate limiting#95

Merged
zeitlinger merged 8 commits intomainfrom
fix/lychee-config-detection
Mar 9, 2026
Merged

fix: improve link checker reliability against GitHub rate limiting#95
zeitlinger merged 8 commits intomainfrom
fix/lychee-config-detection

Conversation

@zeitlinger
Copy link
Copy Markdown
Member

@zeitlinger zeitlinger commented Mar 9, 2026

Summary

  • Content-aware config detection: is_config_modified() in links.sh no longer treats every mise.toml change as a config change — only lychee-related diff lines (version or task config) trigger a full link check
  • Remap blob URLs to raw: all github.com/blob/ URLs are remapped to raw.githubusercontent.com, which serves files without rate limiting and still returns 404 for missing files
  • Reduce retry storm: lower max_retries from 6 to 2 and increase retry_wait_time from 5s to 10s to avoid compounding 429 rate limiting (each retry generates more requests that trigger more 429s)

Context

Renovate branches that only bump tool versions (e.g. npm-renovate) were falsely triggering full link checks because the old logic matched mise.toml by filename alone. Combined with aggressive retries and GitHub rate limiting, this caused CI failures — see grafana-opentelemetry-java CI failure.

Consumer action needed

After this release, consuming repos should update their .github/config/lychee.toml:

  • max_retries = 2 (was 6)
  • retry_wait_time = 10 (was 5)

The blob→raw remap and config detection fixes are picked up automatically via the remote task script.

Test plan

  • CI passes on this PR
  • After merge: create test PR in a consuming repo with only a non-lychee tool version bump in mise.toml — should NOT trigger full check
  • After merge: create test PR with lychee version change in mise.toml — should trigger full check

The config change detection in links.sh treated any mise.toml
modification as a config change, triggering a full link check.
This caused unnecessary 429 rate limiting on Renovate branches
that only bump tool versions.

Now mise.toml is checked content-aware: only lychee-related
lines (version or task config) trigger a full check. Also
lower max_concurrency from 4 to 1 to reduce GitHub rate
limiting on legitimate full checks.
Signed-off-by: Gregor Zeitlinger <[email protected]>
Copilot AI review requested due to automatic review settings March 9, 2026 17:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes false-positive full link checks on Renovate PRs that only bump non-lychee tool versions in mise.toml. Previously, any change to mise.toml triggered a full lychee check; now only lines that mention "lychee" do. It also lowers max_concurrency from 4 to 1 in flint's own lychee.toml to reduce GitHub 429 rate limiting during full checks.

Changes:

  • is_config_modified() in links.sh now applies content-aware filtering to mise.toml diffs, triggering a full check only when lychee-related lines are changed
  • max_concurrency reduced from 4 to 1 in .github/config/lychee.toml
  • README updated to document the new content-aware mise.toml detection and the updated default value of LYCHEE_CONFIG_CHANGE_PATTERN

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
tasks/lint/links.sh Adds content-aware mise.toml diff check to is_config_modified()
.github/config/lychee.toml Reduces max_concurrency from 4 to 1 to limit GitHub 429 errors
README.md Documents new mise.toml content-aware behavior and updated default pattern

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tasks/lint/links.sh
Comment thread .github/config/lychee.toml Outdated
- Skip mise.toml content check when LYCHEE_CONFIG_CHANGE_PATTERN is
  overridden (consumer has taken full control)
- Document recommended max_concurrency=2 in per-repo config section
Signed-off-by: Gregor Zeitlinger <[email protected]>
With max_retries=6, each 429'd link generates 6 more requests that
themselves count against the rate limit, causing more 429s — a retry
storm. Lower to 2 retries with 10s wait to break the cycle.
Signed-off-by: Gregor Zeitlinger <[email protected]>
Remap all github.com/blob/ URLs to raw.githubusercontent.com to
avoid GitHub API rate limiting (429). raw.githubusercontent.com
serves files without rate limiting and returns 404 for missing
files, so link validity is still verified. Fragment checking
also works better since raw serves actual file content.

Tree URLs stay on github.com since raw doesn't serve directories.
Signed-off-by: Gregor Zeitlinger <[email protected]>
@zeitlinger zeitlinger changed the title fix: avoid false full link check on unrelated mise.toml changes fix: improve link checker reliability against GitHub rate limiting Mar 9, 2026
@zeitlinger zeitlinger merged commit 7a5282d into main Mar 9, 2026
8 checks passed
@zeitlinger zeitlinger deleted the fix/lychee-config-detection branch March 9, 2026 18:15
zeitlinger pushed a commit that referenced this pull request Mar 11, 2026
🤖 I have created a release *beep* *boop*
---


## [0.8.0](v0.7.1...v0.8.0)
(2026-03-11)


### Features

* add native linting mode and version mapping infrastructure
([#93](#93))
([24b06da](24b06da))


### Bug Fixes

* add 'mise run fix' hint to lint failure output
([#90](#90))
([5b4ad5d](5b4ad5d))
* improve link checker reliability against GitHub rate limiting
([#95](#95))
([7a5282d](7a5282d))
* remap same-repo GitHub URLs to local file paths
([#100](#100))
([b4feadd](b4feadd))

---
> [!IMPORTANT]
> Close and reopen this PR to trigger CI checks.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
zeitlinger pushed a commit that referenced this pull request Apr 13, 2026
## flint v0.20.0 — Rust rewrite

flint has been rewritten from scratch in Rust. The bash + Docker stack
is replaced by a fast, cross-platform native binary with no container
dependency.

### Highlights

- **Native binary** — no Docker, no bash wrappers; runs anywhere mise
can install a Rust crate
- **Parallel linting** — all checks run concurrently
- **Diff-aware** — only checks files changed since the merge base by
default (`--full` for everything)
- **`flint update`** — migrates obsolete `mise.toml` tool keys
automatically (e.g. `ubi:` → `github:`, `npm:markdownlint-cli` →
`npm:markdownlint-cli2`)
- **`flint hook install`** — installs a pre-commit hook without any mise
task knowledge
- **New linters**: `gofmt`, `google-java-format`, `ktlint`,
`dotnet-format`, `markdownlint-cli2`, `xmllint` (via `cargo:xmloxide`),
`license-header` (pure-Rust, no binary)
- **Windows support** — ktlint self-executing JAR handled via explicit
`java -jar` invocation

### Migration

See
[AGENTS-V2.md](https://github.com/grafana/flint/blob/main/AGENTS-V2.md)
and [README.md](https://github.com/grafana/flint/blob/main/README.md)
for full setup instructions.

Quick reference for existing consumers:

```toml
[tools]
# While installing from source (pre-crates.io release):
"cargo:https://github.com/grafana/flint" = "branch:main"

[env]
FLINT_CONFIG_DIR = ".github/config"

[tasks.lint]
run = "flint run"

[tasks."lint:fix"]
run = "flint run --fix"
```

Remove: `lint:super-linter`, `lint:links`, `lint:renovate-deps`,
`setup:native-lint-tools`, `setup:pre-commit-hook` tasks. Run `flint
update` to auto-migrate any obsolete tool keys.

> [!NOTE]
> The changelog below includes entries from the legacy v1 bash era — see
[README-V1.md](https://github.com/grafana/flint/blob/main/README-V1.md)
for v1 docs.

---

##
[0.20.0](flint-v0.19.0...flint-v0.20.0)
(2026-04-13)


### Features

* add flint v2 Rust binary
([#139](#139))
([19f2b25](19f2b25))
* add native linting mode and version mapping infrastructure
([#93](#93))
([24b06da](24b06da))
* add Renovate shareable preset for consuming repos
([#17](#17))
([8a06590](8a06590))
* consolidate link checking and add autofix flags
([#7](#7))
([086a5e9](086a5e9))
* flint update command, explicit JAR flag, v0.20.0
([#146](#146))
([b43bf52](b43bf52))
* handle line-number anchors and issue comments globally
([#56](#56))
([cf751df](cf751df))
* **links:** add GitHub URL remaps for line-number and fragment anchors
([#28](#28))
([5b59065](5b59065))
* **links:** auto-remap base-branch GitHub URLs to PR branch
([#18](#18))
([dd6cc61](dd6cc61))
* **renovate:** support SHA-pinned URLs in Renovate preset
([#21](#21))
([4fd1f28](4fd1f28))
* **super-linter:** default to slim image
([#24](#24))
([c8eeab8](c8eeab8))
* support NATIVE env var for container-free linting
([#107](#107))
([0a8193d](0a8193d))


### Bug Fixes

* activate mise environment in native lint mode
([#123](#123))
([d0fec45](d0fec45))
* add 'mise run fix' hint to lint failure output
([#90](#90))
([5b4ad5d](5b4ad5d))
* decouple version mapping generation from pinned super-linter version
([#112](#112))
([5370e77](5370e77))
* **deps:** update rust crate crossterm to 0.29
([#156](#156))
([c59ae3e](c59ae3e))
* **deps:** update rust crate similar to v3
([#160](#160))
([684be4e](684be4e))
* **deps:** update rust crate toml to v1
([#161](#161))
([3aae614](3aae614))
* **deps:** update rust crate toml_edit to 0.25
([#158](#158))
([42d9efd](42d9efd))
* exclude GitHub compare links from lychee checks
([#10](#10))
([e714608](e714608))
* fail native lint when enabled tools are missing
([#111](#111))
([163bb6b](163bb6b))
* improve link checker reliability against GitHub rate limiting
([#95](#95))
([7a5282d](7a5282d))
* include staged files in native lint file list
([#135](#135))
([34412d6](34412d6))
* **links:** add regex anchors to remap patterns
([#19](#19))
([2e17348](2e17348))
* native lint in worktrees, trust toml, use ec binary, drop isort
([#134](#134))
([8594bba](8594bba))
* **release-please:** fix footer not appearing on release PRs
([#40](#40))
([d7a55e4](d7a55e4))
* remap same-repo GitHub URLs to local file paths
([#100](#100))
([b4feadd](b4feadd))
* **renovate-deps:** forward GITHUB_TOKEN as GITHUB_COM_TOKEN
([#132](#132))
([4d6510b](4d6510b))
* replace broken release-please PR comment with docs
([#12](#12))
([817b37d](817b37d))
* run shellcheck on .bats files in native mode
([#137](#137))
([a4fd3f8](a4fd3f8))
* strip Scroll to Text Fragment anchors in link checks
([#86](#86))
([b630cdf](b630cdf))
* tighten markdownlint config for native mode
([#106](#106))
([6ef25b2](6ef25b2))
* use remap instead of exclude for issue comment anchors
([#58](#58))
([656f355](656f355))

---
> [!IMPORTANT]
> Close and reopen this PR to trigger CI checks.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This was referenced Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants