Skip to content

fix: establish full baselines when lint coverage changes #213

@zeitlinger

Description

@zeitlinger

Problem

flint v2 currently scopes normal PR runs to changed files, but it does not reliably establish a full baseline when lint coverage changes. If a repo adds a new linter, or changes a linter config, the affected linter must run against all matching files once. Otherwise pre-existing violations remain hidden until an unrelated PR later touches one of those files.

This surfaced during rollout when grafana/docker-otel-lgtm had an unrelated PR trip ShellCheck on an existing source ./logging.sh pattern. A full v2-style ShellCheck run on main would also have failed, but changed-file mode did not establish that baseline during migration.

There is also a ShellCheck parity bug: v2 used shellcheck {FILE}. For scripts in subdirectories that source sibling files, ShellCheck needs -x -P SCRIPTDIR so source ./file.sh resolves relative to the script directory.

Expected behavior

In changed-file mode:

  • normal PRs should keep running only affected files
  • newly active linters should run all matching files once to establish a baseline
  • changed per-linter config files should run all matching files for that linter
  • changed flint.toml should run all matching files for checks configured through flint.toml
  • ShellCheck should follow sourced files relative to the script directory

Local fix in progress

A local patch currently does the following:

  • routes newly active linters to an all-files file list
  • routes changed per-linter config files to all-files for that linter
  • routes changed flint.toml to all-files for special flint-configured checks
  • changes ShellCheck to shellcheck -x -P SCRIPTDIR {FILE}
  • adds regression fixtures for new-linter baseline, config-change baseline, and ShellCheck sourced sibling files

Local verification:

  • cargo test
  • mise run lint:fix

Consumer audit snapshot

Repo State Audit result / action
grafana/mox migrated / merged Full audit blocked locally by rustup trying to write under read-only host home. Needs rerun in CI or a cleaner local env.
grafana/grafana-opentelemetry-java migrated / merged Full audit failed: google-java-format, prettier, hadolint. Needs follow-up baseline/fix PR if main CI would run full.
grafana/docker-otel-lgtm migrated / merged Local full audit blocked by .NET/rustup install paths. ShellCheck sourced-file issue is confirmed and fixed in flint itself.
grafana/oats migrated / merged Full audit failed: markdownlint-cli2, prettier. Needs follow-up if full baseline is expected to pass.
grafana/otel-checker #267 open With patched flint and real merge base, baseline run fails editorconfig-checker on existing long lines. Needs PR branch fix/config adjustment.
prometheus/client_java #1988 merged Full audit failed: prettier. Needs follow-up if full baseline is expected to pass.
open-telemetry/opentelemetry-java-contrib #2776 merged Full audit failed: markdownlint-cli2. Needs follow-up if full baseline is expected to pass.
open-telemetry/opentelemetry-java-instrumentation #18114 open Passes with patched flint once the clone has a real merge base. Earlier markdownlint failure was a shallow-clone audit artifact. No fix needed there.
open-telemetry/opentelemetry-java-instrumentation #17759 draft/open Changed-mode audit failed fixable google-java-format and ktlint. Broader follow-up branch needs formatting fixes before it can be considered clean.
open-telemetry/opentelemetry-configuration #600 open Passes with patched flint and real merge base.
open-telemetry/opentelemetry-java #8315 open One lychee failure appeared in --short, but explicit flint run lychee passed afterward. Treat as transient unless CI reproduces.
open-telemetry/semantic-conventions-java #455 open Baseline exposed a broken CONTRIBUTING.md link. Fixed and pushed to zeitlinger/use-rust-flint-v2; lychee passes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions