Skip to content

feat: mute known findings via an exclusions config key - #1845

Merged
AlexsJones merged 5 commits into
k8sgpt-ai:mainfrom
CJstate:feat/exclusions
Nov 12, 2026
Merged

AlexsJones merged 5 commits into
k8sgpt-ai:mainfrom
CJstate:feat/exclusions

Conversation

@CJstate

@CJstate CJstate commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Part of #372 — implements the core from the design note posted earlier in the issue.

📑 Description

Users have asked since 2023 to silence findings they already know about so an analysis surfaces only new issues. This adds an exclusions key to the existing config file, next to active_filters and custom_analyzers:

exclusions:
  - reason: "openebs local path service is expected on this cluster"   # required
    hash: "a1b2c3d4e5f6"                                              # 12 hex chars, see below
    expires: "2026-12-31"                                             # optional, RFC3339
  • Analyzer-agnostic: ApplyExclusions() runs once after RunAnalysis/RunCustomAnalysis and before GetAIResults/PrintOutput — covering built-in and custom analyzers, skipping the AI call for muted findings, and applying in server mode too (the handler runs it the same way).
  • Hash (FindingHash): SHA-256 over Kind + namespace/name + whitespace-normalized failure text, first 12 hex chars, computed before any anonymization. Whitespace normalization keeps the hash stable across pod restarts (container-restart counts drift between runs) while still changing when the finding's substance changes — the mute lifts on its own, so suppressions cannot go stale.
  • Partial muting: a Result survives unless every failure in it is muted; the JSON output reports dropped failures as excluded, so a mute cannot silently swallow findings.
  • Validation: an entry without a reason is ignored and reported as an analysis error; an expired entry stops applying and the finding surfaces again.
  • A k8sgpt exclusions add/list/remove command will follow as a second PR so this one stays small, as discussed in the issue.

✅ Checks

  • My pull request adheres to the code style of this project
  • My code requires changes to the documentation
  • I have updated the documentation as required
  • All the tests have passed

ℹ Additional Information

Five new tests cover the mute path, whitespace stability, partial muting, expiry, and the reasonless-entry rejection. The hash recipe intentionally matches the design note in #372; if the proto's AnalyzeResponse should also carry excluded, happy to add that in the follow-up PR.

@CJstate
CJstate requested review from a team as code owners October 18, 2026 11:50
@github-project-automation github-project-automation Bot moved this to Proposed in Backlog Oct 18, 2026
@CJstate

CJstate commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Heads-up on the failing License Compliance check: this diff is purely additive Go code (no dependency changes — verified git diff origin/main --stat shows only 6 Go files, +255/-0), so the "1 issues found" is in the dependency tree this branch inherits from the latest main, where containerd was just bumped to v1.7.36 in #1834. Earlier PRs based on the previous main (e.g. #1841, #1804) pass the same check. Happy to help chase it if it turns out to be on this side after all.

@AlexsJones AlexsJones left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @CJstate, this follows the design note closely and the ApplyExclusions placement (after both analysis passes, before GetAIResults, and in the server handler) is exactly right.

A few things before this can go in:

  1. Users have no way to get a hash. The design note says k8sgpt analyze prints each finding's hash (text suffix plus a JSON hash field), but nothing in this diff outputs it, so the only way to write an exclusion today is to recompute SHA-256 by hand. Since the CLI subcommand is a follow-up, the hash output needs to land in this PR or the feature isn't usable on its own.
  2. expires rejects the documented format. The PR description and the design note both use expires: "2026-12-31", but loadExclusions in pkg/analysis/exclusions.go parses with time.RFC3339, so that value goes to a.Errors and the entry is dropped. Accept time.DateOnly as well (treating it as end of that day is probably the least surprising), and add the date-only case to the expiry test.
  3. Excluded count only shows in JSON. Excluded is wired into jsonOutput but the text output doesn't mention it, while the note promised both. A one-line "N findings excluded" in text output is enough.

The License Compliance failure isn't yours, agreed: it's coming from main's dependency tree, so no need to touch it here.

@CJstate

CJstate commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review - all three points are addressed in 0cf69e8.

1. Users can now get a hash. The hash is printed after each finding in text output and exposed as its own hash field in JSON, so writing an exclusion no longer requires recomputing SHA-256 by hand:

`

  • Error: the last termination reason is OOMKilled container=app pod=web hash=d85cb9e0ae06
    `

It is projected at render time from the same inputs ApplyExclusions matches on - Kind, Name and the pre-anonymization failure text - so the printed value is exactly the one the mute path compares. I kept it out of a.Results deliberately: it is a rendering concern, so it can never leak into the AI provider payload or feed back into exclusion matching. TestOutputDoesNotMutateAnalysisResults pins that down.

2. expires accepts the documented format. parseExpiry keeps RFC 3339 as the precise form and treats "2026-12-31" as the end of that local day, which seemed like the least surprising reading. Malformed values are still reported through a.Errors and the entry is dropped, so nothing is silently swallowed. Covered by TestApplyExclusionsAcceptsDateOnlyExpiry, ...DateOnlyExpiryCoversTheWholeDay and ...RejectsMalformedExpiry.

3. Excluded count now shows in text output. A N findings excluded line is printed whenever something was muted, in both the no-problems and the problems-present branch, matching what JSON already exposed.

go test ./pkg/analysis/... passes. The License Compliance failure is untouched and still comes from main's dependency tree, as you noted.

One unrelated note while running the suite on Windows: TestEnsureDirExists in pkg/util fails there because it asserts on the Unix mkdir wording ("no such file or directory" vs "The system cannot find the path specified"). It is orthogonal to this PR, but happy to send a separate fix for it if that is useful.

@CJstate

CJstate commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

One more thing worth flagging, because it affects both this PR and #1841.

The License Compliance status is the only thing left blocking either from merging, and it is not coming from these diffs. It reports exactly 1 issues found on both branches - and also on a branch that adds a single test file with no import changes - which points at the dependency tree rather than at changed code. Since it also failed on main's tree before either branch was cut, a maintainer-side waiver or a fix on main is what actually unblocks these two, not another round of changes here.

I also fixed a DCO slip in 92e4406 (the follow-up commit had gone in without a sign-off); both commits are now signed off and the tree is byte-identical to what was there before, so no re-review of the code is needed for that.

Still happy to take the pkg/util TestEnsureDirExists Windows fix as a separate PR if you want it - it is a one-line assertion that assumes the Unix mkdir wording.

@CJstate

CJstate commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Correction to my previous comment — I need to walk back one claim.

I wrote that the License Compliance failure "comes from main's dependency tree rather than this diff". I now have evidence that contradicts that reasoning, and I would rather retract it than leave a wrong explanation attached to this PR.

What is true:

If the failure came purely from the base dependency tree, #1848 would fail as well. It does not, so my "it's the base tree" explanation cannot be right. Since #1841 adds a single test file and never touches go.mod/go.sum, the difference is being generated by the branch content in some way I cannot observe from outside — Fossa reports only the count, not the offending entry, so I have no way to tell whether that entry is triggered by this branch or is a pre-existing policy item that Fossa simply reports inconsistently across refs.

I cannot fix what I cannot see, so I am not going to guess at a dependency change. What I can offer:

  • go.mod and go.sum are untouched by this PR, so no dependency was introduced here.
  • If you can read the Fossa report for 92e4406 and tell me the entry, I will address it in this PR or in a separate one, whichever you prefer.
  • test: assert the EnsureDirExists failure portably #1848 is a useful control: same base, passing License check.

Sorry for the noise on the diagnosis. The code review points are unaffected — all three are addressed in 92e4406.

@CJstate

CJstate commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

The License Compliance failure on this PR is not from this diff — it is inherited from main. I traced it to #1834 /containerd v1.7.36 pulling in ilepath-securejoin v0.5.1, which relicensed to MPL-2.0. Evidence and options are in #1850.

@AlexsJones AlexsJones left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @CJstate, all three points are covered, and keeping the hash as a render-only projection is a nice touch.

Checked at 92e4406:

  1. Text output prints hash=<12 hex> after each failure, and JSON gets a hash field from resultsWithFindingHashes, which copies the results so the hash stays out of the AI payload. I also checked --anonymize: GetAIResults masks a range copy of each failure, so a.Results keeps the raw text and the printed hash matches what ApplyExclusions checks.
  2. parseExpiry accepts 2026-12-31 and treats it as the end of that day, with tests.
  3. "N findings excluded" now shows in text output, including the no-problems case.

Looks good to me. It's a feat and adds a new config key, so the final call is a manual merge, and the Go build and test workflows haven't run on this head yet (License Compliance is the main issue in #1850).

@CJstate CJstate mentioned this pull request Oct 24, 2026
2 of 4 tasks
@codecov

codecov Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.66667% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.91%. Comparing base (173e4dc) to head (f15aa57).
⚠️ Report is 423 commits behind head on main.

Files with missing lines Patch % Lines
pkg/analysis/output.go 81.81% 2 Missing and 2 partials ⚠️
pkg/analysis/exclusions.go 94.00% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1845       +/-   ##
===========================================
+ Coverage   34.76%   61.91%   +27.14%     
===========================================
  Files          94      131       +37     
  Lines        6342     8710     +2368     
===========================================
+ Hits         2205     5393     +3188     
+ Misses       4046     2961     -1085     
- Partials       91      356      +265     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AlexsJones

Copy link
Copy Markdown
Member

The fork workflows ran at 46ceb9f, and everything is green except golangci-lint, which flags the line this PR touches in pkg/analysis/output.go (line 133):

QF1012: Use fmt.Fprintf(...) instead of WriteString(fmt.Sprintf(...)) (staticcheck)

The linter only reports changed lines, which is why its neighbours pass. Switching that one call to fmt.Fprintf(&output, "- %s %s %s\n", ...) with the same arguments should clear it. The rest of the approval stands, and I'll sign it off once lint is green.

Users have asked since 2023 to silence findings they already know about,
so an analysis surfaces only new issues (k8sgpt-ai#372). Add an `exclusions` key
to the existing viper config, next to active_filters and
custom_analyzers: each entry carries a required reason, a 12-hex-char
content hash, and an optional RFC3339 expiry.

FindingHash derives that hash from Kind, the result's namespace/name
key, and the whitespace-normalized failure text, before any
anonymization; normalizing whitespace keeps the hash stable across pod
restarts (container-restart counts drift) while still changing when the
finding's substance changes. ApplyExclusions runs analyzer-agnostically
after RunAnalysis/RunCustomAnalysis and before GetAIResults/PrintOutput,
so it covers built-in and custom analyzers, skips the AI call for muted
findings, and applies in server mode too. A Result survives unless every
failure in it is muted, and the JSON output reports how many failures
were dropped as `excluded`, so a mute cannot silently swallow findings.

A k8sgpt exclusions add/list/remove command will follow as a second PR
so this one stays small, per the design note in k8sgpt-ai#372.

Signed-off-by: CJstate <[email protected]>
Addresses the three points from review:

1. Findings now carry their exclusion hash. `analyze` prints `hash=<12>`
   after each finding in text output and exposes it as a `hash` field in
   JSON, so the value written into the exclusions config can be read off the
   output instead of being recomputed by hand. The hash is projected at
   render time from the same inputs `ApplyExclusions` matches on (Kind, Name
   and the pre-anonymization failure text), so it is identical to the value
   the mute path compares and it never enters the AI provider payload.

2. `expires` accepts the documented date-only form. `parseExpiry` keeps
   RFC 3339 as the precise form and treats `2026-12-31` as the end of that
   local day, so an exclusion dated today still covers today. Malformed values
   are still reported and the entry is dropped.

3. Text output reports the mute count. A `N findings excluded` line is
   printed when anything was muted, both in the no-problems and the
   problems-present branch, matching what JSON already exposes.

Tests cover the text `hash=` suffix, the JSON `hash` field, the excluded
count in text, the date-only and RFC 3339 expiry forms, a malformed expiry and
the fact that rendering does not mutate `a.Results`.

The License Compliance failure is unchanged and, as noted, comes from main's
dependency tree rather than this diff.

Signed-off-by: CJstate <[email protected]>
staticcheck QF1012 flags the new WriteString(fmt.Sprintf(...)) call; write
straight to the builder instead.

Signed-off-by: CJstate <[email protected]>

@AlexsJones AlexsJones left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rebase and the lint fix, @CJstate. CI is fully green at 881982b, including License Compliance, and the three earlier review points still hold.

One gap before I can sign this off: the MCP analyze tool doesn't apply exclusions. pkg/server/mcp.go line 480 calls analysis.RunAnalysis() and goes straight to GetAIResults, so a muted finding still shows up for MCP clients and still costs an AI call when Explain is set. The CLI and pkg/server/analyze/analyze.go are both covered, so this is the last entry point. Add analysis.ApplyExclusions() right after line 480, with a small test if you can.

Small one while you're there: the FindingHash comment says collapsing whitespace keeps the hash stable across restart counts, but a changed count changes the text and so the hash. Could you reword it to say only whitespace drift is ignored?

The MCP handleAnalyze ran RunAnalysis and went straight to GetAIResults
or PrintOutput, so a finding muted through the exclusions config still
reached MCP clients and still cost an AI call when Explain was set.
Apply ApplyExclusions right after RunAnalysis, matching the CLI and the
server-mode handler, and cover it with a regression test that mutes a
CrashLoopBackOff finding and asserts it disappears from the tool output.

Also reword the FindingHash comment: collapsing whitespace only ignores
whitespace drift; a changed restart count does change the hash.

Signed-off-by: CJstate <[email protected]>
@CJstate

CJstate commented Nov 7, 2026

Copy link
Copy Markdown
Contributor Author

Both points are addressed in f608ad8.

1. MCP analyze now applies exclusions. handleAnalyze calls analysis.ApplyExclusions() right after analysis.RunAnalysis(), matching the CLI and pkg/server/analyze, so a muted finding no longer reaches MCP clients and no longer costs an AI call when Explain is set.

The regression test (TestHandleAnalyzeAppliesExclusions in pkg/server/mcp_handlers_test.go) runs the MCP tool against a fake cluster with a CrashLoopBackOff pod: the first call shows the finding, the second call — with that finding's FindingHash configured under exclusions — must not contain it. It fails on the unpatched handler (the finding still appears, exactly the gap you flagged) and passes with the fix.

2. FindingHash comment reworded. It now says only whitespace drift is ignored; a changed container-restart count does change the hash.

Verified locally: go test ./pkg/server/... ./pkg/analysis/... all green.

@AlexsJones AlexsJones left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @CJstate, both points are handled cleanly.

Delta checked at f608ad8: handleAnalyze now calls analysis.ApplyExclusions() right after RunAnalysis(), so the MCP path matches the CLI and pkg/server/analyze. TestHandleAnalyzeAppliesExclusions shows the finding before the hash is muted and gone after, which is the right shape. The FindingHash comment now says only whitespace drift is ignored.

One thing before this can merge: GitHub reports the branch as conflicting with main (several pkg/server and pkg/ai PRs landed this week). Could you rebase? Once CI is green on the rebased head I'll sign off.

Approving the change itself.

@CJstate

CJstate commented Nov 7, 2026

Copy link
Copy Markdown
Contributor Author

Also covering the lint point from your review: pkg/analysis/output.go switched to fmt.Fprintf for the exclusions finding line in 881982b (pushed Oct 31, after your 46ceb9f run). The only line golangci-lint flagged is gone, and the neighbouring WriteString(fmt.Sprintf(...)) calls are untouched on main so they stay outside the changed-lines report. Your fork's CI should come back clean from f608ad8.

@AlexsJones

Copy link
Copy Markdown
Member

Thanks @CJstate, confirmed: pkg/analysis/output.go uses fmt.Fprintf for the finding line at f608ad8, so the lint point is settled.

The only thing left is the conflict with main (GitHub still reports the branch as conflicting). Once it's rebased and build and tests run green, I'll sign off.

Resolve the test-file conflict by keeping both the new MCP exclusions regression test and the filter validation tests added on main.
@CJstate

CJstate commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Hi @AlexsJones — thanks again for the approval on this one. The branch had fallen 31 commits behind main and picked up a merge conflict in pkg/server/mcp_handlers_test.go, so I rebased it to clear the dirty state.

The conflict was only in the test file: main added filter-validation tests at the same spot as the MCP exclusions regression test from this PR. I kept both sets (and the imports they need) — no production code changed. Verification on the rebased tree: go test ./... -count=1 and go vet ./... are both green, and the diff is still the same 10 files.

No rush, just flagging that it's mergeable again whenever you get a chance. Thanks!

@AlexsJones AlexsJones left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sorting the conflict, @CJstate, and for keeping both sets of tests in pkg/server/mcp_handlers_test.go.

Delta checked at f15aa57: I compared the PR diff before and after the merge of main. The production changes are identical, and the only movement is in the test file, where TestHandleAnalyzeAppliesExclusions now sits after main's filter tests. build, golangci-lint and codecov are green on this head.

This is a feat, signed off under the maintainer's 2026-10-24 decision that a fully reviewed feature with green CI can merge. It implements the exclusions design from #372.

Good to merge

@AlexsJones
AlexsJones merged commit a1b05df into k8sgpt-ai:main Nov 12, 2026
10 checks passed
@github-project-automation github-project-automation Bot moved this from Proposed to Done in Backlog Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants