fix(extract): prevent health checks with state=normal from being marked as noise#1096
Conversation
…ed as noise Adds one clarifying sentence to the is_noise=false rule so the LLM does not silently drop scheduled health checks or normal-state payloads. Tested against all 29 synthetic alert fixtures (eks + rds_postgres). Before: 28/29 correct — eks/000-healthy misclassified as is_noise=true. After: 29/29 correct — no regressions. Closes Tracer-Cloud#655 Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Greptile SummaryThis PR adds one clarifying sentence to the Confidence Score: 5/5Safe to merge — single-line prompt clarification with no logic changes, validated against 29 synthetic fixtures with zero regressions. The change is a one-sentence addition to a prompt string. All remaining findings are P2 style/quality suggestions that do not affect correctness. The PR author demonstrated 29/29 fixture pass rate and confirmed ruff/mypy green. No files require special attention. Important Files Changed
Sequence DiagramsequenceDiagram
participant Caller
participant extract_alert_details
participant LLM
participant AlertDetails
Caller->>extract_alert_details: state (raw_alert)
extract_alert_details->>extract_alert_details: _format_raw_alert()
note over extract_alert_details: Builds prompt with<br/>is_noise classifier rules<br/>(incl. new state=normal sentence)
extract_alert_details->>LLM: prompt (classify + extract)
alt LLM succeeds
LLM-->>extract_alert_details: AlertDetails (is_noise, fields…)
extract_alert_details-->>Caller: AlertDetails
else LLM fails
extract_alert_details->>extract_alert_details: _fallback_details()<br/>is_noise=False always
extract_alert_details-->>Caller: AlertDetails (fallback)
end
Reviews (1): Last reviewed commit: "fix(extract): prevent health checks with..." | Re-trigger Greptile |
|
🎲 Researchers are baffled. @AarushSharmaa opened a PR, got it reviewed without drama, and merged clean. This violates known laws of open source. 🔬
👋 Join us on Discord - OpenSRE : hang out, contribute, or hunt for features and issues. Everyone's welcome. |
…ed as noise (Tracer-Cloud#1096) Adds one clarifying sentence to the is_noise=false rule so the LLM does not silently drop scheduled health checks or normal-state payloads. Tested against all 29 synthetic alert fixtures (eks + rds_postgres). Before: 28/29 correct — eks/000-healthy misclassified as is_noise=true. After: 29/29 correct — no regressions. Closes Tracer-Cloud#655 Co-authored-by: Aarush Sharma <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2 participants
What and why
Closes #655.
The
is_noiseclassifier inextract_alert_detailswas silently dropping scheduled health checks and normal-state payloads. The prompt already said health checks are not noise, but the wording was not strong enough. The LLM would seestate: normalor a summary like "Periodic health check passed. All Kubernetes signals within normal operating bounds." and override the rule anyway.This PR adds one clarifying sentence to the
is_noise=falserule to name the exact patterns that were being misclassified.Change
One sentence added to the prompt in
app/nodes/extract_alert/extract.py(line 57).Before:
After:
No logic, schema, or import changes.
Proof
Tested against all 29 synthetic alert fixtures in
tests/synthetic/using the Groq API (llama-3.3-70b-versatile, temp=0, same prompt template as production).Result: 28/29 to 29/29. Zero regressions.
Full run output (29 alerts, new prompt)
Quality gates
ruff check app/nodes/extract_alert/extract.py: all checks passedmypy app/nodes/extract_alert/extract.py: no issues found