You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Regression detection and diagnostic reasoning #13203
Task: Regression detection and diagnostic reasoning
Description
Implement the regression detection algorithm that compares each response's score against rolling averages and flags quality drops. Includes severity classification (minor/moderate/severe) and diagnostic reasoning that explains why a score was low, correlating sub-score drops with observable signals.
Acceptance Criteria
packages/opencode/src/quality/detector.ts created
Fetches rolling window (last N responses, configurable via regression.windowSize) from SQLite
Computes rolling average for composite score and each sub-score
Calculates delta: current_score - rolling_average
Severity classification: minor (-5 to -15), moderate (-15 to -30), severe (below -30)
Diagnostic reasoning generated for each regression: identifies which sub-scores dropped, correlates with signals (e.g., "3 new type errors", "user reverted change", "tool invocation failed")
Emits Quality.Event.Regression event on bus when severity >= configured alertLevel
Handles edge cases: fewer than windowSize responses (use available data), first response (no comparison, no regression)
Regression detection runs as part of the scoring pipeline (after score computed, before response to user)
Technical Details
Define Quality.Event.Regression using BusEvent.define() pattern
Query rolling averages from SQLite: SELECT AVG(composite_score), AVG(correctness_score), AVG(feedback_score), AVG(analysis_score) FROM responses ORDER BY timestamp DESC LIMIT ?
Diagnostic reasoning uses a template system: check each sub-score delta, match to known signal patterns, compose human-readable explanation
Regression event payload: { responseId, severity, delta, component, description } — TUI can subscribe to show inline warnings
Dependencies
Task 001: SQLite storage layer (for queries)
Task 003: Scoring engine (provides scores to compare)
Activity
github-actions commentedon Apr 13, 2026
To stay organized issues are automatically closed after 90 days of no activity. If the issue is still relevant please open a new one.