Repository navigation
Closed
Description
Task: Scoring engine
Description
Implement the core scoring engine that computes a composite 0-100 quality score for each AI response. The engine calculates three sub-scores (code correctness, user feedback, static analysis) and combines them using configurable weights. Scoring runs asynchronously to avoid blocking response delivery.
Acceptance Criteria
-
packages/opencode/src/quality/scorer.tscreated - Correctness sub-score (0-100): based on tool invocation success/failure, error presence in response, LSP diagnostic delta after changes
- User feedback sub-score (0-100): starts at neutral (70), adjusted retroactively when signals arrive (accept: +, revert: -30, redo: -15, edit-after-apply: -5)
- Static analysis sub-score (0-100): based on LSP diagnostic count delta (new errors: heavy penalty, new warnings: moderate, resolved issues: bonus)
- Composite score = weighted sum using config weights, clamped to 0-100
- Scoring is async — uses
queueMicrotask()orsetTimeout(0)after response delivery - Partial scoring supported — if LSP data unavailable, score with available signals only (weight redistributed)
- Score results persisted to SQLite via store.ts
- Scoring errors caught and logged, never propagated to main workflow
Technical Details
- Import config weights from quality config
- Import store functions from store.ts for persistence
- Correctness signals extracted from message metadata: tool results (success/error), response errors, token usage patterns
- Static analysis signals from LSP diagnostics snapshot before/after response application
- User feedback is deferred — initial score computed without it, updated retroactively
- Expose
Quality.scoreResponse(messageMetadata, lspBefore, lspAfter)as main API
Dependencies
- Task 001: SQLite storage layer (for persistence)
- Task 002: Quality configuration (for weights and settings)
Effort Estimate
- Size: L
- Hours: 8-10
- Parallel: false (depends on 001, 002)
Activity
github-actions commentedon Apr 13, 2026
To stay organized issues are automatically closed after 90 days of no activity. If the issue is still relevant please open a new one.