Skip to content

test(recommendation): add automated scoring tests - #406

Merged
DivyashreeR008 merged 1 commit into
DivyashreeR008:mainfrom
Subodh-17:feat/issue-175-recommendation-scoring-tests
Oct 20, 2026
Merged

DivyashreeR008 merged 1 commit into
DivyashreeR008:mainfrom
Subodh-17:feat/issue-175-recommendation-scoring-tests

Conversation

@Subodh-17

Copy link
Copy Markdown
Contributor

Description

Fixes #175.

This PR introduces an automated, deterministic Vitest test suite for the agent recommendation engine in src/lib/agentRecommendation/scoring.test.js.

Scope and Coverage

The test suite covers the following areas:

  1. Free-Text Tokenization (tokenizeFreeText):

    • Whitespace trimming, null/undefined/empty input handling, and case normalization.
    • Punctuation delimiter splitting (/, &, !, ,, ;) while preserving allowed technical symbols (+, #, ., -).
    • Stop-word filtering against the configured STOP_WORDS set.
    • Short token filtering ($\le 2$ characters) and the 12-token ceiling (FREE_TEXT_TOKEN_LIMIT).
  2. Score Normalization (normalizeScore):

    • Standard integer scaling ($0-100$) and fractional rounding (Math.round).
    • Percentage ceiling clamping strictly at 100%.
    • Safeguards returning 0 for non-positive scores, non-positive denominators (division-by-zero protection), and non-finite inputs (NaN, Infinity, strings, null/undefined).
  3. Agent Scoring (scoreAgent & getMaxPossibleScore):

    • Zero-signal inputs producing score 0 and empty signal collections.
    • Isolated signals: exactCategory (+30), goalCategory (+20), taskType (+18 per task), capabilityKeyword (+6 per capability), providerExact (+8), providerAny / compatible (+4), freeTextName (+10), freeTextDescription (+5), experience (+3), and urgency (+3).
    • Unmatched provider, experience, or budget criteria contributing 0 bonus points.
    • Composite multi-signal accumulation and custom weight overrides.
    • Safe handling of empty or undefined agent objects.
    • Theoretical maximum score formula verification.
  4. Recommendation Ranking & Public Boundary (recommendAgents):

    • Public boundary sanitization: filtering out null, undefined, and agent records missing id.
    • Deduplication of agents sharing the same id.
    • Deterministic 4-level sorting tie-breakers:
      1. Score descending
      2. exactCategory match presence
      3. provider match presence
      4. Agent name alphabetical ascending
    • Result limit enforcement (RESULT_LIMIT = 10 default and custom limits).
    • Strict confidence threshold evaluation across below (11), at (12), and above (13) threshold boundaries under default weights (MIN_CONFIDENT_SCORE = 12).
    • Output schema compliance: verifying result objects contain { agentId, score, matchedSignals, reasons, isConfident, matchPercentage } and do not leak the internal agent reference.
  5. Explanation Integrity (buildRecommendationReasons):

    • Verified isolated tests for each explanation template: primary goal reason, fallback workflow reason, related goal category, task type reasons, capability keywords, exact provider, compatible provider, matched terms summary, fast budget preference, capable budget preference, experience guidance preference, and fallback metadata focus.
    • Deduplication and enforcement of the 4-reason cap.
  6. Issue-Specific Known-Defect Characterization:

    • Characterization of two-letter technical acronyms (AI, ML, Go, JS, DB, C#) being dropped by the token.length > 2 filter.
    • Characterization of fallback explanations generating relevance claims without matching evidence on zero-signal recommendations.
    • Note: These tests document existing production behavior for known defects cited in Issue [NSoC 2026] Recommendation scoring has no automated tests #175 pending future maintainer/product decisions, without endorsing them as intended specifications.

Production Code Notice

No production code, dependencies, or build configurations were modified in this PR. This is a strictly tests-only addition at src/lib/agentRecommendation/scoring.test.js.


Verification Results

All checks freshly verified locally:

  • Focused Unit Tests: npx vitest run src/lib/agentRecommendation/scoring.test.js
    Result: 58 passed (58) in 34ms (Duration: 1.69s, exit code 0).
  • Full Repository Suite: npm test (vitest run)
    Result: 25 passed (25) test files, 165 passed (165) tests in 7.35s (exit code 0).
  • Production Build: npm run build (vite build)
    Result: Built successfully in 9.27s (exit code 0).
  • Whitespace / Formatting: git diff --check
    Result: Clean, 0 warnings (exit code 0).

Review Request

Requesting review from the maintainers. Thank you!

@vercel

vercel Bot commented Oct 20, 2026

Copy link
Copy Markdown

@Subodh-17 is attempting to deploy a commit to the DIVYA's projects Team on Vercel.

A member of the Team first needs to authorize it.

@DivyashreeR008
DivyashreeR008 merged commit a0d1ff8 into DivyashreeR008:main Oct 20, 2026
1 check failed
@DivyashreeR008 DivyashreeR008 added nsoc26 NSoC 2026 season level3 NSoC Level 3 - hard (+10) labels Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

level3 NSoC Level 3 - hard (+10) nsoc26 NSoC 2026 season

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[NSoC 2026] Recommendation scoring has no automated tests

2 participants