Improve user story description parsing for multiple LLM formats#200
Improve user story description parsing for multiple LLM formats#200
Conversation
The parser's extractStoryDescription() previously stopped at any line starting with '**', which broke when LLMs generated descriptions with **Description:** prefixes or **As a**/**I want**/**So that** bold keywords. Now the parser only stops at known metadata fields (Priority, Depends on, etc.) and strips bold formatting from description content. Also updates the LLM prompt guidance to prefer plain-text descriptions, reducing the likelihood of format mismatches in newly generated PRDs.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. WalkthroughThe changes update PRD parsing logic to enforce plain text descriptions without "Description:" prefixes, introducing new validation patterns and stopping conditions in the parser whilst updating guidance documentation and adding comprehensive test coverage. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #200 +/- ##
==========================================
+ Coverage 43.87% 44.23% +0.36%
==========================================
Files 83 84 +1
Lines 23963 24229 +266
==========================================
+ Hits 10514 10718 +204
- Misses 13449 13511 +62
🚀 New features to boost your workflow:
|
…ance Covers extractStoryDescription with multiple LLM output formats: - Plain text descriptions - **Description:** prefixed descriptions - **As a**/**I want**/**So that** bold-keyword format - Stop conditions (metadata fields, headings, separators) - Edge cases (empty descriptions, no acceptance criteria) Also tests buildPrdSystemPromptFromSkillSource to verify the prompt instructs plain-text format and discourages the **Description:** prefix.
These test directories were missing from CI, so Codecov had no coverage data for parser.ts and engine.ts changes, resulting in 16% diff coverage.
…mat-wMTfi Improve user story description parsing for multiple LLM formats
Summary
Enhanced the PRD parser to correctly handle user story descriptions in multiple LLM output formats, and updated the PRD generation guidance to specify plain text descriptions without bold prefixes.
Changes
PRD Generation Guidance (
src/chat/engine.ts)PRD_COMPATIBILITY_GUIDANCEto clarify that user story descriptions should be plain text on the next line following the format: "As a user, I want to ... so that ..."**Description:**bold prefix from the example formatDescription Extraction Logic (
src/prd/parser.ts)DESCRIPTION_STOP_PATTERNregex to identify known metadata fields (**Acceptance Criteria:**,**Priority:**,**Depends on:**,**Labels:**,**Notes:**) that should terminate description extractionDESCRIPTION_LABEL_PATTERNregex to detect and strip the**Description:**label prefix that some LLMs generateextractStoryDescription()function to:line.startsWith('**')check with the more preciseDESCRIPTION_STOP_PATTERNmatch to avoid false positives**Description:**label prefix from lines before processing**As a**→As a) from the final description textImplementation Details
The parser now gracefully handles inconsistent LLM formatting by:
**Description:**label if presentThis makes the parser more robust to variations in LLM output while encouraging the generation of properly formatted descriptions going forward.
Summary by CodeRabbit
Release Notes
Improvements
Tests
✏️ Tip: You can customize this high-level summary in your review settings.