Skip to content

fix: add retry logic for malformed LLM JSON responses#29

Merged
ddnetters merged 2 commits intomainfrom
fix/malformed-json-retry
Jan 31, 2026
Merged

fix: add retry logic for malformed LLM JSON responses#29
ddnetters merged 2 commits intomainfrom
fix/malformed-json-retry

Conversation

@ddnetters
Copy link
Copy Markdown
Owner

@ddnetters ddnetters commented Jan 31, 2026

Summary

Fixes #28 - LLM occasionally returns malformed JSON for short strings like "Submit" or "Execute Operation".

Changes:

  • Add retry logic (up to 2 retries) when JSON parsing fails
  • Improve system prompt to be stricter about JSON-only output (no markdown code blocks, no explanatory text)
  • Add tests for retry behavior

Test plan

  • Existing tests pass
  • New retry tests verify:
    • Retry succeeds after first malformed response
    • Retries up to MAX_RETRIES (2) times before failing
    • Success on third attempt after two failures

@github-actions
fix: add retry logic for malformed LLM JSON responses
783cf34 
Fixes #28 - LLM occasionally returns malformed JSON for short strings.

Changes:
- Add retry logic (up to 2 retries) when JSON parsing fails
- Improve system prompt to be stricter about JSON-only output
- Add tests for retry behavior
@ddnetters
Copy link
Copy Markdown
Owner Author

ddnetters commented Jan 31, 2026

Code review

Found 1 issue:

  1. Cache poisoning with failed parse results. The retry logic is in performCheck(), but caching happens in check(). When all retries fail with malformed JSON, the error result gets cached. Subsequent calls with the same content return the cached error without retrying, even though the LLM may have recovered.

stringly-typed/src/checkers/brand-style-checker.ts

Lines 35 to 45 in 783cf34

const cacheKey = this.getCacheKey(content, options);
if (options.enableCache !== false && this.cache.has(cacheKey)) {
return this.cache.get(cacheKey)!;
}
try {
const result = await this.performCheck(content, options);
if (options.enableCache !== false) {
this.cache.set(cacheKey, result);
}

Consider not caching error results, or only caching successful validations.

🤖 Generated with Claude Code

- If this code review was useful, please react with 👍. Otherwise, react with 👎.

@github-actions
fix: prevent caching of failed LLM parse results
c9ebf31 
Only cache successful validations to avoid cache poisoning when
LLM returns transient malformed responses. Subsequent calls will
retry instead of returning cached failures.
@ddnetters ddnetters merged commit 0b99746 into main Jan 31, 2026
7 checks passed
@ddnetters ddnetters deleted the fix/malformed-json-retry branch January 31, 2026 19:09
ddnetters pushed a commit that referenced this pull request Jan 31, 2026
@semantic-release-bot
chore: release 1.5.0 [skip ci]
9644d1e 
## [1.5.0](v1.4.14...v1.5.0) (2026-01-31)

### ✨ Features

* **website:** add syntax highlighting to How It Works section ([b32741e](b32741e))
* **website:** update hero with real GitHub Action output ([da39353](da39353))

### 🐛 Bug Fixes

* add retry logic for malformed LLM JSON responses ([#29](#29)) ([0b99746](0b99746)), closes [#28](#28)

### 🎨 Styling

* use inline code blocks for filenames in quickstart steps ([b2528fc](b2528fc))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

LLM occasionally returns malformed JSON for short strings

1 participant

@ddnetters