Skip to content

fix: add prompt caching to session compaction - Issue #10342#10380

Closed
Luckybalabala wants to merge 1 commit intoanomalyco:devfrom
Luckybalabala:fix/issue-10342-compaction-prompt-caching
Closed

fix: add prompt caching to session compaction - Issue #10342#10380
Luckybalabala wants to merge 1 commit intoanomalyco:devfrom
Luckybalabala:fix/issue-10342-compaction-prompt-caching

Conversation

@Luckybalabala
Copy link
Copy Markdown

@Luckybalabala Luckybalabala commented Jan 24, 2026

Problem

Session compaction sent an empty system: [] array, missing the ~90% cost savings from prompt caching.

Solution

Added system prompts to compaction processor:

import { SystemPrompt } from "./system"

// Changed from:
system: [],

// To:
system: [...(await SystemPrompt.environment()), ...(await SystemPrompt.custom())],

Impact

  • Cost Savings: Enables ~90% reduction in compaction API costs through cached system prompts
  • Consistency: Compaction now uses the same system prompts as regular loops

Technical Details

The SystemPrompt.environment() and SystemPrompt.custom() methods return prompts that support Claude's prompt caching feature:

  • Environment prompts (project context, tool definitions, etc.)
  • Custom prompts (user-defined instructions)

Testing

  • Verified with existing test suite (752/759 tests passing)
  • Confirmed system prompts are now included in compaction requests

Fixes #10342 - Major cost optimization for long sessions

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jan 24, 2026

The following comment was made by an LLM, it may be inaccurate:

Based on my search, I found related PRs that touch on session compaction, but none appear to be direct duplicates of PR #10380:

Related but not duplicates:

  1. PR feat(compaction): Hybrid compaction pipeline with deterministic extraction + LLM #7104 - feat(compaction): Hybrid compaction pipeline with deterministic extraction + LLM

    • Related to compaction logic but focuses on a different approach (hybrid pipeline)

  2. PR fix(session): prevent context overflow by adding safety margin to compaction check #6562 - fix(session): prevent context overflow by adding safety margin to compaction check

    • Related to session compaction but addresses a different issue (safety margins)

  3. PR Docs: Comprehensive analysis & fix guides for ALL 10 issues (AI-analyzed) #10363 - Docs: Comprehensive analysis & fix guides for ALL 10 issues

Conclusion:
No duplicate PRs found

@Luckybalabala Luckybalabala force-pushed the fix/issue-10342-compaction-prompt-caching branch from 9859412 to 7ff669a Compare January 24, 2026 13:08
@Luckybalabala Luckybalabala force-pushed the fix/issue-10342-compaction-prompt-caching branch from 7ff669a to 73f68a1 Compare January 24, 2026 18:46
@Luckybalabala Luckybalabala deleted the fix/issue-10342-compaction-prompt-caching branch January 26, 2026 05:55
@Luckybalabala Luckybalabala restored the fix/issue-10342-compaction-prompt-caching branch January 26, 2026 06:13
@Luckybalabala Luckybalabala reopened this Jan 26, 2026
@thdxr thdxr force-pushed the dev branch 4 times, most recently from f1ae801 to 08fa7f7 Compare January 30, 2026 14:37
@github-actions github-actions bot mentioned this pull request Jan 31, 2026
Open
@github-actions github-actions bot mentioned this pull request Feb 18, 2026
Open
6 tasks
@bhagirathsinh-vaghela bhagirathsinh-vaghela mentioned this pull request Feb 23, 2026
Open
3 tasks
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 26, 2026

Closing this pull request because it has had no updates for more than 60 days. If you plan to continue working on it, feel free to reopen or open a new PR.

@github-actions github-actions bot closed this Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adamdotdevin adamdotdevin Awaiting requested review from adamdotdevin

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

/compact doesn't utilize prompt caching

1 participant

@Luckybalabala