-
-
Notifications
You must be signed in to change notification settings - Fork 68.9k
sessions.json grows unbounded from heartbeat entries — needs rotation / selective persistence #12899
Description
Problem
sessions.json grows without bound. Every heartbeat poll creates a full session entry, even when the result is just HEARTBEAT_OK. At 30-minute heartbeat intervals, this adds ~48 entries/day per agent. Over weeks, the file can exceed 50MB, causing downstream issues (e.g., Telegram channel goes unresponsive due to memory/parsing overhead).
Observed
sessions.jsonfor a main agent reached 1.6MB in ~2 days- A secondary agent with a social cron hit 2.2MB
- Community reports of files exceeding 50MB with total Telegram outage
Proposed Fix (Layered)
1. Don't Persist HEARTBEAT_OK Sessions (Highest Impact)
~90% of heartbeats produce HEARTBEAT_OK with no user-facing output. These should not create entries in sessions.json or transcript .jsonl files.
// In session finalization
if (isHeartbeatAck(response)) {
await writeJson(path.join(agentDir, 'heartbeat-state.json'), {
lastAck: Date.now(),
consecutiveOKs: state.consecutiveOKs + 1,
});
return; // skip session persistence entirely
}This alone eliminates the majority of the growth.
2. Session Kind Tagging
Tag sessions with a kind field at creation time:
type SessionKind = 'user' | 'heartbeat' | 'cron' | 'subagent';The persistence layer can then route based on kind:
| Kind | Persist to sessions.json | Transcript .jsonl | Notes |
|---|---|---|---|
user |
✅ Full | ✅ Full | Normal behavior |
heartbeat |
❌ Skip if OK | ❌ Skip if OK | Only persist alerts |
cron |
✅ If output | ❌ If silent | Same as heartbeat logic |
subagent |
✅ Full | ✅ Full | Parent needs results |
3. Ring Buffer for Heartbeat History
For observability ('when did heartbeats last run?'), maintain a fixed-size ring buffer instead of unbounded session entries:
// heartbeat-ring.json — fixed-size FIFO
{
"maxEntries": 50,
"entries": [
{ "ts": 1770672000, "result": "alert", "summary": "Calendar event in 90min" },
{ "ts": 1770673800, "result": "ok" }
// oldest entries drop off when maxEntries exceeded
]
}4. File Rotation (Defense in Depth)
As a safety net, add rotation logic regardless of the above:
// On startup or periodically
if (sessionsFile.size > MAX_SIZE) {
rotate(sessionsFile, `sessions-${isoDate}.json.gz`);
keepOnly(sessionsFile, mostRecent(100));
}Priority
- Don't persist HEARTBEAT_OK — minimal code change, fixes 90% of the problem
- Session kind tagging — clean architecture for routing decisions
- Ring buffer — observability without growth
- Rotation — safety net for edge cases
Workaround
Until this is fixed upstream, users can set up a recurring cron to trim the file.