Skip to content

sessions.json grows unbounded from heartbeat entries — needs rotation / selective persistence #12899

@itsGustav

Description

@itsGustav

Problem

sessions.json grows without bound. Every heartbeat poll creates a full session entry, even when the result is just HEARTBEAT_OK. At 30-minute heartbeat intervals, this adds ~48 entries/day per agent. Over weeks, the file can exceed 50MB, causing downstream issues (e.g., Telegram channel goes unresponsive due to memory/parsing overhead).

Observed

  • sessions.json for a main agent reached 1.6MB in ~2 days
  • A secondary agent with a social cron hit 2.2MB
  • Community reports of files exceeding 50MB with total Telegram outage

Proposed Fix (Layered)

1. Don't Persist HEARTBEAT_OK Sessions (Highest Impact)

~90% of heartbeats produce HEARTBEAT_OK with no user-facing output. These should not create entries in sessions.json or transcript .jsonl files.

// In session finalization
if (isHeartbeatAck(response)) {
  await writeJson(path.join(agentDir, 'heartbeat-state.json'), {
    lastAck: Date.now(),
    consecutiveOKs: state.consecutiveOKs + 1,
  });
  return; // skip session persistence entirely
}

This alone eliminates the majority of the growth.

2. Session Kind Tagging

Tag sessions with a kind field at creation time:

type SessionKind = 'user' | 'heartbeat' | 'cron' | 'subagent';

The persistence layer can then route based on kind:

Kind Persist to sessions.json Transcript .jsonl Notes
user ✅ Full ✅ Full Normal behavior
heartbeat ❌ Skip if OK ❌ Skip if OK Only persist alerts
cron ✅ If output ❌ If silent Same as heartbeat logic
subagent ✅ Full ✅ Full Parent needs results

3. Ring Buffer for Heartbeat History

For observability ('when did heartbeats last run?'), maintain a fixed-size ring buffer instead of unbounded session entries:

// heartbeat-ring.json — fixed-size FIFO
{
  "maxEntries": 50,
  "entries": [
    { "ts": 1770672000, "result": "alert", "summary": "Calendar event in 90min" },
    { "ts": 1770673800, "result": "ok" }
    // oldest entries drop off when maxEntries exceeded
  ]
}

4. File Rotation (Defense in Depth)

As a safety net, add rotation logic regardless of the above:

// On startup or periodically
if (sessionsFile.size > MAX_SIZE) {
  rotate(sessionsFile, `sessions-${isoDate}.json.gz`);
  keepOnly(sessionsFile, mostRecent(100));
}

Priority

  1. Don't persist HEARTBEAT_OK — minimal code change, fixes 90% of the problem
  2. Session kind tagging — clean architecture for routing decisions
  3. Ring buffer — observability without growth
  4. Rotation — safety net for edge cases

Workaround

Until this is fixed upstream, users can set up a recurring cron to trim the file.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions