[Bug]: /resume extreme RAM spike — unindexed conversations trigger full gen_metadata Protobuf BLOB deserialization #727

/resume causes extreme RAM spike (10GB+) before entering any conversation — triggered by AGY's fallback mechanism loading all unindexed conversation SQLite files including bloated gen_metadata Protobuf BLOBs into RAM.

Platform: AGY CLI (Windows)
Version: 1.1.9

Steps to Reproduce

1. Use AGY with view_file on media/binary files, or run long conversations with heavy tool calls, across several subagent sessions
2. Allow a number of conversation .db files to accumulate in brain/ over time (especially ones without entries in conversation_summaries.db)
3. Start a fresh AGY session and run /resume
4. Observe: RAM spikes to several GB before any conversation is selected or entered

Root Cause Analysis

Two compounding issues cause this:

1. Missing Conversation Summaries Index

When /resume is invoked, AGY reads conversation_summaries.db. If any past conversation has no index entry in that database, AGY's fallback mechanism opens every raw .db file under conversations/ and loads it into RAM for parsing.

In the affected installation: 99 total conversation files, 59 unindexed, with those 59 totalling 4.41 GB on disk.

2. Bloated gen_metadata Protobuf BLOBs (Snowballing Size)

For conversations involving view_file on media files or large tool call payloads, AGY serializes the complete Generation Metadata (protobuf object) into a BLOB in the gen_metadata SQLite table after every turn. Critically, each turn stores the cumulative context snapshot up to that point:

• Turn 1: ~50 MB BLOB
• Turn 2: ~150 MB BLOB
• Turn 4–5: single SQLite column BLOB reaches 740 MB

When /resume hits an unindexed conversation, SQLite loads the entire BLOB chain into RAM and attempts full deserialization. With multiple such conversations, this consumes 10 GB+ RAM instantly — before any conversation is even opened.

Expected Behavior

• /resume should never require loading full blob data to display the conversation list
• Missing conversation_summaries.db entries should be gracefully skipped or lazy-loaded — not bulk-loaded
• conversation_summaries.db should be automatically maintained as sessions complete, so the fallback path is never triggered under normal use

Workaround Applied (Self-diagnosed and fixed)

Manually applied a two-step fix in a separate clean session:

Step 1 — Rebuilt conversation_summaries.db index:
Parsed all 59 unindexed conversations and inserted their summary records. /resume now only reads the ~50 KB index file and responds in milliseconds.

Step 2 — Cleaned bloated gen_metadata BLOBs + SQLite VACUUM:
Removed oversized generation snapshot BLOBs from completed conversations and ran VACUUM to reclaim disk space.

Prevention Recommendation

The gen_metadata cumulative snapshot design combined with the unindexed-conversation fallback creates a ticking time bomb for any user who:

• Uses view_file on media or binary files
• Runs long conversations with many tool calls
• Accumulates conversation history over time

Suggested fixes:

1. Auto-maintain conversation_summaries.db on session end — eliminate the fallback entirely
2. Cap or prune gen_metadata BLOBs per conversation (e.g., only keep last N snapshots)
3. Store large binary payloads as separate files with references in SQLite, rather than inline BLOBs