Skip to content

Commit 4d75d04

Browse files
authored
docs(memory): implement multi-memory chunking for remory token limits (#101)
* docs(memory): implement multi-memory chunking for remory token limits (#100) - Add token limit warnings to /session-start (256 tokens = ~1,000 chars) - Replace single large workflow memory with 7 focused memories - Update memory-and-remory.md with chunking best practices - Add troubleshooting section for embedding failures - Document safe memory size: 1,200-1,500 characters per entry - Explain why --infer false doesn't affect token limits Prevents 'Failed to generate embedding' errors that blocked session initialization when storing comprehensive workflow documentation. * fix(memory): correct character limits to 800-1,000 chars for 256-token limit (#100) Research confirmed: - Upstream API enforces 256-token hard limit - 1:4 character-to-token ratio verified - Previous 1,200-1,500 char guidance = 300-375 tokens (exceeds limit) - Corrected to 800-1,000 chars = 200-250 tokens (safe margin) Evidence: remory embedding.rs disables truncation, passes to OpenAI API which enforces 256 tokens. Math: 1,000 chars ÷ 4 = 250 tokens < 256 ✓ Files modified: - command/session-start.md (30 line changes) - instructions/memory-and-remory.md (32 line changes)
1 parent acb7317 commit 4d75d04

2 files changed

Lines changed: 31 additions & 31 deletions

File tree

‎command/session-start.md‎

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -198,8 +198,8 @@ Return all results in a structured format for memory storage.
198198
- Token ratio: ~1 token per 4 characters (English)
199199

200200
**Safe Memory Size:**
201-
- Target: 1,200-1,500 characters per memory entry
202-
- Token equivalent: 300-375 tokens (safely under limit)
201+
- Target: 800-1,000 characters per memory entry
202+
- Token equivalent: 200-250 tokens (safely under limit with margin)
203203
- Each memory must be semantically complete and independently searchable
204204

205205
**Chunking Strategy:**
@@ -219,34 +219,34 @@ PHASE 4: MEMORY UPDATE (MANDATORY)
219219

220220
**Store project context using remory CLI with --infer false to preserve full detail:**
221221

222-
**Use multi-memory chunking strategy:** Store 5-7 focused memories (1,200-1,500 chars each) instead of single large memory. This prevents "Failed to generate embedding" errors from exceeding token limits.
222+
**Use multi-memory chunking strategy:** Store 5-7 focused memories (800-1,000 chars each) instead of single large memory. This prevents "Failed to generate embedding" errors from exceeding token limits.
223223

224224
```bash
225-
# Memory 1: Project Overview (Keep project basics together, ~1,200 chars max)
225+
# Memory 1: Project Overview (Keep project basics together, ~800-900 chars max)
226226
remory add "Project: {name} - {concise description}. Stack: {technology stack with versions}. Purpose: {main purpose}. Key features: {main features list}. Architecture: {high-level patterns}. Components: {main components}." --user-id "$PROJECT_ID" --infer false
227227

228-
# Memory 2: Current Status (Project activity snapshot, ~1,000 chars max)
228+
# Memory 2: Current Status (Project activity snapshot, ~800-900 chars max)
229229
remory add "Status: {X} open issues, {Y} open PRs. Recent commits: {recent activity from git log with commit messages}. CI/CD: {current CI state}. Active work: {current focus areas}." --user-id "$PROJECT_ID" --infer false
230230

231-
# Memory 3: Commit Conventions (If found in CONTRIBUTING.md or AGENTS.md, ~1,200-1,300 chars)
231+
# Memory 3: Commit Conventions (If found in CONTRIBUTING.md or AGENTS.md, ~800-900 chars)
232232
remory add "Commit standards and conventions. Use Conventional Commits format: type(scope): description. Types: feat, fix, docs, style, refactor, test, chore, build, ci, perf. Include GitHub issue reference: 'feat: add auth (#123)'. Always atomic changes. Clear descriptions. Present tense, imperative mood. First line <50 chars. Examples from project docs: {examples from CONTRIBUTING/AGENTS}." --user-id "$PROJECT_ID" --infer false
233233

234-
# Memory 4: GitHub Issue Workflow (If found in CONTRIBUTING.md or AGENTS.md, ~1,200-1,300 chars)
234+
# Memory 4: GitHub Issue Workflow (If found in CONTRIBUTING.md or AGENTS.md, ~800-900 chars)
235235
remory add "GitHub issue workflow and quality template. Every development issue requirements: {list checkboxes from docs - TDD, coverage, linting, etc.}. Validate issue content with 'gh issue view'. All work matches issue exactly. Refuse work not listed. Update issue before expanding scope. Complete only when all checkboxes done. {Additional workflow details from docs}." --user-id "$PROJECT_ID" --infer false
236236

237-
# Memory 5: PR & Review Process (If found in CONTRIBUTING.md or AGENTS.md, ~1,200-1,300 chars)
237+
# Memory 5: PR & Review Process (If found in CONTRIBUTING.md or AGENTS.md, ~800-900 chars)
238238
remory add "Pull request and CI verification process. Feature branch workflow: {branch naming conventions from docs}. PR requirements: {title format, description requirements}. Review process: {approval requirements, self-merge policy}. CI verification: {commands used - gh run watch, monitoring requirements}. Merge strategy: {squash/merge/rebase from docs}." --user-id "$PROJECT_ID" --infer false
239239

240-
# Memory 6: Quality Gates & Testing (If found in CONTRIBUTING.md or AGENTS.md, ~1,200-1,300 chars)
240+
# Memory 6: Quality Gates & Testing (If found in CONTRIBUTING.md or AGENTS.md, ~800-900 chars)
241241
remory add "Quality gates and testing requirements. Zero-tolerance policy: {suppression rules from docs}. Coverage requirements: {thresholds - 80%+, per-file vs overall}. Linting: {commands and requirements}. Type checking: {requirements}. Testing approach: {TDD requirements, test patterns, mocking strategies from docs}. Local verification: {what must pass before completion}." --user-id "$PROJECT_ID" --infer false
242242

243-
# Memory 7: Communication & Philosophy (If found in CONTRIBUTING.md or AGENTS.md, ~1,200-1,300 chars)
243+
# Memory 7: Communication & Philosophy (If found in CONTRIBUTING.md or AGENTS.md, ~800-900 chars)
244244
remory add "Communication style and development philosophy. Tone: {professional, concise, etc. from docs}. Minimalist principles: {LESS IS MORE, challenge necessity, etc. from docs}. Code quality: {standards from docs}. Documentation: {when to create, approval requirements}." --user-id "$PROJECT_ID" --infer false
245245
```
246246

247247
**CRITICAL:** Use `--infer false` to preserve FULL TEXT without LLM extraction/shortening.
248248

249-
**Store comprehensive context using MULTIPLE focused memories (1,200-1,500 chars each):**
249+
**Store comprehensive context using MULTIPLE focused memories (800-1,000 chars each):**
250250

251251
**Core Project Info (2 memories):**
252252
- Memory 1: Project overview - name, description, stack, purpose, architecture, components
@@ -260,14 +260,14 @@ remory add "Communication style and development philosophy. Tone: {professional,
260260
- Memory 7: Communication & philosophy - tone, minimalist principles, standards
261261

262262
**Chunking Guidelines:**
263-
- Each memory: 1,200-1,500 characters (safely under 256-token embedding limit)
263+
- Each memory: 800-1,000 characters (safely under 256-token embedding limit with margin)
264264
- Semantically complete: Each memory understandable independently
265265
- Topic-focused: One coherent concept per memory
266266
- Searchable: Use clear topic names for future retrieval
267267
- Natural boundaries: Split at logical sections, not mid-paragraph
268268

269269
**Why Chunking:**
270-
Remory's embedding model (sentence-transformers/all-MiniLM-L6-v2) has 256-token default limit (~1,000 chars). Long texts (2,500+ chars) fail with "Failed to generate embedding" errors. Multiple focused memories succeed where single large memory fails.
270+
Remory's embedding model (sentence-transformers/all-MiniLM-L6-v2) enforces 256-token default limit (200-250 tokens safe with margin = 800-1,000 chars). Long texts (2,500+ chars = 600+ tokens) fail with "Failed to generate embedding" errors. Multiple focused memories succeed where single large memory fails.
271271

272272
**Why this matters:**
273273
- ✅ Each memory successfully embeds (under token limits)
@@ -333,7 +333,7 @@ CRITICAL RULES
333333
- Search memory FIRST with specific semantic queries
334334
- Store ALL gathered context with `--infer false` to preserve full detail
335335
- Use detailed, comprehensive descriptions (not summaries)
336-
- Split large memories into focused chunks (1,200-1,500 chars each)
336+
- Split large memories into focused chunks (800-1,000 chars each)
337337
- Verify each memory is semantically complete and independently searchable
338338
- Use natural topic boundaries (commit conventions, PR process, quality gates)
339339
- Update TodoWrite after each phase
@@ -349,7 +349,7 @@ CRITICAL RULES
349349
- Assume documentation structure - adapt to what the project actually has
350350
- Skip files that don't exist - read what's there, not what you expect
351351
- Skip memory storage - it's MANDATORY for session continuity
352-
- Store memories exceeding 1,500 characters (risk embedding failure)
352+
- Store memories exceeding 1,000 characters (risk embedding failure)
353353
- Create single large "Project workflow" memory (will fail at 2,500+ chars)
354354
- Split mid-paragraph or mid-sentence (breaks semantic coherence)
355355

‎instructions/memory-and-remory.md‎

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ remory list --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-
102102
- Integration patterns with other components
103103

104104
**Length Considerations:**
105-
If your memory content exceeds 1,500 characters, split into multiple focused memories:
105+
If your memory content exceeds 1,000 characters, split into multiple focused memories:
106106
- One memory for architecture decisions
107107
- One memory for solutions to specific problem types
108108
- One memory for testing/linting configuration
@@ -136,17 +136,17 @@ The `--infer false` flag disables auto-consolidation. When enabled, Remory's LLM
136136
- **Token-to-character ratio:** Approximately 1 token per 4 characters (English)
137137

138138
**Safe Memory Size Guidelines:**
139-
- **Target:** 1,200-1,500 characters per `remory add` command
140-
- **Token equivalent:** 300-375 tokens (safely under 256-token default with margin)
141-
- **Maximum:** Do NOT exceed 1,500 characters in a single memory entry
139+
- **Target:** 800-1,000 characters per `remory add` command
140+
- **Token equivalent:** 200-250 tokens (safely under 256-token limit with margin)
141+
- **Maximum:** Do NOT exceed 1,000 characters in a single memory entry
142142
- **Consequence of violation:** "Failed to generate embedding" error
143143

144144
**Why `--infer false` Doesn't Help:**
145145
The `--infer false` flag disables LLM-powered consolidation (happens AFTER embedding). Token limits are enforced DURING embedding (BEFORE consolidation). If embedding fails due to length, consolidation never runs.
146146

147147
**Chunking Strategy (Required for Long Content):**
148148

149-
When storing comprehensive information that exceeds 1,500 characters:
149+
When storing comprehensive information that exceeds 1,000 characters:
150150

151151
1. **Split into focused topic-based memories** instead of one large memory
152152
2. **Each memory must be semantically complete** (independently understandable)
@@ -161,13 +161,13 @@ remory add "Project workflow: commit conventions, PR process, quality gates, tes
161161

162162
**Example - CORRECT (Will Succeed):**
163163
```bash
164-
# Memory 1: Commit conventions (~1,200 chars)
164+
# Memory 1: Commit conventions (~800-900 chars)
165165
remory add "Commit standards and conventions. Use Conventional Commits format: type(scope): description. Types: feat, fix, docs, refactor, test. Include issue reference. Examples: 'feat: add auth (#123)'. Present tense, imperative mood. First line <50 chars." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
166166

167-
# Memory 2: PR process (~1,300 chars)
167+
# Memory 2: PR process (~800-900 chars)
168168
remory add "Pull request workflow. Feature branch naming: feature/123-description. PR requirements: title format, description, CI green. Review process: mandatory approval, no self-merge. Merge strategy: squash. CI monitoring: gh run watch for real-time." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
169169

170-
# Memory 3: Quality gates (~1,200 chars)
170+
# Memory 3: Quality gates (~800-900 chars)
171171
remory add "Zero-tolerance quality enforcement. Never suppress linting (no # noqa, # type: ignore). Coverage: 80%+ per file, CI enforces. Testing: TDD required, tests before implementation. Local verification: run all tests, all linting, type checking before completion." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
172172
```
173173

@@ -227,10 +227,10 @@ Leverage Remory's advanced features:
227227

228228
**⚠️ Length Management:**
229229
When storing delegation history or multi-agent coordination patterns:
230-
- If content exceeds 1,500 characters, split into multiple memories
230+
- If content exceeds 1,000 characters, split into multiple memories
231231
- Example: One memory for Phase 1 delegation patterns, another for Phase 2
232232
- Each memory should cover a coherent delegation scenario or pattern
233-
- Never create single memory exceeding 1,500 characters
233+
- Never create single memory exceeding 1,000 characters
234234

235235
### Specialist Agents (All Subagents)
236236

@@ -292,24 +292,24 @@ remory add "..." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .pro
292292
**Solution: Chunk Your Memory**
293293

294294
1. **Measure your content:** Count characters (not words)
295-
2. **If > 1,500 characters:** Split into multiple memories
295+
2. **If > 1,000 characters:** Split into multiple memories
296296
3. **Use topic-based splitting:**
297297
- Commit conventions
298298
- PR workflow
299299
- Quality gates
300300
- Testing requirements
301-
4. **Each chunk:** 1,200-1,500 characters, semantically complete
301+
4. **Each chunk:** 800-1,000 characters, semantically complete
302302
5. **Test:** Run `remory add` for each chunk separately
303303

304304
**Example Split:**
305305
```bash
306306
# BEFORE (fails at 2,500 chars)
307307
remory add "Huge text block..." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
308308

309-
# AFTER (succeeds - 3 chunks at ~1,200 chars each)
310-
remory add "Commit conventions: [1,200 chars]..." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
311-
remory add "PR workflow: [1,200 chars]..." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
312-
remory add "Quality gates: [1,100 chars]..." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
309+
# AFTER (succeeds - 3 chunks at ~800-900 chars each)
310+
remory add "Commit conventions: [800-900 chars]..." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
311+
remory add "PR workflow: [800-900 chars]..." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
312+
remory add "Quality gates: [800-900 chars]..." --user-id "$(cat .project-id 2>/dev/null || (uuidgen | tee .project-id))" --infer false
313313
```
314314

315315
**Verification:**

0 commit comments

Comments
 (0)