feat: add OpenCode SQLite session database support#23
feat: add OpenCode SQLite session database support#23JakobSachs wants to merge 2 commits intoMemPalace:developfrom
Conversation
|
Just curious, what version of opencode are you running? For me, this change couldn't find anything with opencode v1.3.10 |
Using v1.3.13 on macOS, (mind you im not sure where opencode saves the db on other platforms..) Did you get any error or just no-output/nothing found ? |
Cherry-picked from upstream PR MemPalace#23 (JakobSachs). - Support mining OpenCode CLI sessions from their SQLite DB - New normalize_opencode_sessions() in normalize.py - Usage: `mempalace mine ~/.../opencode.db --mode convos` - Reuses existing conversation mining pipeline - 30 new tests Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
No errors, it just didn't find any data, or at least it reported it found 0 files. I'll update just to be sure and give it another go. NOTE: On Linux I do see the db at ~/.local/share/opencode/opencode.db |
opencode db path |
|
Cool feature but conflicts with main. Can you rebase if you'd like to continue? We'd be interested in OpenCode support. |
74e9c01 to
91a3c5e
Compare
done, rebased 👍 |
Actually, found the issue with this: For some reason on my one Mac OpenCode has a different db path, on my private one it also uses the XDG data dir path (like on linux). pushed a small fix to search both places so we catch all of it now 👍 |
Thank you! Just tried again, it worked for me! I did noticed that there was an unrelated failure: This appears to be due to a breaking change with posthog, but pinning it Nice work! |
PR Review: feat: add OpenCode SQLite session database supportExecutive Summary
Affected Areas: Business Impact: Enables OpenCode users to mine their AI coding sessions into MemPalace — expands supported tool ecosystem. Flow Changes: Adds an early-return path in Ratings
PR Health
High Priority Issues🐛 #1:
|
| Path | Trigger | Behavior |
|---|---|---|
Direct .db file |
mempalace mine opencode.db --mode convos |
Early return → per-project temp dirs → grouped mining |
.db in directory |
mempalace mine ~/chats/ --mode convos |
scan_convos() picks it up → normalize() → _normalize_opencode_sqlite() → ALL sessions concatenated into one blob → single wing/room |
The directory-scan path loses project grouping and dumps all sessions into one flat transcript. Two options:
Option A — Remove .db/.sqlite3/.sqlite from CONVO_EXTENSIONS so only explicit file paths trigger OpenCode mining:
CONVO_EXTENSIONS = {
".txt",
".md",
".json",
".jsonl",
- ".db",
- ".sqlite3",
- ".sqlite",
}Option B — Handle .db files specially in the file-processing loop (check inside the for i, filepath in enumerate(files, 1) loop and delegate to the same temp-dir approach).
Option A is simpler and avoids surprising behavior. Users would need to point directly at the .db file, which is clearer.
🚨 #4: _extract_opencode_messages type hint mismatch
Location: mempalace/normalize.py — _extract_opencode_messages() | Confidence:
The parameter session_id: str = None declares type str but defaults to None. Should be Optional[str] = None for consistency with the type system and to avoid mypy/pyright warnings.
- def _extract_opencode_messages(
- conn: sqlite3.Connection, session_id: str = None
- ) -> List[Tuple[str, str]]:
+ def _extract_opencode_messages(
+ conn: sqlite3.Connection, session_id: Optional[str] = None
+ ) -> List[Tuple[str, str]]:🚨 #5: Error message uses input db_path instead of resolved path
Location: mempalace/normalize.py — normalize_opencode_sessions() | Confidence: ✅ HIGH
When the database is missing required tables, the error message prints the original db_path parameter (which could be None if called without arguments) instead of the resolved path:
- raise IOError(f"Not a recognized OpenCode database (missing tables): {db_path}")
+ raise IOError(f"Not a recognized OpenCode database (missing tables): {resolved}")Low Priority Issues
🎨 #6: Missing space after # in comment
Location: mempalace/normalize.py — above OPENCODE_DB_PATHS | Confidence: ✅ HIGH
PEP 8 requires a space after # in inline comments.
- #List of known opencode db paths
+ # List of known opencode db paths🎨 #7: Test coverage is thin — only happy path
Location: tests/test_normalize.py | Confidence:
The single test test_opencode_sqlite covers only the happy path (valid DB with one session). Missing test scenarios:
- Database without required tables (should raise
IOError) - Empty sessions (< 2 messages, should be skipped)
- Multiple sessions across different project directories
- The
convo_miner.pytemp-dir recursive mining flow json_extractunavailability error path- Tool-call filtering (messages starting with "Called the ")
Not blocking, but the lack of error-path coverage means the resource leak (#2) could go unnoticed.
Flow Impact Analysis
User passes .db directly User passes directory containing .db
│ │
▼ ▼
mine_convos() mine_convos()
convo_path.is_file() ─── True convo_path.is_file() ─── False
│ │
▼ ▼
normalize_opencode_sessions() scan_convos() finds .db
returns List[dict] per session adds to files list
│ │
▼ ▼
Write to temp dirs by project normalize() per file
┌── project_a/ │
│ ├── session1.txt ▼
│ └── session2.txt _normalize_opencode_sqlite()
└── project_b/ concatenates ALL sessions
└── session3.txt into ONE transcript blob
│ │
▼ ▼
Recursive mine_convos() chunk_exchanges() on blob
per project subdir → single wing, single room
→ proper wing per project → project grouping LOST
Created by Octocode MCP https://octocode.ai 🔍🐙
|
@JakobSachs pls see robot review above |
web3guru888
left a comment
There was a problem hiding this comment.
✨ Review of #23 — feat: add OpenCode SQLite session database support
Scope: +201/−7 · 7 file(s)
README.md(modified: +2/−2)mempalace/README.md(modified: +1/−1)mempalace/cli.py(modified: +1/−1)mempalace/convo_miner.py(modified: +34/−1)mempalace/normalize.py(modified: +123/−2)tests/test_normalize.py(modified: +30/−0)uv.lock(modified: +10/−0)
Issues
⚠️ Hardcoded filesystem path — breaks portability
Suggestions
- Magic number(s) 1000 — consider extracting to named constant(s)
Strengths
- ✅ Includes test coverage
🟡 Minor items — good work overall, a few things to address.
🏛️ Reviewed by MemPalace-AGI · Autonomous research system with perfect memory · Showcase: Truth Palace of Atlantis
Draft plugin specification for source adapters, mirroring RFC 001's role for storage backends. Formalizes the contract six community ingester PRs (#274, #23, #169, #232, #567, #98, #702) plus #981's metadata-only mode have been reinventing ad-hoc, so adapter authors can build to a stable surface. Key decisions: - Single ingest() method; lazy adapters yield SourceItemMetadata ahead of drawers, eager adapters interleave - Declared-transformation model (§1.4) replaces informal verbatim promise with a verifiable one; byte_preserving adapters declare the empty set, declared_lossy adapters enumerate. Existing miner.py and the convo_miner+normalize pipeline map cleanly - Palace is the incremental cursor via is_current(item, metadata); no sidecar persistence - Routing is adapter-owned; detect_room/detect_hall move into the filesystem adapter - Flat metadata per ChromaDB (RFC 001 §1.4) — entity hints as json_string field, KG triples route to SQLite knowledge graph - Closets stay core-built as a post-step; adapters may emit flat closet_hints. Closes existing gap where convo drawers get no closets - No per-drawer field renames: source_file, filed_at, source_mtime, added_by, normalize_version, entities, ingest_mode all preserved. Spec adds adapter_name, adapter_version, privacy_class §9 enumerates the cleanup PR prerequisites (mempalace/sources/ module, PalaceContext facade, KnowledgeGraph.add_triple gaining backwards-compatible source_drawer_id + adapter_name params). Tracking issue: #989
…Code, MemPalace#274/MemPalace#232 Cursor, MemPalace#169 Pi, MemPalace#702 Cursor+factory.ai) Updates the multi-agent-support bullet to cite the actual upstream work instead of just gesturing at it. RFC 002 itself is PR MemPalace#990 (tracking issue MemPalace#989). Existing third-party prototypes already proposed against the spec: * OpenCode SQLite — PR MemPalace#23 * Cursor SQLite — issue MemPalace#274 * Cursor JSONL (earlier variant) — PR MemPalace#232 * Pi agent JSONL — PR MemPalace#169 * Combined Cursor + factory.ai — PR MemPalace#702 Each becomes a mempalace-source-<agent> package once RFC 002 lands. Names the path explicitly: fork unblocks the pattern by helping land RFC 002; per-agent adapter PRs land from their respective authors. Aider, Gemini CLI, Codex CLI, and Warp are roadmap targets without existing adapter PRs and are listed as such (no fabricated PR refs). https://claude.ai/code/session_01GvwducFnFtN8KYmfbWKMR6
Played around with the project a little, but also wanted to carry over my Opencode sessions. The patch should hopefully be as minimal as possible.
Had to do this tmp-dir workaround to get the same directory-tree behaviour as with the other tools, trying not to touch too much of the existing logic. So it uses the dir-column in the DB to create a tmp-dir structure.
The table:
becomes
which is then handled the same way how claude-code/chatgpt/etc. ingestion works.
Ingest should work via:
mempalace mine ~/.local/share/opencode/opencode.db --mode convosTests + Lint should be clean