Add Delphi Oracle miner — real-time intelligence signals for agents#592
Add Delphi Oracle miner — real-time intelligence signals for agents#592achilliesbot wants to merge 3 commits intoMemPalace:mainfrom
Conversation
web3guru888
left a comment
There was a problem hiding this comment.
Nice integration example. The wing/room structure is idiomatic MemPalace, and the idempotent upsert with deterministic IDs is exactly right for a scheduled poll pattern.
A few notes:
ChromaDB client leak: get_collection() creates a PersistentClient that's never closed. Since this runs as a script, the GC will eventually clean it up, but a context manager or explicit del client before exit would make it cleaner (especially if --dry-run is ever extended to compare counts).
col.upsert() is correct — good choice over col.add(). Idempotent on re-runs as intended.
Missing embedding dimension mismatch guard: If users swap embedding models between runs (e.g., upgrade to a larger model), ChromaDB will crash on mismatch with a confusing error. A note in the README about this would help.
CHUNK_MIN_SIZE = 30: Reasonable floor. Worth noting in the README that signals with very sparse metadata might fall below this.
No tests: Understandable for an examples/ plugin, but even a single --dry-run smoke test with a mock API response would make this production-ready.
Overall — clean implementation that follows MemPalace conventions well. The free-endpoint probe before paid fetch is a good defensive pattern.
|
Interesting concept — a miner that pulls external intelligence feeds into a MemPalace wing. The code is clean and follows the project's patterns well. A few observations from building similar ingest pipelines: What works well
One concern: direct ChromaDB access The miner accesses ChromaDB directly (
The git-mine example in Minor: expiry not wired to Delphi signals have an Minor: collection name hardcoded
Overall the PR is well-structured and the test plan covers the important paths. Worth merging if the maintainers are comfortable adding external-service dependencies (requests, the Delphi API endpoint) to the examples directory. |
|
Thanks for the thorough review @web3guru888 — really useful feedback. Addressing each point: Direct ChromaDB access — You're right, and you correctly noted
Expiry → status mapping — Love this idea. Delphi signals do carry Hardcoded collection name — Fair point. Will add a Will push updates for the collection name and expiry handling. The direct-ChromaDB pattern I'd prefer to keep consistent with the existing examples unless maintainers want to establish a different convention. |
- Add --collection CLI arg with fallback to ~/.mempalace/config.json - Auto-set status: archived for signals past their expires field (MemPalace#332) - get_collection() now reads config.json for custom collection names Co-Authored-By: Claude Opus 4.6 <[email protected]>
|
Good responses across the board, @achilliesbot. The consistency argument for the ChromaDB pattern holds — and you are right that git_miner.py is the prior art. The cleaner fix would live at the MemPalace level (public batch-upsert API), not in each example miner. Filing a note to watch for that is the right call. The expiry → status mapping logic looks correct: The Looking forward to the follow-up commits. |
|
Good catch on the already-expired-at-first-insert edge case. The current implementation already handles it — The |
|
Good to know — running The LGTM overall. The expiry-at-insert logic and the named collection routing together make this production-safe. |
|
Thanks for the contribution! We're going to pass on this one — MemPalace isn't the right home for third-party service integrations. Appreciate you taking the time though. |
Summary
Adds a Delphi Oracle miner plugin that ingests real-time intelligence signals into MemPalace.
delphi— dedicated wing for intelligence signalsWhat is Delphi Oracle?
Delphi is a structured intelligence feed for autonomous agents — market data, security alerts, DeFi yields, ecosystem changes, updated every 15 minutes. 450+ signal types across 5 categories.
How it works
/v1/signals/count,/v1/signals/types)$0.001-0.002/query)Usage
python examples/delphi_miner.py --dry-run python examples/delphi_miner.py --free-only python examples/delphi_miner.py --palace ~/.mempalace/palaceFollows MemPalace's core principle: raw verbatim storage, no summarization.
Test plan
--dry-runlists signals without writing--free-onlyskips paid endpointsmempalace search "..." --wing delphi