SurrealDB + GPU Embeddings: Local Memory for AI Agents

A prototype showing how to build a local semantic memory system using SurrealDB (vector + graph + full-text search) and Qwen3-Embedding-4B on a consumer GPU.

Stack: SurrealDB 2.6.1 · Qwen3-Embedding-4B · PyTorch · FastAPI · RocksDB

No API keys. No cloud. Everything runs on your machine.

What's Here

File	Purpose
schema.surql	SurrealDB table definitions, indexes, and analyzers
seed.example.surql	Example seed data (concepts, memories, graph edges)
embed.py	Batch embed memory entries into SurrealDB using GPU
search.py	Semantic search CLI over embedded memories
embedding-server.py	OpenAI-compatible /v1/embeddings FastAPI server
requirements.txt	Pinned Python dependencies

Quick Start

1. Install SurrealDB

curl -sSf https://install.surrealdb.com | sh

2. Start the server

surreal start \
  --bind 127.0.0.1:8000 \
  --user root --pass root \
  --log warn \
  "rocksdb://./data"

3. Import schema and seed data

surreal import \
  --endpoint http://127.0.0.1:8000 \
  --username root --password root \
  --namespace agent --database memory \
  schema.surql

surreal import \
  --endpoint http://127.0.0.1:8000 \
  --username root --password root \
  --namespace agent --database memory \
  seed.example.surql

4. Set up Python environment

python3 -m venv .venv
source .venv/bin/activate

# Install PyTorch with CUDA support
pip install torch --index-url https://download.pytorch.org/whl/cu128

# Install remaining deps
pip install -r requirements.txt

5. Embed memories

python3 embed.py        # embed entries missing vectors
python3 embed.py --all  # re-embed everything

6. Search

python3 search.py "debugging network issues"
python3 search.py "project decisions" -k 3

Embedding Server

A local FastAPI server that exposes an OpenAI-compatible embedding endpoint, backed by the GPU model. Any tool expecting the OpenAI API can point at this instead.

python3 embedding-server.py
# http://127.0.0.1:8678

# Test it
curl http://127.0.0.1:8678/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"input": "hello world", "model": "text-embedding-ada-002"}'

Schema Design

Tables

Table	Type	Purpose
memory	Document	Memory entries with text, tags, timestamps, and vector embeddings
concept	Document	Reusable concept nodes for graph linking
references	Relation (memory → concept)	Links memories to concepts they mention
related_to	Relation (concept → concept)	Links related concepts
follows	Relation (memory → memory)	Temporal/causal ordering

Indexes

• BM25 full-text on content and title (English snowball stemming)
• MTREE vector index on embedding (1024 dimensions, cosine distance)
• UNIQUE on concept name

Why 1024 Dimensions?

Qwen3-Embedding-4B natively outputs 2560-dim vectors but supports Matryoshka Representation Learning (MRL) for truncation. 1024 retains ~95% retrieval quality at 2.5× less storage and compute.

Performance (RTX 5070 Ti, 12GB VRAM)

• Model load (cached): ~3.4s
• Embedding speed: ~14 entries/s
• VRAM usage: ~7.6GB (fp16)
• Single query: <1s after model load

Adjust for your GPU. Any CUDA-capable card with ≥8GB VRAM should work (may need to reduce batch size).

Design Decisions

1. SCHEMAFULL — strict schema prevents garbage data
2. Graph edges as relation tables — native SurrealDB -> traversal
3. BM25 + vector search — combine keyword and semantic retrieval
4. Local GPU inference — no API costs, no data leaving your machine
5. OpenAI-compatible API — drop-in replacement for any tool expecting OpenAI embeddings
6. RocksDB backend — persistent, crash-safe, single-node

Example Queries

-- Full-text search
SELECT id, title, search::score(1) AS score
  FROM memory WHERE content @1@ "debugging"
  ORDER BY score DESC;

-- Graph traversal: what concepts does a memory reference?
SELECT ->references->concept.name FROM memory:setup;

-- Deep traversal: memory → concepts → related concepts
SELECT ->references->concept->related_to->concept.name FROM memory:model_selection;

-- Tag filtering
SELECT id, title FROM memory WHERE tags CONTAINS "decision";

License

MIT