Repository navigation
refactor(embedding): split embedding_engine.py along three seams (#173) - #175
Merged
Merged
Conversation
Fowler Extract Class. Introduce embedding_provider.py: - EmbeddingProvider Protocol — the text→vector interface the store, handlers and hooks consume, decoupled from which encoder produces the vectors. This is the seam a second (download-free) encoder plugs into without any consumer change. - _EmbeddingMathMixin — the stateless vector arithmetic (cache key, L2 normalize, cosine similarity, blob⇄list) shared by any provider, kept as a mixin so EmbeddingEngine still exposes _cache_key/similarity/to_list/ from_list/_normalize on the class exactly as before. Behaviour-preserving: methods moved verbatim; EmbeddingEngine now inherits the mixin. The existing embedding test suite passes unchanged. Co-Authored-By: Claude Fable 5 <[email protected]>
Fowler Extract Class + Replace Type Code with explicit state. Introduce embedding_model_lifecycle.py: - ModelState enum names the distinct load outcomes — UNINITIALIZED, LOADED, PACKAGE_ABSENT (sentence_transformers not installed), MODEL_FILES_ABSENT (package present, weights not cached → download), LOAD_RAISED (construction failed). This is the state space where a silent-degradation bug can hide; naming it is the seam a second encoder will map each state onto. - _EmbeddingLifecycleMixin owns device detection/resolution and the load path (_detect_device, _resolve_device, _fallback_to_cpu, _ensure_model split into _construct_model/_download_model/_load_model, _trigger_background_install). - DEFAULT_MODEL_REVISION and embedding_cache_dir move here; embedding_engine re-exports them for import compatibility. embedding_engine.py is now the concrete neural provider (encode path + LRU cache) plus the composition root (get_embedding_engine). Behaviour preserved: PACKAGE_ABSENT → hash fallback + background install; MODEL_FILES_ABSENT → download; LOAD_RAISED → propagates — identical to the pre-split loader. Existing embedding tests pass unchanged. File sizes: engine 263, lifecycle 294, provider 115 (all <=300); every method <=40 lines. Closes #173 Co-Authored-By: Claude Fable 5 <[email protected]>
Fowler Move Function / Extract Class. Introduce embedding_factory.py — the "engine selection / wiring" seam (brief seam 3): the single place that reads the runtime settings (EMBEDDING_DIM / EMBEDDING_DEVICE) and assembles the process-wide encoder. get_embedding_engine / reset_embedding_engine move here; embedding_engine re-exports them so every existing consumer import (`from ...embedding_engine import get_embedding_engine`) keeps working. The concrete EmbeddingEngine + get_memory_settings are imported lazily inside get_embedding_engine so the factory has no module-load dependency on the engine (no import cycle; verified importing in either order). Result — the three seams are now three modules plus the concrete provider: embedding_provider.py 115 (seam 1: interface) embedding_model_lifecycle.py 294 (seam 2: load state machine) embedding_factory.py 52 (seam 3: selection/composition root) embedding_engine.py 245 (concrete neural provider) All <=300 lines, every method <=40. Behaviour-preserving; embedding tests pass unchanged. Co-Authored-By: Claude Fable 5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase A — pure refactoring, zero feature change
Splits
infrastructure/embedding_engine.py(506 lines on main) intobehaviour-preserving modules along the three real seams, so a second encoder
(the #169 download-free fallback) can later plug in as a second implementation
without touching the monolith. No fallback embedder is added here.
Closes #173Seam design
Seam 1 — encoder provider interface (
embedding_provider.py, 115 lines)EmbeddingProvider— aruntime_checkableProtocol: the text→vectorinterface the store, handlers, and hooks depend on, decoupled from which
encoder produces the vectors.
EmbeddingEngineis its first (neural)implementation and structurally satisfies it (asserted in a smoke check).
_EmbeddingMathMixin— the stateless vector arithmetic shared by any provider(cache key, L2 normalize, cosine similarity, blob⇄list), kept as a mixin so
EmbeddingEngine._cache_key/.similarity/.to_liststay on the class.Seam 2 — model lifecycle state machine (
embedding_model_lifecycle.py, 294 lines)ModelStateenum makes the load state space explicit — this is exactlywhere a silent-degradation bug can hide:
UNINITIALIZED— no load attempted (lazy engine)LOADED— a neural model is in handPACKAGE_ABSENT—import sentence_transformersfailed (not installed)MODEL_FILES_ABSENT— package present, weights not cached → download attemptedLOAD_RAISED— construction raised (corrupt cache / offline download failure)_EmbeddingLifecycleMixinowns device detection/resolution and the load path(
_ensure_modeldecomposed into_construct_model/_download_model/_load_model, each ≤40 lines). Behaviour is preserved verbatim:PACKAGE_ABSENT→ hash fallback + background install;MODEL_FILES_ABSENT→download;
LOAD_RAISED→ propagates.embedding_cache_dirmove here.Seam 3 — engine selection / composition root (
embedding_factory.py, 52 lines)get_embedding_engine/reset_embedding_engine— the single place that readsEMBEDDING_DIM/EMBEDDING_DEVICEand assembles the process-wide encoder.Lazy internal import of
EmbeddingEngineavoids an import cycle (verified inboth import orders).
embedding_enginere-exports these so every existingconsumer import keeps working.
Concrete provider (
embedding_engine.py, 245 lines) — the neuralEmbeddingEngine: encode path + LRU cache, composing the two mixins. Re-exportsDEFAULT_MODEL_REVISION,embedding_cache_dir,EmbeddingProvider,ModelState,get_embedding_engine,reset_embedding_enginefor importcompatibility.
Proof of behaviour preservation
test_embedding_engine.py,test_embedding_cache.py, 56 tests) passes unchanged — no test edited.documented pre-existing test: test_rebuild_profiles depends on live ~/.claude data — 3 failures reproduce on main #174
test_rebuild_profilesfailures (depend on live~/.claudedata; reproduce identically onmain) — unrelated to this PR.ruff check .andruff format --checkclean.One concern per commit (Fowler Extract Class ×2 + Move Function):
🤖 Generated with Claude Code