Repository navigation
Invalid configured model path silently falls back to the cache #149
Description
ModelStore.resolve_spec_path (sdk/src/unplug/ml/store.py:310) drops an explicitly configured spec.path when it fails validation and substitutes whatever is in the local cache. No warning, no error, and ml_degraded stays False.
Repro on any machine that has run unplug-models download tiny once:
import tempfile, pathlib, os
os.environ.pop("UNPLUG_MODEL_PATH", None)
from unplug.config import GuardConfig
from unplug.core.runtime.model_runtime import merge_catalog_models
from unplug import Guard
d = pathlib.Path(tempfile.mkdtemp()) / "corrupt"
d.mkdir()
(d / "config.json").write_text("not-json")
cfg = merge_catalog_models(
GuardConfig(active_model="tiny", require_ml=False, auto_download_model=False)
)
m = dict(cfg.models)
m["tiny"] = m["tiny"].model_copy(update={"path": str(d)})
g = Guard(config=cfg.model_copy(update={"models": m}))
print(g.ml_model_loaded) # True, expected False
print(g.ml_degraded) # FalseIt loads ~/.cache/unplug/models/tiny instead. The code path:
if spec.path and self.is_valid_checkpoint(Path(spec.path)):
return spec
cached = self.resolve_local_path(tier or spec.name)
if cached is not None:
return spec.model_copy(update={"path": str(cached)})The cache lookup is a fallback for when path is unset. It also fires when path is set and invalid.
Note the asymmetry with the branch twenty lines above: an invalid UNPLUG_MODEL_PATH logs "is set but is not a valid checkpoint ... ignoring" before falling through. An invalid configured path gets nothing.
require_ml=True does not help. The fallback loads, so require_ml is satisfied and no ModelError is raised.
Why it matters: an operator pins a vetted checkpoint and turns downloads off. A path typo or a bad mount gets them a different set of weights, with ml_model_loaded=True and ml_degraded=False. Nothing in the result says the loaded model is not the pinned one.
Two existing tests already assert the correct behaviour and fail on any machine with a populated cache:
tests/unit/ml/test_model_store.py::test_guard_survives_corrupt_checkpoint
tests/unit/ml/test_model_store.py::test_guard_require_ml_corrupt_checkpoint_raises_model_error
They pass in CI only because CI has no model cache, which is also why this went unnoticed.
Fix: when spec.path is set but invalid, mirror the env branch. Log the warning and do not substitute the cached checkpoint. Raise ModelError when require_ml=True.
Related: #107, since this is a case where degraded should be set and is not.
Activity
chiruu12 commentedon Aug 29, 2026
Repro, and it is worse than the title says.
Prints the cache path, not the configured one:
sdk/src/unplug/ml/store.py:310-315. When the configuredpathfailsis_valid_checkpoint, the function falls through toresolve_local_path(tier)and returns whatever the cache holds. Nothing is logged.require_ml=Truedoes not protect against this. The substituted model satisfies it, so the flag currently means "refuse to run without a model" rather than "refuse to run without the model I configured". Someone who audits a checkpoint, pins its path, and later has that path go missing gets a different model scanning their traffic andguard.ml_model_loadedreporting True.UNPLUG_MODEL_PATHreaches the same substitution through store.py:304. That branch does warn, but it says the value is being ignored and never says what got used instead, which reads as "running without ML" when it is not.Two tests on dev already fail because of this:
They pass in CI only because the runner has no
~/.cache/unplug/models, so there is nothing available to substitute. Filed separately as the CI blind spot.Fix I plan to take: an explicitly configured path that is not a valid checkpoint is an error, never a fallback. Cache resolution stays for the case where no path was configured at all.