Repository navigation
Open
Labels
acceptedAccepted by the accountable Workgroup or Maintainer; not necessarily scheduled.Accepted by the accountable Workgroup or Maintainer; not necessarily scheduled.bugSomething isn't workingSomething isn't workingwg/router-models-inference-runtimeOwned by the Router Models and Inference Runtime Workgroup.Owned by the Router Models and Inference Runtime Workgroup.
Description
Proposed Workgroup
Router Models & Inference Runtime
Describe the bug
When configuring multi-model serving in the model runtime via a --models file or ServeConfig, entries without an explicit name field bypass duplicate identity validation.
Specifically:
- In
src/model-runtime/vllm_srun/config.py:268-271(load_models_file), uniqueness is only enforced whenconfig.nameis truthy:if config.name: if config.name in names: raise ValueError(f"{path}: duplicate model name {config.name!r}") names.add(config.name)
- In
src/model-runtime/vllm_srun/runtime.py:872-874(Runtime.__init__), uniqueness verification also ignores entries wherenameis omitted:labels = [served.config.name for served in self.served if served.config.name] if len(labels) != len(set(labels)): raise ValueError("served model names must be unique")
Because ServedModel.label and ServedModel.names() fall back to _name_of(config.model) when config.name is None:
- Duplicate model entries (e.g., identical models targeted to different devices or with different configuration options) pass validation without error.
- Name collisions between an explicit
name(e.g.name: "my-model") and another model's derived name (e.g.model: "org/my-model") pass validation without error. - At runtime,
ProcessHealth.describe()(/health) keys the returned dictionary byserved.label. Colliding model entries overwrite earlier entries in the dictionary comprehension, silently corrupting or concealing model health states. - In
Runtime.lookup(name), requests resolve to the first matching model inself.served. The shadowed model instance is permanently unreachable for that name, despite consuming GPU/CPU memory and worker processes.
To Reproduce
Case 1: Minimal reproduction script using vllm_srun.runtime.Runtime
Run the following in Python with src/model-runtime on PYTHONPATH:
from vllm_srun.config import ModelConfig, ServeConfig
from vllm_srun.runtime import Runtime
# Two model configs without explicit 'name' (e.g. allocated across GPUs or with distinct options)
cfg = ServeConfig(
models=[
ModelConfig(model="facebook/opt-125m", device="cpu"),
ModelConfig(model="facebook/opt-125m", device="cpu"),
]
)
# 1. Validation passes silently instead of raising ValueError("served model names must be unique")
rt = Runtime(cfg)
print("Served models loaded:", len(rt.served))
# Output: Served models loaded: 2
# 2. Labels collide
print("Model labels:", [s.label for s in rt.served])
# Output: Model labels: ['opt-125m', 'opt-125m']
# 3. /health dict keys collide and overwrite earlier model's health status
print("Health describe keys:", list(rt.health.describe().keys()))
# Output: Health describe keys: ['opt-125m'] (only 1 model reported instead of 2!)
# 4. Request lookup silently shadows served[1]
print("Lookup targets served[0]:", rt.lookup("opt-125m") is rt.served[0])
# Output: Lookup targets served[0]: True (served[1] is permanently unreachable)Case 2: Config file loader reproduction with --models file
Create models.yaml:
models:
- model: org1/qwen-2.5-7b
- model: org2/qwen-2.5-7bRun:
from vllm_srun.config import load_models_file
models = load_models_file("models.yaml")
print(f"Loaded {len(models)} models with names:", [m.name for m in models])
# Output: Loaded 2 models with names: [None, None]
# Fails to catch that both derive effective label 'qwen-2.5-7b'Expected behavior
- When multiple model entries in
--modelsorServeConfigresolve to identical effective names/labels (config.name or _name_of(config.model)), validation should fail early with a descriptiveValueError:load_models_fileshould reject duplicates with:
ValueError: <path>: duplicate model name 'qwen-2.5-7b'Runtime.__init__should reject duplicates with:
ValueError: served model names must be unique
- Distinct instances of the same underlying model artifact should require explicit, unique
namedeclarations (e.g.,name: "opt-125m-0"andname: "opt-125m-1").
Impact
- Silent Resource Waste: Multi-model processes load model weights and initialize workers/caches for the shadowed model, consuming significant GPU memory and CPU resources, while that instance is impossible to query.
- Observability & Health Distortion: In
/health,ProcessHealth.describe()returns a dictionary keyed byserved.label. When labels collide, dictionary iteration silently drops earlier models. If the first model fails while the second succeeds (or vice versa), the health endpoint misrepresents cluster readiness. - Silent Traffic Misrouting: If an explicit alias matches another model's short name, client traffic meant for model B is silently processed by model A without any warning.
Additional context
Affected Call Sites
src/model-runtime/vllm_srun/config.py:268-271(load_models_file):effective_name = config.name or _name_of(config.model) if effective_name in names: raise ValueError(f"{path}: duplicate model name {effective_name!r}") names.add(effective_name)
src/model-runtime/vllm_srun/runtime.py:872-874(Runtime.__init__):labels = [served.label for served in self.served] if len(labels) != len(set(labels)): raise ValueError("served model names must be unique")
Suggested Unit / Regression Tests
- Unit test in
src/model-runtime/tests/test_server.pyortest_api_contract.pyasserting thatRuntime(ServeConfig(models=[...]))with duplicate derived labels raisesValueError. - Test in
src/model-runtime/tests/test_cli.pyortest_engine_args.pyasserting thatload_models_fileraisesValueErrorwhen a YAML contains entries with identical model repos or colliding effective names without uniquenamedeclarations.
Metadata
Metadata
Assignees
Labels
acceptedAccepted by the accountable Workgroup or Maintainer; not necessarily scheduled.Accepted by the accountable Workgroup or Maintainer; not necessarily scheduled.bugSomething isn't workingSomething isn't workingwg/router-models-inference-runtimeOwned by the Router Models and Inference Runtime Workgroup.Owned by the Router Models and Inference Runtime Workgroup.
Type
Projects
- StatusShow more project fieldsIn progress
Activity
koladefaj commentedon Oct 21, 2026
Hi, I'd like to work on this once it's accepted.
Kaap10 commentedon Oct 21, 2026
Thanks for the interest @koladefaj! As the issue author who discovered and reproduced this, I already have the fix and regression tests prepared locally and plan to claim the /assign and open the PR once maintainers accept it.
koladefaj commentedon Oct 21, 2026
Sounds good, all yours @Kaap10. One extra case I hit while reproducing, in case it's useful for the tests: an unnamed
org/foonext to an entry explicitly namedfooalso gets through both checks. The two entries share the labelfooanddescribe()keeps one health entry.adaamko commentedon Oct 26, 2026
recommend accepting, clear repro and the fix is small
Kaap10 commentedon Oct 29, 2026
Thanks @adaamko! I have the fix and regression tests (covering both duplicate model IDs and inferred-vs-explicit name collisions) implemented and verified locally.
Could someone from the workgroup run
/acceptso I can open the PR?adaamko commentedon Oct 30, 2026
/accept
Kaap10 commentedon Oct 30, 2026
/assign
Kaap10 commentedon Oct 30, 2026
Opened PR #5650 with the fix and regression test suite. Ready for review!
[Bug] Reject duplicate and colliding model entries when name is omitt…
[Bug] Reject duplicate and colliding model entries when name is omitt…