Skip to content

[Bug] Multi-model runtime accepts duplicate and colliding model entries when name is omitted, causing silent shadowing and health status corruption #5347

Open

Description

@Kaap10

Proposed Workgroup

Router Models & Inference Runtime

Describe the bug

When configuring multi-model serving in the model runtime via a --models file or ServeConfig, entries without an explicit name field bypass duplicate identity validation.

Specifically:

  1. In src/model-runtime/vllm_srun/config.py:268-271 (load_models_file), uniqueness is only enforced when config.name is truthy:
    if config.name:
        if config.name in names:
            raise ValueError(f"{path}: duplicate model name {config.name!r}")
        names.add(config.name)
  2. In src/model-runtime/vllm_srun/runtime.py:872-874 (Runtime.__init__), uniqueness verification also ignores entries where name is omitted:
    labels = [served.config.name for served in self.served if served.config.name]
    if len(labels) != len(set(labels)):
        raise ValueError("served model names must be unique")

Because ServedModel.label and ServedModel.names() fall back to _name_of(config.model) when config.name is None:

  • Duplicate model entries (e.g., identical models targeted to different devices or with different configuration options) pass validation without error.
  • Name collisions between an explicit name (e.g. name: "my-model") and another model's derived name (e.g. model: "org/my-model") pass validation without error.
  • At runtime, ProcessHealth.describe() (/health) keys the returned dictionary by served.label. Colliding model entries overwrite earlier entries in the dictionary comprehension, silently corrupting or concealing model health states.
  • In Runtime.lookup(name), requests resolve to the first matching model in self.served. The shadowed model instance is permanently unreachable for that name, despite consuming GPU/CPU memory and worker processes.

To Reproduce

Case 1: Minimal reproduction script using vllm_srun.runtime.Runtime

Run the following in Python with src/model-runtime on PYTHONPATH:

from vllm_srun.config import ModelConfig, ServeConfig
from vllm_srun.runtime import Runtime

# Two model configs without explicit 'name' (e.g. allocated across GPUs or with distinct options)
cfg = ServeConfig(
    models=[
        ModelConfig(model="facebook/opt-125m", device="cpu"),
        ModelConfig(model="facebook/opt-125m", device="cpu"),
    ]
)

# 1. Validation passes silently instead of raising ValueError("served model names must be unique")
rt = Runtime(cfg)
print("Served models loaded:", len(rt.served))
# Output: Served models loaded: 2

# 2. Labels collide
print("Model labels:", [s.label for s in rt.served])
# Output: Model labels: ['opt-125m', 'opt-125m']

# 3. /health dict keys collide and overwrite earlier model's health status
print("Health describe keys:", list(rt.health.describe().keys()))
# Output: Health describe keys: ['opt-125m'] (only 1 model reported instead of 2!)

# 4. Request lookup silently shadows served[1]
print("Lookup targets served[0]:", rt.lookup("opt-125m") is rt.served[0])
# Output: Lookup targets served[0]: True (served[1] is permanently unreachable)

Case 2: Config file loader reproduction with --models file

Create models.yaml:

models:
  - model: org1/qwen-2.5-7b
  - model: org2/qwen-2.5-7b

Run:

from vllm_srun.config import load_models_file

models = load_models_file("models.yaml")
print(f"Loaded {len(models)} models with names:", [m.name for m in models])
# Output: Loaded 2 models with names: [None, None]
# Fails to catch that both derive effective label 'qwen-2.5-7b'

Expected behavior

  1. When multiple model entries in --models or ServeConfig resolve to identical effective names/labels (config.name or _name_of(config.model)), validation should fail early with a descriptive ValueError:
    • load_models_file should reject duplicates with:
      ValueError: <path>: duplicate model name 'qwen-2.5-7b'
    • Runtime.__init__ should reject duplicates with:
      ValueError: served model names must be unique
  2. Distinct instances of the same underlying model artifact should require explicit, unique name declarations (e.g., name: "opt-125m-0" and name: "opt-125m-1").

Impact

  1. Silent Resource Waste: Multi-model processes load model weights and initialize workers/caches for the shadowed model, consuming significant GPU memory and CPU resources, while that instance is impossible to query.
  2. Observability & Health Distortion: In /health, ProcessHealth.describe() returns a dictionary keyed by served.label. When labels collide, dictionary iteration silently drops earlier models. If the first model fails while the second succeeds (or vice versa), the health endpoint misrepresents cluster readiness.
  3. Silent Traffic Misrouting: If an explicit alias matches another model's short name, client traffic meant for model B is silently processed by model A without any warning.

Additional context

Affected Call Sites

  1. src/model-runtime/vllm_srun/config.py:268-271 (load_models_file):
    effective_name = config.name or _name_of(config.model)
    if effective_name in names:
        raise ValueError(f"{path}: duplicate model name {effective_name!r}")
    names.add(effective_name)
  2. src/model-runtime/vllm_srun/runtime.py:872-874 (Runtime.__init__):
    labels = [served.label for served in self.served]
    if len(labels) != len(set(labels)):
        raise ValueError("served model names must be unique")

Suggested Unit / Regression Tests

  • Unit test in src/model-runtime/tests/test_server.py or test_api_contract.py asserting that Runtime(ServeConfig(models=[...])) with duplicate derived labels raises ValueError.
  • Test in src/model-runtime/tests/test_cli.py or test_engine_args.py asserting that load_models_file raises ValueError when a YAML contains entries with identical model repos or colliding effective names without unique name declarations.

Activity

  1. added
    needs-acceptanceAwaiting explicit Maintainer or Workgroup acceptance before implementation.
    on Oct 21, 2026
  2. koladefaj commented on Oct 21, 2026

    @koladefaj
    Contributor

    Hi, I'd like to work on this once it's accepted.

  3. Kaap10 commented on Oct 21, 2026

    @Kaap10
    ContributorAuthor

    Thanks for the interest @koladefaj! As the issue author who discovered and reproduced this, I already have the fix and regression tests prepared locally and plan to claim the /assign and open the PR once maintainers accept it.

  4. koladefaj commented on Oct 21, 2026

    @koladefaj
    Contributor

    Sounds good, all yours @Kaap10. One extra case I hit while reproducing, in case it's useful for the tests: an unnamed org/foo next to an entry explicitly named foo also gets through both checks. The two entries share the label foo and describe() keeps one health entry.

  5. adaamko commented on Oct 26, 2026

    @adaamko
    Collaborator

    recommend accepting, clear repro and the fix is small

  6. Kaap10 commented on Oct 29, 2026

    @Kaap10
    ContributorAuthor

    Thanks @adaamko! I have the fix and regression tests (covering both duplicate model IDs and inferred-vs-explicit name collisions) implemented and verified locally.

    Could someone from the workgroup run /accept so I can open the PR?

  7. adaamko commented on Oct 30, 2026

    @adaamko
    Collaborator

    /accept

  8. added
    acceptedAccepted by the accountable Workgroup or Maintainer; not necessarily scheduled.
    and removed
    needs-acceptanceAwaiting explicit Maintainer or Workgroup acceptance before implementation.
    on Oct 30, 2026
  9. Kaap10 commented on Oct 30, 2026

    @Kaap10
    ContributorAuthor

    /assign

  10. moved this from Backlog to In progress in vLLM Semantic Routeron Oct 30, 2026
  11. Kaap10 commented on Oct 30, 2026

    @Kaap10
    ContributorAuthor

    Opened PR #5650 with the fix and regression test suite. Ready for review!

  12. added 2 commits that reference this issue on Nov 11, 2026
    ee037dd
    47c3472
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

acceptedAccepted by the accountable Workgroup or Maintainer; not necessarily scheduled.bugSomething isn't workingwg/router-models-inference-runtimeOwned by the Router Models and Inference Runtime Workgroup.

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions