Motivation

Currently, EngineWorkerQueue requires explicit port configuration. This PR adds support for anonymous port allocation (port=0), allowing the system to automatically assign available ports. This enhancement improves flexibility in deployment scenarios where predefined ports might be unavailable or when running multiple instances on the same host.

isn’t compatible with the multi_api_server interface

Modifications

• Added support in EngineWorkerQueue to handle anonymous port configuration (port=0)
• Updated port assignment logic to use system-allocated ports when requested
• Added port discovery and reporting mechanisms to ensure proper communication between components

Usage or Command

# Example usage with anonymous port
worker_queue = EngineWorkerQueue(port=0)  # System will allocate an available port
actual_port = worker_queue.get_server_port()  # Get the system-assigned port

--cache-queue-port and --engine-worker-queue-port args default 0
python -m fastdeploy.entrypoints.openai.api_server

Accuracy Tests

This change does not affect model outputs or inference accuracy as it only modifies the networking infrastructure. All existing functionality remains unchanged.

Checklist

• Add at least a tag in the PR title.
  • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
  • You can add new tags based on the PR content, but the semantics must be clear.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.