Description

When deploying Dynamo with disaggregated P/D (Prefill/Decode) serving, the frontend component panics with a futures-util Unfold stream error, preventing worker discovery and making all query requests hang indefinitely with no response.

Environment

• Dynamo version: v0.9.0 (also checked v0.9.1 release notes — not fixed)
• GPU: NVIDIA V100 (compute capability 7.0)
• Model: DeepSeek-R1-Distill-Qwen-7B
• Configuration: P/D disaggregated serving with 2x V100 GPUs (TP=2 per worker)
• Backend: vLLM

Error Log

The frontend pod logs show the following panic:

thread 'tokio-runtime-worker' panicked at /usr/local/cargo/registry/src/index.crates.io-.../futures-util-0.3.31/src/stream/unfold.rs:62:9:
Unfold must not be polled after it returned Poll::Ready(None)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

After this panic, the frontend's worker discovery reports entries=0, meaning no prefill or decode workers are discovered, and all incoming requests receive no response.

Steps to Reproduce

1. Deploy Dynamo with P/D disaggregated serving (1 frontend, 1 prefill worker, 1 decode worker)
2. Wait for all pods to reach Running state and workers to register
3. Send a query to the frontend endpoint
4. Observe: the request hangs with no response
5. Check frontend logs: Unfold panic and entries=0 in discovery

Root Cause Analysis

The panic originates from futures-util 0.3.31 in stream/unfold.rs:62. The Unfold stream is being polled after it has already completed (Poll::Ready(None)), which is a violation of the Stream contract. This is a known issue in futures-util 0.3.31.

The bug affects the frontend's endpoint discovery mechanism (likely the etcd/nats watch stream), causing it to lose track of all registered workers.

Expected Behavior

• Frontend should discover and maintain connections to prefill and decode workers
• Queries should be routed to prefill workers, then decode workers, and responses streamed back

Actual Behavior

• Frontend panics on the Unfold stream
• Worker discovery returns entries=0
• All queries hang indefinitely with no response

Workaround

None found. Restarting the frontend pod does not resolve the issue — the panic recurs.

Suggested Fix

Upgrade futures-util from 0.3.31 to a patched version, or pin to 0.3.30 where this Unfold polling bug does not exist.

Additional Notes

• Checked v0.9.1 release notes — this issue is not addressed
• The prefill and decode workers themselves start successfully and log model loading completion
• The issue is isolated to the frontend's stream-based discovery mechanism