Multi-Machine Deployment

This system supports distributed deployment across multiple nodes. To configure:

pod-ips Parameter

• Type: string
• Format: Comma-separated IPv4 addresses
• Description: Specifies all node IPs in the deployment pod
• Required: For multi-node deployments only
• Example: "192.168.1.101,192.168.1.102,192.168.1.103"

Deployment Workflow

1. Parameter passing:
   • Concatenate all node IPs in the deployment cluster
   • Pass as a single string using the pod-ips parameter
2. Unified Node Startup:
   • Same launch command for all nodes (master and workers)
   • Nodes self-identify roles based on IP position in pod-ips

    python fastdeploy.entrypoints.openai.api_server \
    --pod-ips "IP1,IP2,IP3,...,IPN"
   

Usage

1. Only support sending request to the master node （the first ip of the pod-ips parameter)