RoboOS

RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration

  ⭐️ Project (Coming soon)   │   🌎 Demo (Coming soon)   │   📑 Technical Report  

  🤖 RoboBrain 2.0: Advanced version of RoboBrain. See Better. Think Harder. Do Smarter.

  🤖 RoboBrain 1.0: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete.

💬 If you have any questions, feel free to contact us via WeChat.

🔥 Overview

The rise of embodied intelligence has intensified the need for robust multi-agent collaboration in industrial automation, service robotics, and smart manufacturing. However, current robotic systems struggle with critical limitations, including poor cross-embodiment adaptability, inefficient task scheduling, and inadequate dynamic error correction. While end-to-end vision-language-action (VLA) models (e.g., OpenVLA, RDT, Pi-0) exhibit weak long-horizon planning and task generalization, hierarchical VLA models (e.g., Helix, Gemini-Robotics, GR00T-N1) lack cross-embodiment compatibility and multi-agent coordination capabilities. To address these challenges, we present RoboOS, the first open-source embodied operating system based on a Brain-Cerebellum hierarchical architecture, facilitating a paradigm shift from single-agent to swarm intelligence. Specifically, RoboOS comprises three key components: (1) the Embodied Cloud Model, a multimodal large language model (MLLM) for global perception and high-level decision-making; (2) the Cerebellum Skill Library, a modular, plug-and-play toolkit for seamless multi-skill execution; and (3) Real-Time Shared Memory, a spatiotemporal synchronization mechanism for multi-agent state coordination. By integrating hierarchical information flow, RoboOS bridges the Embodied Brain and Cerebellum Skill Library, enabling robust planning, scheduling, and error correction for long-horizon tasks while ensuring efficient multi-agent collaboration by Real-Time Shared Memory. Moreover, we optimize edge-cloud communication and cloud-based distributed inference to support high-frequency interactions and scalable deployment. Extensive real-world experiments across diverse scenarios (e.g., restaurant, household, supermarket) demonstrate RoboOS’s versatility, supporting heterogeneous embodiments (single-arm, dual-arm, humanoid, wheeled), which provides a scalable and practical solution for cross-embodiment collaboration, pushing the boundaries of embodied intelligence.

Structure for RoboOS 2.0 (SaaS + MCP)

🎯 RoadMap

• Release RoboOS-1.0 version
• Release Technical Report of RoboOS.
• Release RoboOS-2.0 stand-alone s version
• Release friendly and detailed User Guide Manual​.
• Release more comprehensive multi-agent collaboration DEMOs based on RoboOS​.

⭐️ Guide Manual

1. Prerequisites

• Python 3.10+
• Redis server
• pip package manager

🎥 Deployment Video Tutorial

👉 Click here to watch the deployment tutorial (Recommended before getting started)

🚀 2. Deployment Methods

Two deployment options are available based on your needs:

🐳 2.1 Method One: Docker Deployment (Recommended)

Recommended for quick setup and production environments.

2.1.1 Pull the Docker Image

docker pull flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:flagscale_agent_RoboOS_v2.0

2.1.3 Open the Deployment Web Page

# You may choose to mount the RoboBrain model into the container:

docker run -itd \
    --gpus all \
    --shm-size=500g \
    --name agent \
    --hostname flagscale-agent \
    -v {your_local_path}/BAAI/RoboBrain2.0-7B:/workspace/RoboBrain2.0-7B \
    --network=host \
    -p 8888:8888 \
    -w /workspace/RoboOS \
    flagrelease-registry.cn-beijing.cr.aliyuncs.com/flagrelease/flagrelease:flagscale_agent_RoboOS_v2.0

2.3 Open the Deployment Web Page

http://127.0.0.1:8888

2.2 Method Two: Run from Source (For Development or Customization)

Recommended for local development, debugging, or feature extension.

2.2.1 Clone the Repository

git clone -b stand-alone https://github.com/FlagOpen/RoboOS.git
cd RoboOS

2.2.2 Install Dependencies

# It is recommended to use a virtual environment:

pip install -r requirements.txt

# Install flag_scale from GitHub

git clone https://github.com/FlagOpen/FlagScale 
cd FlagScale
git checkout 3fc2037f90917227bd4aebabd9d7b330523f437c

# Install in editable mode with PYTHONPATH
PYTHONPATH=./:$PYTHONPATH pip install . --verbose --no-build-isolation 

2.2.3 Start the Deployment Service

cd deploy

python run.py

2.2.4 Open the Deployment Web Page

http://127.0.0.1:8888

⚙️ 3. Skill Store Configuration

RoboOS supports two modes for accessing robot skills: Local Mode and Remote Mode. For detailed instructions on configuring specific robot skills (e.g., Realman RMC-LA), please refer to the dedicated documentation below:

📘 Realman RMC-LA Skill README Includes setup instructions, example skills, and integration guide for Realman robots.

✅ Local Mode

1. Clone the RoboSkill repository:

git clone https://github.com/FlagOpen/RoboSkill

2. Place skill.py at the local path specified in the web UI

Example: slaver/demo_robot_local/skill.py

🌐 Remote Mode

1. Host the skill.py file on a remote server accessible over the network (Robot)
2. Start the skill server:

python skill.py

✅ 4. Final Step

Visit the web UI at http://127.0.0.1:8888 and follow the on-screen instructions to complete configuration. Once finished, you can control the robot and trigger skills from the interface.

⚡️ 5. Start vLLM Model Service

RoboOS requires a large language model backend to handle reasoning and tool calls.
We recommend using vLLM to serve the RoboBrain2.0-7B model.

5.1 Install vLLM

pip install vllm

5.2 Prepare Chat Template

The tool_chat_template_hermes.jinja file must be provided for tool-call parsing. Place it in the following directory:

RoboOS/deploy/templates/tool_chat_template_hermes.jinja

5.3 Launch vLLM

Run the following command to start the model service:

vllm serve RoboBrain2.0-7B \
    --gpu-memory-utilization=0.9 \
    --max-model-len=10000 \
    --max-num-seqs=256 \
    --port=4567 \
    --trust-remote-code \
    --enable-chunked-prefill \
    --enable-auto-tool-choice \
    --tool-call-parser hermes \
    --chat-template RoboOS/deploy/templates/tool_chat_template_hermes.jinja 

⚙️ 6. Master & Slaver Configuration

Before running the system, you need to configure both the master and slaver agents.
Each agent requires a config.yaml file to define model connection, audio, and logging settings.

6.1 Configuration Files

• master/config.yaml
• slaver/config.yaml

A default template is provided below (you may adjust according to your environment):

# Cloud Server (vLLM) Model Parameters
model:
  model_select: "/workspace/model/BAAI/RoboBrain2.0-7B"
  model_retry_planning: 5
  model_dict:
    cloud_model: "/workspace/model/BAAI/RoboBrain2.0-7B"
    cloud_type: "default"
    cloud_api_key: "EMPTY"
    cloud_server: "http://localhost:4567/v1/"
    max_chat_message: 50

# Redis Collaborator
collaborator:
  host: "127.0.0.1"
  port: 6379
  db: 0
  clear: true
  password: ""

# Slaver Robot 
robot:
  # "local" with a fold name such as "demo_robot"
  # "remote" with URL such as "http://127.0.0.1:8000", and run the Python script 'skill.py' on the robot itself.
  # call_type: local
  # path: "demo_robot_local"
  name: demo_robot
  call_type: remote
  path: "http://127.0.0.1:8000"

# Master Scene profile
profile:
  path: ./scene/profile.yaml

# Slaver
tool:
  # Has the model undergone targeted training on tool_calls
  support_tool_calls: false

6.2 Key Parameters

• model.cloud_server: Must point to your vLLM service (default: http://localhost:4567/v1/)

• collaborator: Redis server configuration (default: 127.0.0.1:6379)

• profile： Path to the scene profile YAML file that defines environment and task settings (e.g., ./scene/profile.yaml)

• tool： Enable or disable tool-call support. Set support_tool_calls: true if your model has been trained for tool calls

• robot: Two modes of calling robot tools

⚠️ Make sure these fields are correctly configured; otherwise, RoboOS may fail to connect to vLLM, Redis, or load scene/tool profiles.

🔧 Manual Deployment (Advanced)

If you prefer to manually run RoboOS without using the deployment web UI, follow the steps below to start the system components directly from source.

1️⃣ Start the Master Agent

The master is responsible for receiving tasks, decomposing them, and assigning subtasks to available slaver agents.

cd master

python run.py

⚠️ You must start the master agent first, otherwise the slaver will fail to register.

2️⃣ Start the Slaver Agent

The slaver connects to the master and executes the assigned subtasks on the physical robot.

cd slaver
python run.py

You can run multiple slaver agents on different robots or terminals, each connected to the same master.

📤 Sending Tasks Manually

After starting both the master and slaver agents, you can send tasks in either of the following ways:

✅ Option 1: Use Python Script (Direct HTTP Request)

import requests

# Replace with your master agent's actual IP or hostname
MASTER_URL = "http://localhost:5000/publish_task"

payload = {
    "task": "Now you are at the kitchen table, pick up the apple from the kitchen table, navigate to the serving table, place the apple on the serving table, pick up the bowl from the serving table, navigate to the kitchen table, place the bowl on the kitchen table."
}

response = requests.post(MASTER_URL, json=payload)

print("Status:", response.status_code)
print("Response:", response.json())

Make sure the master service is running and accessible on port 5000.

✅ Option 2: Use the Release Deployment Web Interface

You can also start the deployment service and access a simple web-based task submission interface:

cd deploy
python run.py

# Then visit: http://127.0.0.1:8888/release

✨ Example Demo

Web-Based Deployment and Configuration

After launching the container or running the deployment script, you can access the RoboOS deployment interface in your browser at:

http://127.0.0.1:8888

Click the Start Deployment button to begin configuring your system.

You’ll then be guided through several steps:

Basic Settings

1. Conda Environment Selection: Select the conda environment to run
2. 🧠 Inference Service Configuration
   During the deployment process, you will be asked whether to enable the Inference Service Configuration option:

• ✅ Checked: A built-in inference service will be automatically started inside the container.
• ❌ Unchecked: You can connect to your own externally hosted inference service instead (e.g., RoboBrain running on a remote server).

Advanced Settings

Here you can customize and inspect advanced configurations before starting the deployment:

1. Master Agent Configuration:

2. Slaver Agent Configuration:

3. Robot Tools Config Preview the registered skills available to the robot.

⚠️ Ensure the skill service is already running before checking this section.

4. Start Deployment After completing the setup, click Start Deployment to launch the system.

🚀 Task Publishing and Execution

After deployment, you can send tasks to the system for execution:

📝 Step 1: Click the Publish Task button

📤 Step 2: Send a Natural Language Task Command

Example:

Now you are at the kitchen table, pick up the apple from the kitchen table, navigate to the serving table, place the apple on the serving table, pick up the bowl from the serving table, navigate to the kitchen table, place the bowl on the kitchen table.

⚙️ Step 3: Task Decomposition and Execution Results

The master agent will automatically decompose the task into subtasks and assign them to the

📑 Citation

If you find this project useful, welcome to cite us.

@article{tan2025roboos,
  title={RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration}, 
  author={Tan, Huajie and Hao, Xiaoshuai and Lin, Minglan and Wang, Pengwei and Lyu, Yaoxu and Cao, Mingyu and Wang, Zhongyuan and Zhang, Shanghang},
  journal={arXiv preprint arXiv:2505.03673},
  year={2025}
}

@article{ji2025robobrain,
  title={RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete},
  author={Ji, Yuheng and Tan, Huajie and Shi, Jiayu and Hao, Xiaoshuai and Zhang, Yuan and Zhang, Hengyuan and Wang, Pengwei and Zhao, Mengdi and Mu, Yao and An, Pengju and others},
  journal={arXiv preprint arXiv:2502.21257},
  year={2025}
}