Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method (UniScenev2)

---

📌 Introduction

UniScenev2 is a unified occupancy-centric framework for large-scale 4D dynamic scene generation in autonomous driving. This repository provides the data processing pipeline and the Nuplan-Occ dataset, the largest semantic occupancy dataset to date, built upon the NuPlan benchmark.

Overview of Nuplan-Occ dataset and the UniScenev2 pipeline. We introduce the largest semantic occupancy dataset to date, featuring dense 3D semantic annotations that contain ~19× more annotated scenes and ~18× more frames than Nuscenes-Occupancy. Facilitated with Nuplan-Occ, UniScenev2 scales up both model architecture and training data to enable high-quality occupancy spatial expansion and temporal forecasting, as well as occupancy-based sparse point map condition for video generation, and sensor-specific LiDAR generation.

---

🗃️ Nuplan-Occ Dataset

We introduce Nuplan-Occ, a large-scale semantic occupancy dataset featuring:

• ✅ 3.6M frames with dense 3D semantic annotations
• ✅ High-resolution voxel grids (400×400×32)
• ✅ Surround-view (8 cameras) and LiDAR data
• ✅ Foreground-Background Separate Aggregation (FBSA) for precise labeling

📊 Dataset Comparison

Comparison between Nuplan-Occ and other occupancy/LiDAR datasets. ''Surrounded'' represents surround-view image inputs. ''View'' means the number of image view inputs. ''C'', ''D'', and ''L'' denote camera, depth, and LiDAR, respectively.

---

🛠️ Data Pipeline

1. Environment Setup

conda env create -f data_pipeline_env.yaml
conda activate uniscenev2_data_pipeline

Install dependencies:

WORK_DIR=YOUR_WORK_DIR
cd $WORK_DIR/gs_render/data_process/nksr
pip install . -v
cd $WORK_DIR/gs_render/data_process/kiss-icp/python
pip install . -v
# Optional: for Gaussian splatting rendering
cd $WORK_DIR/gs_render/diff-gaussian-rasterization
pip install . -v
cd $WORK_DIR/gs_render/gsplat
pip install . -v

Place the neural kernel model ks.pth in ./ks.pth.

2. Prepare Data

a. Download NuPlan Dataset

Download from NuPlan and place the dataset under ./data/nuplan:

./data/nuplan
└── sensor_blobs
    ├── mini
    └── trainval

b. Download Pickle Files

Download from Huggingface and place the provided pickle files under ./data/nuplan_pkls:

./data/nuplan_pkls
├── mini
│   ├── nuplan_mini_10hz_train.pkl
│   └── nuplan_mini_10hz_val.pkl
└── trainval
    ├── nuplan_trainval_10hz_train_chunk_0_.pkl
    ...
    └── nuplan_trainval_10hz_val.pkl

3. Run Data Pipeline (Optional)

You can generate occupancy data from scratch:

# Single GPU
python gs_render/data_process/generate_occ_nuplan.py --save_path $OCC_SAVE_PATH

# Multiple GPUs
torchrun --nproc_per_node=$GPU_NUM gs_render/data_process/generate_occ_nuplan.py --save_path $OCC_SAVE_PATH

4. Download Preprocessed Data

Download the preprocessed Nuplan-Occ dataset from:
👉 Arlolo0/Nuplan-Occupancy on Hugging Face

Use merge_chunk.py to merge chunks if needed.
Visualize occupancy with gs_render/vis_occ/vis_occ.py.

📝 Note on Z-axis Ranges

Split	Resolution	Z Range	Config
Mini train	800	-5 ~ 3	config_nuplan_r800_old
Mini val	800	-3 ~ 5	config_nuplan_r800
Trainval train	400	-3 ~ 5	config_nuplan_r400
Trainval val	400	-3 ~ 5	config_nuplan_r400

⚠️ Please note: The z-axis range of Miniset train is -5 to 3.

5. Downsample Occupancy

python3 occ_process_parallels.py \
  --quantize_size 200 200 16 \
  --data_base_path $OCC_SAVE_PATH \
  --save_base_path $DOWNSAMPLED_OCC_PATH \
  --config_path "gs_render/data_process/nuplan.yaml" \
  --method "max" \
  --workers 64 \
  --pkl_path $PKL_PATH

6. Map BEV Layout to Occupancy

Refer to add_bev_layout.py for mapping BEV layouts to occupancy grids.

📋 Run and Evaluate Semantic Occupancy Prediction (SOP) Baseline

We provide a reproduced baseline using MonoScene trained on NuPlan-Occ miniset, please refer to: MonoScene Baseline for NuPlan-Occ Dataset.

---

🎨 Rendering

To render Gaussian-based sparse point maps:

Modify paths in gs_render/run_render_nuplan_mini_val_nomap_r400_ut and run:

bash gs_render/run_render_nuplan_mini_val_nomap_r400_ut.sh 0 100000

Note: If you want to render the full set or another split, remember to change the corresponding path and config.

---

📜 Citation

If you use UniScenev2 or the Nuplan-Occ dataset, please cite our paper:

@article{li2024uniscene,
  title={UniScene: Unified Occupancy-centric Driving Scene Generation},
  author={Li, Bohan and Guo, Jiazhe and Liu, Hongsi and Zou, Yingshuang and Ding, Yikang and Chen, Xiwu and Zhu, Hu and Tan, Feiyang and Zhang, Chi and Wang, Tiancai and others},
  journal={arXiv preprint arXiv:2412.05435},
  year={2024}
}

@article{li2025scaling,
  title={Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method},
  author={Li, Bohan and Jin, Xin and Zhu, Hu and Liu, Hongsi and Li, Ruikai and Guo, Jiazhe and Cai, Kaiwen and Ma, Chao and Jin, Yueming and Zhao, Hao and others},
  journal={arXiv preprint arXiv:2510.22973},
  year={2025}
}

---

⭐ Star Us!

If you find this project helpful, please give it a ⭐ on GitHub!