logo

Introduction

OmniHD-Scenes is a multimodal dataset designed for omnidirectional environmental perception. OmniHD-Scenes integrates data from high-beam LiDAR, high-resolution cameras, and 4D imaging radars, capturing complex urban scenarios and hazardous test cases in closed facilities. This dataset aims to serve as a robust foundation for developing and validating advanced autonomous driving algorithms, particularly for 3D object detection, multi-object tracking, and semantic occupancy prediction.

OmniHD-Scenes provides synchronized raw data and high-quality ground truth labels across multiple modalities:

  • Multimodal Input: Time-synchronized data streams from 128-beam LiDAR points, 6-view camera images, and 6-view 4D radar point clouds.

  • 3D Bounding Boxes: Accurate 3D annotations for dynamic objects (cars, pedestrians, riders, large vehicles) projected onto both image planes and 3D point clouds.

  • Semantic Occupancy: Voxelized semantic occupancy grids that describe the state and semantics of the spatial environment, offering richer detail than standard bounding boxes.

  • Scene Diversity: Samples include varied lighting (day/night) and weather conditions (sunny/rainy), demonstrating the robustness of the sensor suite.

Dataset Overview

Overall Scale

OmniHD-Scenes comprises 1,501 continuous clips (each approx. 30 seconds), totaling over 450,000 synchronized frames and 5.85 million synchronized sensor data points.

1501

Total Clips

~30s

Per Clip

450,000+

Total Frames

5,850,000+

Sensor Data Points

Sensor Configuration

LiDAR

  • Quantity: 1

  • Beams: 128-beam (Robosense RS-Ruby-128)

  • Frequency: 10 FPS

  • Points per Frame: Concentrated between 180K and 210K

Camera

  • Quantity: 6

  • Front/Rear-view (1 each): 8MP (3840×2160), 30 FPS

  • Surround-view (4): 2MP (1920×1080), 30 FPS

4D Imaging Radar

  • Quantity: 6

  • Frequency: 13 FPS

  • Points per Frame: Primarily ranging from 2K to 4K

Scenarios

Extensive coverage of complex urban traffic, highways, and overpasses. It includes challenging environmental conditions such as rainy weather (33%) and night scenes (28%), as well as critical closed-site test cases (e.g., AEB scenarios).

  • Rainy
  • Cloudy
  • Sunny

Weather

  • Urban
  • Overpass
  • Highway
  • Countryside

Road

  • Daytime
  • Night

Light

Annotation Statistics

  • Annotated Clips
  • Unannotated Clips

Annotation Statistics

3D Bounding Boxes

514,600

Key Frames

11,921

Data Sample

To access the OmniHD-Scenes dataset for research purposes, please follow the steps below:

  1. Download Agreement: Click here to download the OmniHD-Scenes Data Use Agreement (PDF).

  2. Submit Application: Please fill out the agreement completely. Send an email with the signed agreement attached to contact@2077ai.com.

  3. Requirements: In the body of your email, you must strictly provide your Full Name and Affiliated Institution.

Once your application is reviewed and approved, you will receive the download credentials via email. The full dataset (~1.3TB) is hosted on Alibaba Cloud OSS with global high-speed acceleration. It can be downloaded directly via our provided Python script.

BibTeX

Codebibtex
@article{zheng2024omnihd,
  title   =  {OmniHD-Scenes: A Next-Generation Multimodal Dataset for Autonomous Driving},
  author  =  {Zheng, Lianqing and Yang, Long and Lin, Qunshu and Ai, Wenjin and Liu, Minghao and Lu, Shouyi and Liu, Jianan and Ren, Hongze and Mo, Jingyue and Bai, Xiaokai and Bai, Jie and Ma, Zhixiong and Zhu, Xichan},
  journal =  {arXiv preprint arXiv:2412.10734},
  year    =  {2024}
}

Designed by 2077AI Team