MAN TruckScenes: A multimodal dataset for autonomous trucking in diverse conditions

Abstract

Autonomous trucking is a promising technology that can greatly impact modern logistics and the environment. Ensuring its safety on public roads is one of the main duties that requires an accurate perception of the environment. To achieve this, machine learning methods rely on large datasets, but to this day, no such datasets are available for autonomous trucks. In this work, we present MAN TruckScenes, the first multimodal dataset for autonomous trucking. MAN TruckScenes allows the research community to come into contact with truck-specific challenges, such as trailer occlusions, novel sensor perspectives, and terminal environments for the first time. It comprises more than 740 scenes of 20s each within a multitude of different environmental conditions. The sensor set includes 4 cameras, 6 lidar, 6 radar sensors, 2 IMUs, and a high-precision GNSS. The dataset's 3D bounding boxes were manually annotated and carefully reviewed to achieve a high quality standard. Bounding boxes are available for 27 object classes, 15 attributes, and a range of more than 230m. The scenes are tagged according to 34 distinct scene tags, and all objects are tracked throughout the scene to promote a wide range of applications. Additionally, MAN TruckScenes is the first dataset to provide 4D radar data with 360° coverage and is thereby the largest radar dataset with annotated 3D bounding boxes. Finally, we provide extensive dataset analysis and baseline results. The dataset, development kit, and more are available online.

Figures (18)

• Figure 1: Exemplary selection of a terminal, rain, and snow scene of the MAN TruckScenes dataset. The top row shows the fused lidar point cloud, the center row shows images of the front left camera, and the fused radar point cloud is shown at the bottom.
• Figure 2: Placement of the sensors and their corresponding coordinate systems from a top view perspective. The right sensor module is shown in the detailed drawing, the left module is identical but mirrored and the radar sensors are flipped around the x-axis. The top-mounted lidar sensors (at the sides and the front) are tilted downward. The GNSS uses a virtual coordinate system similar to the vehicle frame but is located next to the chassis IMU.
• Figure 3: Distribution of the scene tags for all 747 dataset scenes.
• Figure 4: Number of annotated 3D bounding boxes for all 27 object classes across all dataset splits.
• Figure A1: Lidar point clouds of all six lidar sensors color-coded by their time difference to the earliest recorded point in all six point clouds.