Datasets
TABLE I. List of publicly available datasets for 3D-scene understanding, categories by data acquisition method, the content of the dataset, used hardware, data representation, and extent of available annotation classes. The digital version (.csv) of this table can be downloaded here. Declaration of data typ real-world (R), synthetic (S).
| Nr. | Year | Name | Resource | Data type | Objects | Indoor sites | Urban (S) | Urban (D) | Industrial | Infrastructure / Rural | Panoramic cameras | Stereo camera | RGB-D | TLS | MLS | ALS | Aerial photogrammetry | IMU | GPS | RGB sequence | Depth sequence | Point cloud | 3D model | RGB | Intensity | Mesh | Normals | # Sem. classes | Object detection | Pose estimation | Shape classfication | Object tracking | Semantic segmentation | Instance sem. segmentation | PC registration | Scene reconstruction | Surface reconstruction | Volume reconstruction | SLAM | # Points | # Frames | # Scenes | # Scans |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2009 | Oakland 3-D | link | R | 1 | 1 | 1 | 5 | 1 | 1,6M | |||||||||||||||||||||||||||||||||
| 2 | 2011 | Ford Campus Vision and Lidar Data Set | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 2 | ||||||||||||||||||||||||||
| 3 | 2012 | KITTI stereo evaluation 2012 | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 | 1 | 1 | 1 | 1 | 1,5K | 22 | ||||||||||||||||||||||
| 4 | 2013 | NYUv2 | link | R | 1 | 1 | 1 | 1 | 14 | 1 | 407,0K | 464 | |||||||||||||||||||||||||||||||
| 5 | 2013 | SUN3D | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 254 | 415 | ||||||||||||||||||||||||||||||
| 6 | 2013 | Sydney Urban Objects | link | R | 1 | 1 | 1 | 14 | 1 | 613 | |||||||||||||||||||||||||||||||||
| 7 | 2014 | Paris-rue-Madame database | link | R | 1 | 1 | 1 | 1 | 17 | 1 | 1 | 2,0M | 1 | 2 | |||||||||||||||||||||||||||||
| 8 | 2015 | iQmulus | link | R | 1 | 1 | 1 | 1 | 8 | 1 | 1 | 300,0M | 10 | ||||||||||||||||||||||||||||||
| 9 | 2015 | NCTL Dataset | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 27 | ||||||||||||||||||||||||||||
| 10 | 2015 | SceneNet | link | S | 1 | 1 | 1 | 1 | 1 | 1 | 59 | ||||||||||||||||||||||||||||||||
| 11 | 2015 | ShapeNet | link | S | 1 | 1 | 1 | 270 | 1 | 1 | 1 | 51300 | |||||||||||||||||||||||||||||||
| 12 | 2015 | SUN RGB-D | link | R | 1 | 1 | 1 | 1 | 53 | 1 | 1 | 1 | 10,3K | 10335 | 10335 | ||||||||||||||||||||||||||||
| 13 | 2016 | ObjectNet3D | link | S | 1 | 1 | 100 | 1 | 1 | 1 | 1 | 90,1K | |||||||||||||||||||||||||||||||
| 14 | 2016 | Oxford RobotCar | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 20,0M | |||||||||||||||||||||||||||||||
| 15 | 2016 | SceneNet RGB-D | link | S | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 5,0M | 57 | 15000 | |||||||||||||||||||||||||
| 16 | 2016 | SceneNN | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 40 | 1 | 1 | 1 | 100 | 100 | |||||||||||||||||||||||||
| 17 | 2017 | 2D-3D-S Dataset | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 13 | 1 | 1 | 1 | 1 | 695,9M | 70,5K | 7 | ||||||||||||||||||||||
| 18 | 2017 | Matterport3D | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 40 | 1 | 1 | 1 | 1 | 194,4K | 90 | 10800 | ||||||||||||||||||||||
| 19 | 2017 | Redwood | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 5 | |||||||||||||||||||||||||||
| 20 | 2017 | S3DIS | link | R | 1 | 1 | 1 | 1 | 1 | 13 | 1 | 1 | 6 | 271 | |||||||||||||||||||||||||||||
| 21 | 2017 | Semantic3D | link | R | 1 | 1 | 1 | 1 | 1 | 9 | 1 | 1 | 1 | 4,0B | 30 | ||||||||||||||||||||||||||||
| 22 | 2018 | Paris-Lille-3D | link | R | 1 | 1 | 1 | 1 | 1 | 50 | 1 | 1 | 1 | 143,1M | 11 | 115 | |||||||||||||||||||||||||||
| 23 | 2018 | ScanNet V2 | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 20 | 1 | 1 | 1 | 1 | 1 | 1 | 2,5M | 707 | 1513 | ||||||||||||||||||||||
| 24 | 2018 | WHU-TLS dataset | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1,7B | 11 | 115 | ||||||||||||||||||||||||||||||
| 25 | 2019 | A*3D | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 7 | 1 | 39,2K | ||||||||||||||||||||||||||||||
| 26 | 2019 | Agroverse1 | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 17 | 1 | 1 | 1 | 6,6K | 113 | 324557 | ||||||||||||||||||||||||
| 27 | 2019 | ApolloScape | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 25 | 1 | 1 | 1 | 1 | 1 | 70,0K | 140,0K | |||||||||||||||||||||||
| 28 | 2019 | BLVD | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 3 | 1 | 1 | 1 | 120,0K | ||||||||||||||||||||||||||||
| 29 | 2019 | H3D | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 | 1 | 1 | 1 | 27,7K | 160 | 27721 | ||||||||||||||||||||||
| 30 | 2019 | Lyft Level 5 | link | R | 1 | 1 | 1 | 1 | 3 | 1 | 1 | 1 | 1 | 170000 | |||||||||||||||||||||||||||||
| 31 | 2019 | PartNet | link | S | 1 | 1 | 1 | 24 | 1 | 1 | 1 | 26671 | |||||||||||||||||||||||||||||||
| 32 | 2019 | PreSIL | link | S | 1 | 1 | 1 | 1 | 1 | 2 | 1 | 1 | 1 | 1 | 50,0K | ||||||||||||||||||||||||||||
| 33 | 2019 | Replica | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 88 | 1 | 1 | 18 | ||||||||||||||||||||||||||||
| 34 | 2019 | ScanObjectNN | link | R | 1 | 1 | 1 | 1 | 1 | 15 | 1 | 1 | 1 | 2902 | |||||||||||||||||||||||||||||
| 35 | 2019 | SemanticKITTI | link | R | 1 | 1 | 1 | 28 | 1 | 1 | 1 | 22 | 43552 | ||||||||||||||||||||||||||||||
| 36 | 2019 | Structured3D | link | S | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 40 | 1 | 1 | 1 | 1 | 196,0K | 3500 | 21835 | |||||||||||||||||||||||
| 37 | 2019 | SynthCity | link | S | 1 | 1 | 1 | 1 | 9 | 1 | 367,9M | 1 | 9 | ||||||||||||||||||||||||||||||
| 38 | 2020 | Campus3D | link | R | 1 | 1 | 1 | 1 | 24 | 1 | 1 | 1 | 1 | 1,0B | 6 | ||||||||||||||||||||||||||||
| 39 | 2020 | DALES | link | R | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 | 505,0M | 40 | |||||||||||||||||||||||||||||
| 40 | 2020 | nuScenes-lidarseg | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 32 | 1 | 1 | 1 | 1 | 1,4B | 1,4M | 1000 | |||||||||||||||||||||||||
| 41 | 2020 | Toronto-3D | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 8 | 1 | 1 | 78,3M | 1 | 4 | ||||||||||||||||||||||||||
| 42 | 2020 | Waymo Open | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 23 | 1 | 1 | 1 | 390,0K | 1150 | 230000 | ||||||||||||||||||||||||||
| 43 | 2021 | 3D-FRONT | link | S | 1 | 1 | 1 | 1 | 6813 | 18797 | |||||||||||||||||||||||||||||||||
| 44 | 2021 | Agroverse2 | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 30 | 1 | 1 | 1 | 1000 | |||||||||||||||||||||||||||
| 45 | 2021 | BuildingNet | link | S | 1 | 1 | 1 | 31 | 1 | 1 | 2000 | ||||||||||||||||||||||||||||||||
| 46 | 2021 | ONCE | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 5 | 1 | 1 | 1,0M | 581 | ||||||||||||||||||||||||||||
| 47 | 2021 | Paris-CARLA-3D | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 23 | 1 | 1 | 60,0M | 12 | ||||||||||||||||||||||||||
| 48 | 2021 | Paris-CARLA-3D | link | S | 1 | 1 | 1 | 1 | 23 | 1 | 1 | 700,0M | 12 | ||||||||||||||||||||||||||||||
| 49 | 2021 | PSNet5 | link | R | 1 | 1 | 1 | 1 | 1 | 5 | 1 | 80,0M | |||||||||||||||||||||||||||||||
| 50 | 2022 | LiSurveying | link | R | 1 | 1 | 1 | 1 | 1 | 1 | 54 | 1 | 1 | 1 | 2,5B | 3 | |||||||||||||||||||||||||||
| 51 | 2022 | VASAD | link | S | 1 | 1 | 1 | 1 | 11 | 1 | 1 | 1 | 1 | 6 |
Point Cloud Semantic Segmentation on the S3DIS dataset benchmark.
TABLE 2: Reported results for semantic segmentation task on the large-scale indoor S3DIS benchmark (including all 6 areas, 6-fold cross validation). Ranked in descending order based on mIoU performance.
Declaration: C—convolution-based, G—graph-based, H—hybrid, P—pooling-based, R—RNN-based, T—Transformer-based, V—voxel-based.
| Rank | Year | Model Name | Link | Method | mIoU | mAcc | oAcc |
|---|---|---|---|---|---|---|---|
| 1 | 2022 | WindowNorm+StratifiedTransformer | link | T | 77.60 | 85.8 | |
| 2 | 2022 | PointMetaBase-XXL | link | MLP | 77.00 | - | |
| 3 | 2022 | PointNeXt-XL | link | MLP | 74.90 | 83.0 | |
| 4 | 2022 | DeepViewAgg | link | H | 74.70 | 83.8 | |
| 5 | 2022 | RepSurf-U | link | MLP | 74.30 | 82.6 | |
| 6 | 2022 | WindowNorm+PointTransformer | link | T | 74.10 | 82.5 | |
| 7 | 2022 | PointNeXt-L | link | MLP | 73.90 | 82.2 | |
| 8 | 2020 | PointTransformer | link | T | 73.50 | 81.9 | |
| 9 | 2022 | CBL | link | C | 73.10 | 79.4 | |
| 10 | 2021 | BAAF-Net | link | MLP | 72.20 | 83.1 | |
| 11 | 2021 | SCF-Net | link | MLP | 71.60 | 82.7 | |
| 13 | 2020 | FG-Net | link | C | 70.80 | 82.9 | |
| 12 | 2021 | RPNet-D27 | link | MLP | 70.80 | - | |
| 14 | 2019 | KPConv | link | C | 70.60 | 79.1 | |
| 15 | 2021 | FastPointTransformer (small) | link | T | 70.30 | - | |
| 16 | 2018 | PointSIFT | link | MLP | 70.23 | - | |
| 17 | 2019 | RandLA-Net | link | T | 70.00 | 81.5 | |
| 18 | 2020 | MuGNet | link | G | 69.80 | - | |
| 19 | 2020 | PointASNL | link | MLP | 68.70 | 79.0 | |
| 20 | 2020 | FPConv | link | C | 68.70 | - | |
| 21 | 2019 | SSP+SPG | link | G | 68.40 | 78.3 | |
| 22 | 2020 | FKAConv | link | C | 68.40 | - | |
| 23 | 2019 | ConvPoint | link | C | 68.20 | - | |
| 24 | 2019 | HPEIN | link | G | 67.82 | 76.26 | |
| 25 | 2020 | JSENet | link | C | 67.70 | - | |
| 26 | 2020 | CT2 | link | T | 67.40 | - | |
| 27 | 2019 | ShellNet | link | MLP | 66.80 | - | |
| 28 | 2019 | PointWeb | link | MLP | 66.70 | 76.2 | |
| 29 | 2019 | InterpCNN | link | C | 66.70 | - | |
| 30 | 2019 | PAG | link | G | 65.90 | - | |
| 32 | 2018 | PointCNN | link | C | 65.40 | 75.6 | |
| 31 | 2019 | MinkowskiNet | link | V | 65.40 | - | |
| 33 | 2019 | DPAM | link | G | 64.50 | - | |
| 34 | 2019 | PAT | link | T | 64.28 | - | |
| 35 | 2021 | DSPoint | link | H | 63.30 | 70.9 | |
| 36 | 2019 | A-CNN | link | C | 62.90 | - | |
| 37 | 2019 | LSANet | link | MLP | 62.20 | - | |
| 38 | 2017 | SPG | link | G | 62.10 | 73.0 | |
| 39 | 2019 | JSNet | link | MLP | 61.70 | 71.7 | |
| 40 | 2019 | DeepGCN | link | G | 60.00 | - | |
| 41 | 2019 | ASIS | link | MLP | 59.30 | 70.1 | |
| 43 | 2018 | Engelmann | link | MLP | 58.27 | 67.77 | |
| 42 | 2018 | PCNN | link | C | 58.27 | 67.01 | |
| 44 | 2018 | RSNet | link | RNN | 56.50 | 66.5 | |
| 45 | 2018 | 3P-RNN | link | RNN | 56.30 | 73.6 | |
| 46 | 2018 | DGCNN | link | G | 56.10 | - | |
| 47 | 2019 | PyramNet | link | G | 55.60 | - | |
| 48 | 2017 | 3DContextNet | link | MLP | 55.60 | 74.5 | |
| 49 | 2020 | Point-PlaneNet | link | MLP | 54.80 | - | |
| 50 | 2017 | PointNet++ | link | MLP | 54.49 | 67.05 | |
| 51 | 2018 | A-SCN | link | MLP | 52.72 | - | |
| 52 | 2021 | SMS | link | G | 51.74 | - | |
| 53 | 2018 | G+RCU | link | RNN | 49.70 | 66.4 | |
| 54 | 2016 | PointNet | link | MLP | 47.71 | 66.2 |