Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OpenSUN3D: 1st Workshop Challenge on Open-Vocabulary 3D Scene Understanding

Francis Engelmann, Ayca Takmaz, Jonas Schult, Elisabetta Fedele, Johanna Wald, Songyou Peng, Xi Wang, Or Litany, Siyu Tang, Federico Tombari, Marc Pollefeys, Leonidas Guibas, Hongbo Tian, Chunjie Wang, Xiaosheng Yan, Bingwen Wang, Xuanyang Zhang, Xiao Liu, Phuc Nguyen, Khoi Nguyen, Anh Tran, Cuong Pham, Zhening Huang, Xiaoyang Wu, Xi Chen, Hengshuang Zhao, Lei Zhu, Joan Lasenby

TL;DR

An overview of the challenge hosted at the OpenSUN3D Workshop on Open-Vocabulary 3D Scene Understanding is provided, the challenge dataset, the evaluation methodology, and brief descriptions of the winning methods are presented.

Abstract

This report provides an overview of the challenge hosted at the OpenSUN3D Workshop on Open-Vocabulary 3D Scene Understanding held in conjunction with ICCV 2023. The goal of this workshop series is to provide a platform for exploration and discussion of open-vocabulary 3D scene understanding tasks, including but not limited to segmentation, detection and mapping. We provide an overview of the challenge hosted at the workshop, present the challenge dataset, the evaluation methodology, and brief descriptions of the winning methods. For additional details, please see https://opensun3d.github.io/index_iccv23.html.

OpenSUN3D: 1st Workshop Challenge on Open-Vocabulary 3D Scene Understanding

TL;DR

An overview of the challenge hosted at the OpenSUN3D Workshop on Open-Vocabulary 3D Scene Understanding is provided, the challenge dataset, the evaluation methodology, and brief descriptions of the winning methods are presented.

Abstract

This report provides an overview of the challenge hosted at the OpenSUN3D Workshop on Open-Vocabulary 3D Scene Understanding held in conjunction with ICCV 2023. The goal of this workshop series is to provide a platform for exploration and discussion of open-vocabulary 3D scene understanding tasks, including but not limited to segmentation, detection and mapping. We provide an overview of the challenge hosted at the workshop, present the challenge dataset, the evaluation methodology, and brief descriptions of the winning methods. For additional details, please see https://opensun3d.github.io/index_iccv23.html.
Paper Structure (21 sections, 4 figures, 2 tables)

This paper contains 21 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Challenge
  3. Task
  4. Dataset
  5. Challenge Phases
  6. Assets and Tools
  7. Method Constraints
  8. Submission Instructions
  9. Evaluation
  10. Challenge Participation
  11. Results
  12. Winning Methods
  13. PICO-MR (1$^{st}$ Place)
  14. Method.
  15. VinAI-3DIS (2$^{nd}$ Place)
  16. ...and 6 more sections

Figures (4)

  • Figure 1: Challenge overview. Given a 3D scene in the form of a point cloud and associated posed images, the task is to segment all object instances described by a given free-form text query.
  • Figure 2: Overview of the method proposed by the PICO-MR team. An image-level NMS method based on Grounding SAM is designed to suppress the false positives generated by original Grounding SAM. Bidirectional Merging (BM) is the post process of SAM3D which iteratively merges adjacent point clouds to final 3D masks.
  • Figure 3: Overview of the method proposed by the VinAI-3DIS team.
  • Figure 4: Overview of the method proposed by the CLIP-ranked-projection (CRP) team.