SydneyScapes: Image Segmentation for Australian Environments

TL;DR

The paper addresses the gap in locally relevant data for autonomous vehicle perception in Australia by introducing SydneyScapes, a 756-image dataset with pixel-level semantic, instance, and panoptic annotations collected in New South Wales. It covers data collection (Day/Night/People subsets), a privacy-preserving anonymisation pipeline, a seven-group labeling scheme with train/validation splits, and a Colab-based visualization tool. The authors benchmark semantic and instance segmentation using Cityscapes-pretrained models and demonstrate significant gains from local fine-tuning, with Transformer-based methods (e.g., Mask2Former) achieving the highest performance and showing robustness to domain shifts in Australian scenes. Overall, SydneyScapes provides a practical resource and benchmarking baseline to advance perception algorithms tailored to Australian urban environments and supports domain adaptation for AV deployment.

Abstract

Autonomous Vehicles (AVs) are being partially deployed and tested across various global locations, including China, the USA, Germany, France, Japan, Korea, and the UK, but with limited demonstrations in Australia. The integration of machine learning (ML) into AV perception systems highlights the need for locally labelled datasets to develop and test algorithms in specific environments. To address this, we introduce SydneyScapes - a dataset tailored for computer vision tasks of image semantic, instance, and panoptic segmentation. This dataset, collected from Sydney and surrounding cities in New South Wales (NSW), Australia, consists of 756 images with high-quality pixel-level annotations. It is designed to assist AV industry and researchers by providing annotated data and tools for algorithm development, testing, and deployment in the Australian context. Additionally, we offer benchmarking results using state-of-the-art algorithms to establish reference points for future research and development. The dataset is publicly available at https://hdl.handle.net/2123/33051.

Figures (8)

• Figure 1: Segmentation techniques applied to an urban scene, Fig. \ref{['fig:semantic']} shows a city street, Fig. \ref{['fig:semantic']} categorises areas by class using different colours. Fig. \ref{['fig:instance']} highlights individual objects with unique colours and Fig. \ref{['fig:panoptic']} identifies objects by both class and instance.
• Figure 2: Camera's positioning and coverage area for data collection.
• Figure 3: Sample images from the SydneyScapes dataset: (a) Day subset, (b) Night subset, and (c) People subset.
• Figure 4: Original images, which are publicly available on the internet, alongside the processed images after applying the face and number plate anonymisation.
• Figure 5: Selection panel for dataset inspection.