Computer Vision with a Superpixelation Camera

Abstract

Conventional cameras generate a lot of data that can be challenging to process in resource-constrained applications. Usually, cameras generate data streams on the order of the number of pixels in the image. However, most of this captured data is redundant for many downstream computer vision algorithms. We propose a novel camera design, which we call SuperCam, that adaptively processes captured data by performing superpixel segmentation on the fly. We show that SuperCam performs better than current state-of-the-art superpixel algorithms under memory-constrained situations. We also compare how well SuperCam performs when the compressed data is used for downstream computer vision tasks. Our results demonstrate that the proposed design provides superior output for image segmentation, object detection, and monocular depth estimation in situations where the available memory on the camera is limited. We posit that superpixel segmentation will play a crucial role as more computer vision inference models are deployed in edge devices. SuperCam would allow computer vision engineers to design more efficient systems for these applications.

Figures (23)

• Figure 1: Comparison of memory-restricted SNIC and SuperCam superpixel images. Images taken from the BSD500 dataset, shown at different memory settings in kilobytes (KB). These are raw output images without Gaussian blur applied. SuperCam results show better visual details and less aliasing.
• Figure 2: The SuperCam Algorithm
• Figure 2: Comparisons of SuperCam with other superpixel algorithms. Quantitative comparisons of SuperCam with other learning based and non-learning based superpixel algorithms. (a) Comparisons of SuperCam with learning based superpixel algorithms: We show comparisons of SuperCam with two recent learning based algorithms SPAM and LNS-Net. (b) Comparisons of SuperCam with non-learning based superpixel algorithms: Here we show comparisons of SuperCam with memory restricted SNIC, SLIC and ERS on the BSD500 dataset. We can see that the overall trend is similar for all the memory restricted algorithms and SuperCam performs better that all algorithms.
• Figure 3: Quantitative comparison with non-learning based methods. We compare the superpixel segmentation quantitatively using the BSD500, NYUV2, SBD and SUNRGBD datasets. (a) Under Segmentation Error Under Segmentation Error comparison for SuperCam and SNIC for all datasets. SuperCam does at least twice as well as SNIC when using the same amount of memory. (b) Superpixel Performance Precision vs recall plot for the SuperCam and SNIC algorithms. SuperCam has better recall than SNIC on all datasets, although it has a lower precision than SNIC due to the higher quantity of superpixels used for the same amount of memory.
• Figure 3: Results of downstream computer vision applications for other non-learning based superpixel algorithms. We show a snapshot of qualitative comparisons for SuperCam against memory restricted SNIC, ERS and SLIC. SuperCam performs better that all the other algorithms on all computer vision tasks.