Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward Efficient Visual Gyroscopes: Spherical Moments, Harmonics Filtering, and Masking Techniques for Spherical Camera Applications

Yao Du, Carlos M. Mateo, Mirjana Maras, Tsun-Hsuan Wang, Marc Blanchon, Alexander Amini, Daniela Rus, Omar Tahri

TL;DR

A novel visual gyroscope is introduced, which combines an Efficient Multi-Mask-Filter Rotation Estimator (EMMFRE) and a Learning based optimization (LbTO) to provide a more efficient and accurate rotation estimation from spherical images.

Abstract

Unlike a traditional gyroscope, a visual gyroscope estimates camera rotation through images. The integration of omnidirectional cameras, offering a larger field of view compared to traditional RGB cameras, has proven to yield more accurate and robust results. However, challenges arise in situations that lack features, have substantial noise causing significant errors, and where certain features in the images lack sufficient strength, leading to less precise prediction results. Here, we address these challenges by introducing a novel visual gyroscope, which combines an Efficient Multi-Mask-Filter Rotation Estimator(EMMFRE) and a Learning based optimization(LbTO) to provide a more efficient and accurate rotation estimation from spherical images. Experimental results demonstrate superior performance of the proposed approach in terms of accuracy. The paper emphasizes the advantages of integrating machine learning to optimize analytical solutions, discusses limitations, and suggests directions for future research.

Toward Efficient Visual Gyroscopes: Spherical Moments, Harmonics Filtering, and Masking Techniques for Spherical Camera Applications

TL;DR

A novel visual gyroscope is introduced, which combines an Efficient Multi-Mask-Filter Rotation Estimator (EMMFRE) and a Learning based optimization (LbTO) to provide a more efficient and accurate rotation estimation from spherical images.

Abstract

Unlike a traditional gyroscope, a visual gyroscope estimates camera rotation through images. The integration of omnidirectional cameras, offering a larger field of view compared to traditional RGB cameras, has proven to yield more accurate and robust results. However, challenges arise in situations that lack features, have substantial noise causing significant errors, and where certain features in the images lack sufficient strength, leading to less precise prediction results. Here, we address these challenges by introducing a novel visual gyroscope, which combines an Efficient Multi-Mask-Filter Rotation Estimator(EMMFRE) and a Learning based optimization(LbTO) to provide a more efficient and accurate rotation estimation from spherical images. Experimental results demonstrate superior performance of the proposed approach in terms of accuracy. The paper emphasizes the advantages of integrating machine learning to optimize analytical solutions, discusses limitations, and suggests directions for future research.
Paper Structure (11 sections, 14 equations, 6 figures)

This paper contains 11 sections, 14 equations, 6 figures.

Table of Contents

  1. INTRODUCTION
  2. Related Works
  3. Contribution
  4. Methods
  5. From Spherical Harmonics to Spherical Moments
  6. Fast Implementation of Mask on Spherical Moments
  7. LbTO: Learning-based Triplet Optimizer
  8. Experiments
  9. Investigate the dimensions of the mask
  10. Comparison with Other Approaches
  11. Conclusion and Discussion

Figures (6)

  • Figure 1: Overview of the novel efficient visual gyroscope. The proposed visual gyroscope method consists of two key computation blocks: Efficient Multi-Mask-Filter Rotation Estimator (EMMFRE) and the Learning based optimization (LbTO), which conjunctively lead to a more accurate and efficient final image rotation estimation.
  • Figure 2: Image pre-processing for the analytical computation of triplets derived from spherical moments. First filtering and masking (total of 100 masks) are applied to the images, and then the spherical moments are computed before the feature triplets.
  • Figure 3: Masking method explained. (a) and (d) show the images on a half sphere and around the complete spherical surface. (b) presents an icosahedral sample delicately positioned on a half sphere. The figure (e) extends its scope to showcase the icosahedral sample enveloping the entire surface of a sphere. (c) and (f) introduce a mask on a sphere and an image overlaid with a mask.
  • Figure 4: Images generated from the Blender simulation environment. We generate 500 images for the training and 150 for the testing of the MLP, by randomly sampling the rotation metrics. (a)-(c) show three examples of the generated images.
  • Figure 5: Analysis of the impact of mask range on the accuracy of rotation estimations. The full analytical-LbTO method (fast visual gyroscope) is compared to the analytical method alone, to reveal optimal range value and significantly higher accuracy of our proposed analytical-LbTO method. (a) displays the comparison of error for various values of the range $r$ in a pure rotation sequence with half sphere images, (b) shows comparison of error for various values of the range $r$ in pure rotation sequence with whole sphere image.
  • ...and 1 more figures