BVE + EKF: A viewpoint estimator for the estimation of the object's position in the 3D task space using Extended Kalman Filters

TL;DR

This work tackles the challenge of robust 3D object position estimation in open-field environments using monocular vision by introducing a Gaussian viewpoint estimator (BVE) powered by an Extended Kalman Filter (EKF). The method fuses multiple Gaussian observations through a product-of-Gaussians framework, leverages a camera-to-world rotation, and optimizes viewpoint with differentiable loss functions that minimize observability uncertainty. Two loss functions are studied—covariance dispersion and maximum eigenvalue of the updated covariance—along with an improved loss that guides the camera toward the fruit, all within carefully designed spatial and view-cone restrictions. In MATLAB simulations with Gaussian noise, the approach achieves approximately $32~\text{mm}$ average error, demonstrating potential for monocular, robot-assisted harvesting while highlighting trade-offs between restriction tightness, computation, and accuracy; future work includes robotic realization and exploring UKF as an alternative to EKF.

Abstract

RGB-D sensors face multiple challenges operating under open-field environments because of their sensitivity to external perturbations such as radiation or rain. Multiple works are approaching the challenge of perceiving the 3D position of objects using monocular cameras. However, most of these works focus mainly on deep learning-based solutions, which are complex, data-driven, and difficult to predict. So, we aim to approach the problem of predicting the 3D objects' position using a Gaussian viewpoint estimator named best viewpoint estimator (BVE) powered by an extended Kalman filter (EKF). The algorithm proved efficient on the tasks and reached a maximum average Euclidean error of about 32 mm. The experiments were deployed and evaluated in MATLAB using artificial Gaussian noise. Future work aims to implement the system in a robotic system.

Figures (7)

• Figure 1: Definition of the camera's frame and the main frame.
• Figure 2: Restriction---fruit inside the camera's field of view
• Figure 3: Restriction---camera cannot cross the fruits' trees
• Figure 4: Sample paths generated by the different experiments to assess the fruit's position. Light blue -- poses; red -- real fruit position; green -- estimated fruit position.
• Figure 5: Average error for the recoverability of the loss functions for the bve + ekf considering different initial estimation errors. Blue -- rmse; red -- mse; yellow -- mae