Augmented-Reality Enabled Crop Monitoring with Robot Assistance

TL;DR

This work aims to address the gap in real-time data input and control output of a mobile robot for precision agriculture through a virtual environment enabled by an AR headset interface by providing practical solutions for real-time data management and control enabled by human-robot interaction.

Abstract

The integration of augmented reality (AR), extended reality (XR), and virtual reality (VR) technologies in agriculture has shown significant promise in enhancing various agricultural practices. Mobile robots have also been adopted as assessment tools in precision agriculture, improving economic efficiency and productivity, and minimizing undesired effects such as weeds and pests. Despite considerable work on both fronts, the combination of a versatile User Interface (UI) provided by an AR headset with the integration and direct interaction and control of a mobile field robot has not yet been fully explored or standardized. This work aims to address this gap by providing real-time data input and control output of a mobile robot for precision agriculture through a virtual environment enabled by an AR headset interface. The system leverages open-source computational tools and off-the-shelf hardware for effective integration. Distinctive case studies are presented where growers or technicians can interact with a legged robot via an AR headset and a UI. Users can teleoperate the robot to gather information in an area of interest, request real-time graphed status of an area, or have the robot autonomously navigate to selected areas for measurement updates. The proposed system utilizes a custom local navigation method with a fixed holographic coordinate system in combination with QR codes. This step toward fusing AR and robotics in agriculture aims to provide practical solutions for real-time data management and control enabled by human-robot interaction. The implementation can be extended to various robot applications in agriculture and beyond, promoting a unified framework for on-demand and autonomous robot operation in the field.

Figures (13)

• Figure 1: Snapshots of a controlled reaching task supported by our pediatric assistive wearable device utilizing the novel soft actuators developed in this work. The accompanying video offers more visual information and contains footage regarding the experiments conducted herein.
• Figure 2: Two types of soft actuators used in the wearable device: (a) shoulder abduction/adduction (b) shoulder flexion/extension.
• Figure 3: (Left) Coordinate frames for motion capture motion analysis, (center) adaptable soft links for the mannequin, with known (user-adjusted) added weights, and (right) back support for shoulder F/E motion.
• Figure 4: Soft actuators on the wearable device. (Left) Shoulder adduction and abduction with range of motion $\theta_{1_{si}}=10^o$ to $\theta_{1_{sf}} = 80^o$ and (right) bellow type soft actuator for the shoulder flexion and extension, with the added support and maximum range of motion $\theta_{2_{si}}=10^o$ to $\theta_{2_{sf}} = 32^o$.
• Figure 5: Shoulder range of motion considering both typically developed and diskynetic cerebral palsy children vanmechelen2022psychometric with shoulder F/E actuator's range of motion indicated between the red dashed lines.