Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Multicore and Edge TPU-Accelerated Multimodal TinyML System for Livestock Behavior Recognition

Qianxue Zhang, Eiman Kanjo

TL;DR

This work tackles the need for low-cost, reliable livestock monitoring in Internet-poor farming environments by delivering a multicore, edge-deployed TinyML system. It fuses vision and accelerometer data through a late-fusion multimodal network and implements both object detection and behavior recognition on constrained microcontrollers, accelerated by Edge TPU. The system achieves high accuracy with substantially reduced model sizes, end-to-end latencies under 80 ms, and offline operation, while enabling wireless alerts to remote farmers. The study demonstrates practical viability and scalability for IoT-edge livestock management, with future work focusing on broader field deployments and longer temporal analyses.

Abstract

The advancement of technology has revolutionized the agricultural industry, transitioning it from labor-intensive farming practices to automated, AI-powered management systems. In recent years, more intelligent livestock monitoring solutions have been proposed to enhance farming efficiency and productivity. This work presents a novel approach to animal activity recognition and movement tracking, leveraging tiny machine learning (TinyML) techniques, wireless communication framework, and microcontroller platforms to develop an efficient, cost-effective livestock sensing system. It collects and fuses accelerometer data and vision inputs to build a multimodal network for three tasks: image classification, object detection, and behavior recognition. The system is deployed and evaluated on commercial microcontrollers for real-time inference using embedded applications, demonstrating up to 270$\times$ model size reduction, less than 80ms response latency, and on-par performance comparable to existing methods. The incorporation of the wireless communication technique allows for seamless data transmission between devices, benefiting use cases in remote locations with poor Internet connectivity. This work delivers a robust, scalable IoT-edge livestock monitoring solution adaptable to diverse farming needs, offering flexibility for future extensions.

A Multicore and Edge TPU-Accelerated Multimodal TinyML System for Livestock Behavior Recognition

TL;DR

This work tackles the need for low-cost, reliable livestock monitoring in Internet-poor farming environments by delivering a multicore, edge-deployed TinyML system. It fuses vision and accelerometer data through a late-fusion multimodal network and implements both object detection and behavior recognition on constrained microcontrollers, accelerated by Edge TPU. The system achieves high accuracy with substantially reduced model sizes, end-to-end latencies under 80 ms, and offline operation, while enabling wireless alerts to remote farmers. The study demonstrates practical viability and scalability for IoT-edge livestock management, with future work focusing on broader field deployments and longer temporal analyses.

Abstract

The advancement of technology has revolutionized the agricultural industry, transitioning it from labor-intensive farming practices to automated, AI-powered management systems. In recent years, more intelligent livestock monitoring solutions have been proposed to enhance farming efficiency and productivity. This work presents a novel approach to animal activity recognition and movement tracking, leveraging tiny machine learning (TinyML) techniques, wireless communication framework, and microcontroller platforms to develop an efficient, cost-effective livestock sensing system. It collects and fuses accelerometer data and vision inputs to build a multimodal network for three tasks: image classification, object detection, and behavior recognition. The system is deployed and evaluated on commercial microcontrollers for real-time inference using embedded applications, demonstrating up to 270 model size reduction, less than 80ms response latency, and on-par performance comparable to existing methods. The incorporation of the wireless communication technique allows for seamless data transmission between devices, benefiting use cases in remote locations with poor Internet connectivity. This work delivers a robust, scalable IoT-edge livestock monitoring solution adaptable to diverse farming needs, offering flexibility for future extensions.

Paper Structure

This paper contains 42 sections, 4 equations, 10 figures, 14 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. State-of-the-Art TinyML Models
  4. AI Applications in Farm Sensing
  5. Discussion
  6. System Design
  7. Structural Overview
  8. Technical Details and Methodologies
  9. Movement Tracking
  10. Activity Recognition
  11. Wireless Communication
  12. Embedded System Development
  13. TinyML Models Development
  14. Image Classification
  15. Model Design
  16. ...and 27 more sections

Figures (10)

  • Figure 1: Structural overview of the monitoring system prototype with two types of device.
  • Figure 2: Detailed system architecture with movement tracking, activity recognition and wireless communication function.
  • Figure 3: Architecture of the MCUNet model with MB block details.
  • Figure 4: Details of the operations in MCUNet MB blocks.
  • Figure 5: Best search space configurations $S^*$ under various SRAM and Flash settings.
  • ...and 5 more figures