A multi-head deep fusion model for recognition of cattle foraging events using sound and movement signals

TL;DR

This paper tackles automatic recognition of cattle foraging events by fusing acoustic signals and inertial movement data to detect jaw-movement events such as bite, chew, and chew-bite. It introduces a multi-head deep fusion architecture that performs feature-level fusion of modality-specific CNN branches, followed by a temporal RNN and dense classifier, achieving a micro F1-score of 0.802 and outperforming state-of-the-art unimodal methods. Through comprehensive fusion-level comparisons, window-size analyses, and an ablation study, the authors demonstrate the superiority of feature-level fusion with three heads (audio, accelerometer, and gyroscope) for JM-event recognition. The work advances precision livestock farming by enabling more accurate, robust, and scalable monitoring of individual cattle feeding behavior, with implications for diet optimization, health monitoring, and welfare assessment.

Abstract

Monitoring feeding behaviour is a relevant task for efficient herd management and the effective use of available resources in grazing cattle. The ability to automatically recognise animals' feeding activities through the identification of specific jaw movements allows for the improvement of diet formulation, as well as early detection of metabolic problems and symptoms of animal discomfort, among other benefits. The use of sensors to obtain signals for such monitoring has become popular in the last two decades. The most frequently employed sensors include accelerometers, microphones, and cameras, each with its own set of advantages and drawbacks. An unexplored aspect is the simultaneous use of multiple sensors with the aim of combining signals in order to enhance the precision of the estimations. In this direction, this work introduces a deep neural network based on the fusion of acoustic and inertial signals, composed of convolutional, recurrent, and dense layers. The main advantage of this model is the combination of signals through the automatic extraction of features independently from each of them. The model has emerged from an exploration and comparison of different neural network architectures proposed in this work, which carry out information fusion at different levels. Feature-level fusion has outperformed data and decision-level fusion by at least a 0.14 based on the F1-score metric. Moreover, a comparison with state-of-the-art machine learning methods is presented, including traditional and deep learning approaches. The proposed model yielded an F1-score value of 0.802, representing a 14% increase compared to previous methods. Finally, results from an ablation study and post-training quantization evaluation are also reported.

Figures (10)

• Figure 1: GRU cell diagram including the different gates and their connections.
• Figure 2: Proposed method architecture: input signals correspond to audio and movement chunks extracted using fixed length time windows. Each convolution layer shows the number of kernels and kernel size (ReLU was used as activiation function), whereas max pooling layers specify the filter size. Dense layers indicate the number of neurons and activation function. At each step the feature dimensions are given, L being the number of windows in the sequence.
• Figure 3: Illustration of the architectures for different fusion levels, where each level represents the configuration that reached the best results. a) data-level fusion; b) feature fusion with two independent CNN and feature concatenation; c) feature fusion with three independent CNN and feature concatenation (proposed model); d) decision fusion using an FNN for the final decision model. In all cases, the best results were obtained with a window size of 0.3 s.
• Figure 4: Satellite image of the dairy facilities detailing experimental paddock area, water source, surveillance camera position, and milking parlour.
• Figure 5: Experimentation setup description. A) Cow in the paddock during a rumination period with external microphone (1), halter (2), and plastic box (3). B) Moto G6 placed in a plastic box; C) axis from IMU sensors orientation: x-axis is aligned with a tail-to-head vector of the animal, y-axis describes sideway movements, whereas z-axis captures up and down movements.