Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding

TL;DR

This work addresses the need for large-scale, fine-grained insect understanding by introducing Insect-1M, a dataset with over 1 million images, six-level taxonomy labels, and descriptive text. It presents the Insect Foundation Model, featuring Patch-wise Relevant Attention and Description Consistency loss to capture micro-features and align image and text descriptions, achieving state-of-the-art results on IP102 classification and detection. Key contributions include the large-scale, richly labeled dataset, a micro-feature-focused self-supervised learning paradigm, and a vision-language alignment mechanism tailored to insects, all with clear benefits for precision agriculture. Overall, the approach advances foundation-model capabilities in a challenging domain with high ecological and agricultural relevance, enabling more accurate insect identification, monitoring, and management.

Abstract

In precision agriculture, the detection and recognition of insects play an essential role in the ability of crops to grow healthy and produce a high-quality yield. The current machine vision model requires a large volume of data to achieve high performance. However, there are approximately 5.5 million different insect species in the world. None of the existing insect datasets can cover even a fraction of them due to varying geographic locations and acquisition costs. In this paper, we introduce a novel "Insect-1M" dataset, a game-changing resource poised to revolutionize insect-related foundation model training. Covering a vast spectrum of insect species, our dataset, including 1 million images with dense identification labels of taxonomy hierarchy and insect descriptions, offers a panoramic view of entomology, enabling foundation models to comprehend visual and semantic information about insects like never before. Then, to efficiently establish an Insect Foundation Model, we develop a micro-feature self-supervised learning method with a Patch-wise Relevant Attention mechanism capable of discerning the subtle differences among insect images. In addition, we introduce Description Consistency loss to improve micro-feature modeling via insect descriptions. Through our experiments, we illustrate the effectiveness of our proposed approach in insect modeling and achieve State-of-the-Art performance on standard benchmarks of insect-related tasks. Our Insect Foundation Model and Dataset promise to empower the next generation of insect-related vision models, bringing them closer to the ultimate goal of precision agriculture.

Figures (8)

• Figure 1: Our Proposed Patch-wise Relevant Attention. Given masked insect images and separated image patches, our model can discriminate these patches that have small differences via relevant scores computed between masked images and image patches.
• Figure 2: Examples of Our Insect-1M Dataset. The left figure illustrates the samples of the four Subphylums, including Chelicerata, Crustacea, Hexapoda, and Myriapoda. The right figure shows an example of hierarchical descriptions of the Aurantia Species.
• Figure 3: The Distribution of Subphylum and Its Classes (Left) and The Distribution of Class and Its Orders (Right). Best viewed in color.
• Figure 4: Comparisons of Self-supervised Methods. MAE he2022masked fails to reconstruct the details of the insect since it learns general information about the image. Micron-BERT nguyen2023micron hardly distinguishes the insect and background. Jigsaw-ViT chen2023jigsaw cannot correct shuffled patches due to confusion between the background and the object. Meanwhile, our approach can find separated patches belonging to the insect by scoring each patch. Best viewed in color.
• Figure 5: The Overview Framework of Our Proposed Approach to Insect Foundation Model.