Vendi Information Gain for Active Learning and its Application to Ecology

TL;DR

Presents Vendi information gain (VIG), a dataset-wide active learning policy for ecological image classification that uses MC dropout to sample plausible labels and measures reduction in the Vendi entropy $H_V$ across the unlabeled pool. VIG selects images that maximize information gain, enabling efficient labeling and improved generalization; on Snapshot Serengeti, it achieves about 75% accuracy with 150 labels and ~90% with 500 labels, outperforming standard baselines. The method is model-agnostic and applicable to data-limited monitoring tasks beyond ecology, with demonstrated gains in both predictive performance and data diversity. Overall, VIG offers a principled, scalable approach to leverage diversity-aware uncertainty for data-efficient ecological monitoring and beyond.

Abstract

While monitoring biodiversity through camera traps has become an important endeavor for ecological research, identifying species in the captured image data remains a major bottleneck due to limited labeling resources. Active learning -- a machine learning paradigm that selects the most informative data to label and train a predictive model -- offers a promising solution, but typically focuses on uncertainty in the individual predictions without considering uncertainty across the entire dataset. We introduce a new active learning policy, Vendi information gain (VIG), that selects images based on their impact on dataset-wide prediction uncertainty, capturing both informativeness and diversity. We applied VIG to the Snapshot Serengeti dataset and compared it against common active learning methods. VIG needs only 3% of the available data to reach 75% accuracy, a level that baselines require more than 10% of the data to achieve. With 10% of the data, VIG attains 88% predictive accuracy, 12% higher than the best of the baselines. This improvement in performance is consistent across metrics and batch sizes, and we show that VIG also collects more diverse data in the feature space. VIG has broad applicability beyond ecology, and our results highlight its value for biodiversity monitoring in data-limited environments.

Figures (6)

• Figure 1: Overview of Vendi information gain (VIG) for active learning. We use a trained dropout neural network to sample labels for a candidate datapoint. The neural network is then retrained on this fantasized data to sample labels of the entire pool. Uncertainty in these predictions is captured by VIG, and we select the candidate that yields the highest information gain (i.e., lowest uncertainty) in the predictions to label. The result is then added to the training dataset, and the process repeats until the labeling budget is exhausted.
• Figure 2: Average test accuracy ($\pm 1$ standard error) by various active learning policies. VIG obtains a large gain right from the start and maintains its lead throughout the active learning loop. It takes VIG only 150 datapoints to achieve the accuracy of 75% that other methods need 500 points to achieve. Meanwhile, at 500 points, VIG achieves close to 90% accuracy. In comparison, training on all available training data (5000+ images) yields an accuracy of 99%.
• Figure 3: The 5 most (top row) and 5 least (bottom row) accurate predictions by the model trained with data collected by VIG. In the bottom row, the model understandably makes mistakes on instances where the animal is barely visible.
• Figure 4: Diversity of the collected data by various active learning policies. Left: The Shannon entropy of the class distribution of the collected data. Here, all methods are comparable. Right: The Vendi score of the collected data using the embedding in the second-to-last layer of the neural network classifier trained on all available data. VIG selects more diverse data right from the beginning.
• Figure 5: Test accuracy by various active learning policies under different batch sizes. VIG's superior performance stays consistent in both low- and high-throughput settings, underscoring its robustness to selection frequency.