WildFit: Autonomous In-situ Model Adaptation for Resource-Constrained IoT Systems

TL;DR

This work tackles the challenge of domain shifts in resource-constrained IoT by proposing WildFiT, an autonomous on-device adaptation framework for camera-trap wildlife monitoring. It introduces background-aware data synthesis to generate domain-adjusted training data and drift-aware fine-tuning to trigger updates only when necessary, using drift detection (background and class distribution) and drift validation on synthesized samples. The approach yields Pareto-optimal tuning behavior, reduces unnecessary updates, and delivers end-to-end accuracy gains (up to ~35% over domain-adaptation baselines) with exceptional energy efficiency (11.2 Wh over 37 days) and millisecond-scale synthesis on edge devices. WildFiT demonstrates practical, battery-friendly deployment for continuous, autonomous adaptation in dynamic environmental conditions, with broad applicability to other resource-constrained IoT tasks facing similar domain shifts.

Abstract

Resource-constrained IoT devices increasingly rely on deep learning models, however, these models experience significant accuracy drops due to domain shifts when encountering variations in lighting, weather, and seasonal conditions. While cloud-based retraining can address this issue, many IoT deployments operate with limited connectivity and energy constraints, making traditional fine-tuning approaches impractical. We explore this challenge through the lens of wildlife ecology, where camera traps must maintain accurate species classification across changing seasons, weather, and habitats without reliable connectivity. We introduce WildFit, an autonomous in-situ adaptation framework that leverages the key insight that background scenes change more frequently than the visual characteristics of monitored species. WildFit combines background-aware synthesis to generate training samples on-device with drift-aware fine-tuning that triggers model updates only when necessary to conserve resources. Our background-aware synthesis surpasses efficient baselines by 7.3% and diffusion models by 3.0% while being orders of magnitude faster, our drift-aware fine-tuning achieves Pareto optimality with 50% fewer updates and 1.5% higher accuracy, and the end-to-end system outperforms domain adaptation approaches by 20-35% while consuming only 11.2 Wh over 37 days-enabling battery-powered deployment.

Figures (8)

• Figure 1: Spatial and temporal domain shifts in camera trap applications. Specifically, Location 2's camera position shifted over time while Location 1 and 3's backgrounds show seasonal changes. As we show in Table \ref{['tab:generalization_problem']}, these domain shift can cause an 9% - 60% drop in wildlife recognition accuracy.
• Figure 2: The trade-off between model complexity and generalization performance across (a) spatial and (b) temporal domain shifts, evaluated on five EfficientNet models (B0-B4) on the representative location of camera trap dataset lila_datasets. Fine-tuning is an effective approach for adapting models to new domains across varying model sizes.
• Figure 3: Overview of WildFiT. The system runs a lightweight classification model on the IoT device and maintains accuracy under domain shifts through Drift-Aware Fine-Tuning, which uses Background Drift Detection and Class Distribution Drift Detection to identify domain shifts, validates their impact in the Drift Validation module, and triggers on-device model adaptation using Background-Aware Data Synthesis.
• Figure 4: The background-aware data synthesis. It illustrates two images produced from the Synthesizer. The animal objects repository are frozen on IoT devices.
• Figure 5: Class distribution changes across time (location U10 from Serengeti S4 dataset)