DevPiolt: Operation Recommendation for IoT Devices at Xiaomi Home

TL;DR

DevPiolt addresses the challenge of recommending multi-device IoT operations by leveraging an LLM-based framework that first grounds the model in IoT operation knowledge through pre-training on historical actions, then specializes it via fine-tuning with annotated data, followed by direct preference optimization to tailor suggestions to individual users. A confidence-based exposure control mechanism mitigates suboptimal recommendations by gating exposure based on estimated confidence. Across extensive Xiaomi Home datasets and real-world deployment to 255k daily users, DevPiolt significantly outperforms baselines on multiple metrics and delivers notable online gains in device coverage and suggestion acceptance, illustrating strong practical impact for IoT operation management.

Abstract

Operation recommendation for IoT devices refers to generating personalized device operations for users based on their context, such as historical operations, environment information, and device status. This task is crucial for enhancing user satisfaction and corporate profits. Existing recommendation models struggle with complex operation logic, diverse user preferences, and sensitive to suboptimal suggestions, limiting their applicability to IoT device operations. To address these issues, we propose DevPiolt, a LLM-based recommendation model for IoT device operations. Specifically, we first equip the LLM with fundamental domain knowledge of IoT operations via continual pre-training and multi-task fine-tuning. Then, we employ direct preference optimization to align the fine-tuned LLM with specific user preferences. Finally, we design a confidence-based exposure control mechanism to avoid negative user experiences from low-quality recommendations. Extensive experiments show that DevPiolt significantly outperforms baselines on all datasets, with an average improvement of 69.5% across all metrics. DevPiolt has been practically deployed in Xiaomi Home app for one quarter, providing daily operation recommendations to 255,000 users. Online experiment results indicate a 21.6% increase in unique visitor device coverage and a 29.1% increase in page view acceptance rates.

Figures (8)

• Figure 1: The workflow of device operation recommendation in Xiaomi Home app.
• Figure 2: The process of constructing operation dataset.
• Figure 3: The overview of DevPiolt.
• Figure 4: An operation dataset sample. The black text represents the operation recommendation prompt $\mathcal{P}$, and the italicized part indicates the target used for loss calculation. The pre-training only includes operation history, while fine-tuning additionally includes environment and device data.
• Figure 5: The acceptance rate of curtain operation recommendations by users at different times of the day.