Edge-FIT: Federated Instruction Tuning of Quantized LLMs for Privacy-Preserving Smart Home Environments
Vinay Venkatesh, Vamsidhar R Kamanuru, Lav Kumar, Nikita Kothari
TL;DR
Edge-FIT addresses privacy-preserving training of large language models on IoT data by integrating Federated Learning with 4-bit QLoRA. The approach reduces both communication and local compute requirements, enabling a single global IoT-domain model to be learned across edge gateways. Empirical results show near-centralized performance for Llama 2-7B (0.89 F1 vs 0.93 centralized) and viable results for Phi-3-mini (0.80 vs 0.81 centralized), validating a scalable edge AI blueprint for smart homes. This work demonstrates a practical path to deploying privacy-preserving LLMs on budgeted edge hardware while maintaining strong utility.
Abstract
This paper proposes Edge-FIT (Federated Instruction Tuning on the Edge), a scalable framework for Federated Instruction Tuning (FIT) of Large Language Models (LLMs). Traditional Federated Learning (TFL) methods, like FedAvg, fail when confronted with the massive parameter size of LLMs [3], [6]. Our Edge-FIT framework combines federated learning with 4-bit Quantized Low-Rank Adaptation (QLORA), mitigating the core issues of communication and computational overhead. We demonstrate this by filtering the general-purpose Databricks Dolly 15k dataset for the IoT domain. Experimental results show the Edge-FIT tuned Llama 2(7B) achieves an F1-Score of 0.89. We also demonstrate a viable trade-off using the 3.8B Phi-3-mini model, validating Edge-FIT as a scalable framework for decentralized LLM deployment on home compute gateways.