Federated Knowledge Transfer Fine-tuning Large Server Model with Resource-Constrained IoT Clients
원문 초록
The training of large models, involving fine-tuning, faces the scarcity of high-quality data. Compared to the solutions based on centralized data centers, updating large models in the Internet of Things (IoT) faces challenges in coordinating knowledge from distributed clients by using their private and heterogeneous data. To tackle such a challenge, we propose KOALA (Federated Knowledge Transfer Fine-tuning Large Server Model with Resource-Constrained IoT Clients) to impel the training of large models in IoT. Since the resources obtained by IoT clients are limited and restricted, it is infeasible to locally execute large models and also update them in a privacy-preserving manner. Therefore, we leverage federated learning and knowledge distillation to update large models through collaboration with their small models, which can run locally at IoT clients to process their private data separately and enable large-small model knowledge transfer through iterative learning between the server and clients. Moreover, to support clients with similar or different computing capacities, KOALA is designed with two kinds of large-small model joint learning modes, namely to be homogeneous or heterogeneous. Experimental results demonstrate that compared to the conventional approach, our method can not only achieve similar training performance but also significantly reduce the need for local storage and computing power resources.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Domain Borders Are There to Be Crossed With Federated Few-Shot Adaptation
Geo-ORBIT: A Federated Digital Twin Framework for Scene-Adaptive Lane Geometry Detection
S2FGL: Spatial Spectral Federated Graph Learning
FedRef: Communication-Efficient Bayesian Fine Tuning with Reference Model
FedSC: Federated Learning with Semantic-Aware Collaboration
FedBKD: Distilled Federated Learning to Embrace Gerneralization and Personalization on Non-IID Data
토론
이 논문과 연결된 포럼 글이 아직 없습니다.