Efficient Image Representation Learning with Federated Sampled Softmax
원문 초록
Learning image representations on decentralized data can bring many benefits in cases where data cannot be aggregated across data silos. Softmax cross entropy loss is highly effective and commonly used for learning image representations. Using a large number of classes has proven to be particularly beneficial for the descriptive power of such representations in centralized learning. However, doing so on decentralized data with Federated Learning is not straightforward as the demand on FL clients' computation and communication increases proportionally to the number of classes. In this work we introduce federated sampled softmax (FedSS), a resource-efficient approach for learning image representation with Federated Learning. Specifically, the FL clients sample a set of classes and optimize only the corresponding model parameters with respect to a sampled softmax objective that approximates the global full softmax objective. We examine the loss formulation and empirically show that our method significantly reduces the number of parameters transferred to and optimized by the client devices, while performing on par with the standard full softmax method. This work creates a possibility for efficiently learning image representations on decentralized data with a large number of classes under the federated setting.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
CorrFL: Correlation-based Neural Network Architecture for Unavailability Concerns in a Heterogeneous IoT Environment
Describe Anything Model for Visual Question Answering on Text-rich Images
Assay2Mol: large language model-based drug design using BioAssay context
FLsim: A Modular and Library-Agnostic Simulation Framework for Federated Learning
Domain Borders Are There to Be Crossed With Federated Few-Shot Adaptation
Dual Dimensions Geometric Representation Learning Based Document Dewarping
토론
이 논문과 연결된 포럼 글이 아직 없습니다.