FedGCS: Addressing Class Imbalance in Long-Tail Federated Learning

Federated learning is a privacy-preserving distributed machine learning method, which facilitates clients to cooperate in training a shared model while safeguarding original data. The different distribution and quantity of training data between clients can pose significant challenges, such as data heterogeneity and class imbalance, which can greatly influence the performance of the shared model. Although many methods have been proposed to eliminate the deleterious influence of non-IID data, existing solutions usually do not perform well on tail-classes owing to the absence of attention for the long-tail distribution. We present a long-tail federated learning framework FedGCS, which can solve the global and local class imbalance problem via generic to compensate for specific. Specifically, clients separate features from the training data based on the class activation map and selectively fuse the separated class-specific features and class-generic features to restore the distribution of tail-classes. We also design a loss function—TailDistillation Loss to lessen the bias of the classifier towards head-classes. To appraise the effectiveness of FedGCS, we adapted multiple benchmark datasets to the long-tail federated learning setting. Experiments indicate that the FedGCS is an useful method, and is superior to previous approaches.