sesa 20(26): e2

Research Article

Aggregation for Privately Trained Different Types of Local Models

Download204 downloads
  • @ARTICLE{10.4108/eai.21-6-2021.170237,
        author={Chunling Han and Rui Xue},
        title={Aggregation for Privately Trained Different Types of Local Models},
        journal={EAI Endorsed Transactions on Security and Safety},
        volume={7},
        number={26},
        publisher={EAI},
        journal_a={SESA},
        year={2020},
        month={10},
        keywords={Secure aggregation, Generative adversarial network, Differential privacy},
        doi={10.4108/eai.21-6-2021.170237}
    }
    
  • Chunling Han
    Rui Xue
    Year: 2020
    Aggregation for Privately Trained Different Types of Local Models
    SESA
    EAI
    DOI: 10.4108/eai.21-6-2021.170237
Chunling Han1,2,*, Rui Xue1
  • 1: SKLOIS, Institute of Information Engineering, CAS; School of Cyber Security, University of Chinese Academy of Sciences
  • 2: Indiana University Bloomington
*Contact email: hanchun@iu.edu

Abstract

Machine learning has been a thriving topic in recent years, with many practical applications and active research aspects. In machine learning, model aggregation is an important area. The idea of model aggregation is to aggregate a global model from trained local models. However, traditional aggregation methods based on parameter averaging can not aggregate models which have different types and structures. Because parameter averaging will fail to average different types of values (parameters). To address this problem, we propose a new aggregation method which will suit for different types of local models. To achieve our goal, we transfer knowledge from local models to the global model. To do so, firstly, we propose differentially private GANs, let local parties generate synthetic data related to their training data. Secondly, we use the majority of prediction votes from local models to label those synthetic samples. Finally, use the labelled synthetic datato train the global model. By combining synthetic data and labels from local models, knowledge can be transferred from local models to the global model. We evaluate our scheme on Adult, MNIST and Fashion MNIST datasets under different settings, experimental results show that our scheme can achieve an accurate global model with low privacy loss. Besides, the easily implemented building blocks make our scheme efficient and practical for applications.