DSAK: Distillation of Self-Adaptive Knowledge for Membership Privacy Protection

The utilization of machine learning models is extensive in a wide array of significant applications. However, their vulnerability to security and privacy attacks is a serious concern, for example, for the protection of financially sensitive data such as account flow. Particularly troubling is the threat of membership inference, which enables attackers to determine whether a given data sample is included in the training set of a targeted machine-learning model. Existing knowledge distillation techniques have shown promise in balancing model performance with data privacy. However, achieving superior privacy during the training process of the target model is challenging due to the teacher model’s performance limitations and the scarcity of unlabeled benchmark data. To address this issue, we propose a novel framework called Distillation of Self-Adaptive Knowledge (DSAK). DSAK utilizes self-duplicated teacher and noise-generative models to introduce specialized self-adaptive noise for privacy training in the target model. By incorporating new data features derived from this noise, DSAK improves model performance and reduces the risk of memorizing member data. Experimental results demonstrate DSAK’s effectiveness in defending against existing attack schemes across multiple datasets while surpassing other membership inference defense schemes in terms of efficiency.