What type of study is this?

September 10, 2025

A Dual-Teacher and Multi-Scale Adversarial Example-Based Framework for Adversarial Distillation

Key Points

DTAD-MS improves weighted robust accuracy by up to 2.0% on CIFAR-10, indicating enhanced model robustness.
Experimental results demonstrate improved performance on ResNet-18 and MobileNet-V2 datasets CIFAR-10 and CIFAR-100.
This approach utilizes multi-scale adversarial examples and attention-guided distillation to enhance feature learning and accuracy.
The dual-teacher strategy balances clean accuracy and robustness, addressing common pitfalls in adversarial training.

Abstract

Deep neural networks (DNNs) are vulnerable to adversarial attacks, while conventional adversarial training often degrades accuracy on clean examples and lacks sensitivity to specific-scale perturbations. This paper introduces DTAD-MS, a novel adversarial distillation framework using dual teachers and multi-scale adversarial examples to address these limitations. DTAD-MS distinctively integrates three components: 1) multi-scale adversarial example generation and fusion, leveraging multi-scale feature extraction and deep attention to bolster robustness against varied perturbation scales; 2) a dual-teacher strategy where a clean teacher preserves clean accuracy and a robust teacher enhances adversarial robustness, balanced by an adaptive loss function; and 3) attention-guided distillation, using the clean teacher’s attention maps to direct the student model’s focus on critical features, thereby improving robust feature assimilation. Experimental results on CIFAR-10/100 demonstrate that DTAD-MS improves Weighted Robust Accuracy (WRobustAcc) by up to 2. 0% (CIFAR-10) and 1. 5% (CIFAR-100) for ResNet-18, and by 0. 5% (CIFAR-10) and 3. 0% (CIFAR-100) for MobileNet-V2, achieving an effective balance.

Demander à l'IA

Bookmark