Layered Distillation Training: A Study of Adversarial Attacks and Defenses

While extensive research has been done on adversarial defense techniques in the past, layering of defense techniques to make the defense more robust is an area this paper will delve into. All the attacks and defenses have been implemented on the MNIST dataset and tested against two models, a CNN model, and an XG Boost model. A few of the most popular adversarial attacks, such as FGSM (Fast Gradient Sign Method), DeepFool Method, PGD (Projected Gradient Descent), and I-FGSM (Iterated Fast Gradient Sign Method) are implemented in this paper to test the adversarial defenses against. The adversarial defenses that are tested individually are Adversarial Training and APE-GAN. Adversarial Training layered with Defensive Distillation proves to improve the accuracy of the CNN Model that has been attacked by the FGSM method, from 59.97% with a loss of 5.810 to 99.18% with a loss of 0.027. This is a significant improvement, and on comparing with adversarial training without layering, the layered defense has higher accuracy by 2% (Post Adversarial Training Accuracy: 97.18% and Post Layered Distillation Training: 99.18%) and lower loss by 0.14 (Loss Post Adversarial Training: 0.167 and Loss Post Layered Distillation Training: 0.027).