BatchNorm Layers have an Outsized Effect on Adversarial Robustness

Key Points

Adversarial robustness is influenced by how BatchNorm layers are optimized, showing variations across different strategies.
Alterations in the optimization techniques for BatchNorm layers yield distinct changes in clean accuracy when confronted with adversarial examples.
The approach includes a partial adversarial pre-training methodology for investigating BatchNorm layers within models.
Understanding the effect of BatchNorm layers on adversarial robustness could inform better model design choices for enhanced security.

Abstract

Training different layers differently may affect resulting adversarial robustness and clean accuracy in adversarial training. We focus on the BatchNorm layers and study their unique role in adversarial training. Through a partial adversarial (pre-)training methodology we investigate how different optimization strategies for the BatchNorm layers affect adversarial robustness, and interplay with other model design choices.

BatchNorm Layers have an Outsized Effect on Adversarial Robustness

Key Points

Abstract

Cite This Study