Black box phase-based adversarial attacks on image classifiers

We propose a new method of utilizing a spatial light modulator to generate adversarial examples against image classifiers within a black box scenario. The method incorporates a simple-shape-focused strategy that queries the target network and estimates the effect of perturbing specific regions of the Fourier plane. This work is an extension of previous work that uses a spatial light modulator to perturb the phase of incoming light to generate adversarial patterns using l2-norm optimization. Our new method simply uses the final logits of the target network, allowing for it to be used not only in "white box" scenarios but also in the information-constrained "black box" scenarios. Our shape-based algorithm is shown to be widely effective on the original dataset benchmark without the requirement of knowledge about the target network architecture. Our experiments explore how manipulating the size, shape, number, and magnitude of the regions tested affects the efficacy and pattern cycles needed to generate a successful attack. Different combinations showed a range of average efficacy between 32% and 63% under a consistent objective function. Our new method also proved to be effective on a smaller dataset (meaning fewer classes for classification to be misdirected towards). We validate our method using a physical setup.