All-optical convolutional neural network with on-chip integrable optical average pooling for image classification

Optical convolutional neural networks (OCNNs) have shown great potential with respect to bandwidth and power consumption. However, while the convolution operations have been implemented with various optical elements, the optical implementation of necessary pooling operations remains a challenging issue, which hinders the realization of all-optical CNNs. This paper proposes two types of optical architectures to perform the average pooling operation based on the singular value decomposition (SVD) and fast Fourier transform (FFT) algorithms, where the basic optical elements are Mach–Zehnder interferometers (MZIs). Then, the OCNN is constructed with these two pooling architectures embedded separately, in addition to an optical convolutional layer and a fully connected optical layer. With an ex situ training procedure, the OCNNs with either of these two pooling architectures exhibit a high classification accuracy of ∼98% on the MNIST dataset. When imprecision is introduced in the optical components, the component imprecision of the pooling layer has a much smaller impact on the OCNN’s classification accuracy than those of the other layers. This is especially true for the OCNN with the FFT pooling architecture, which exhibits stronger robustness to component imprecision. Furthermore, OCNNs with these two pooling architectures are trained separately on-chip. The results indicate that, when the component imprecisions of MZIs exceed a certain threshold (the standard deviation of phase noise >0.013), the on-chip trained OCNN exhibits significantly higher classification accuracy than the ex situ trained OCNN. Our proposed optical pooling architectures would contribute to the realization of all-optical CNN for further research.