Ultra-compact multi-task processor based on in-memory optical computing

Abstract To enhance the computational density and energy efficiency of on-chip neuromorphic hardware, this study introduces a novel network architecture for multi-task processing with in-memory optical computing. On-chip optical neural networks are celebrated for their capability to transduce a substantial volume of parameters into optical form while conducting passive computing, yet they encounter challenges in scalability and multitasking. Leveraging the principles of transfer learning, this approach involves embedding the majority of parameters into fixed optical components and a minority into adjustable electrical components. Furthermore, with deep regression algorithm in modeling physical propagation process, a compact optical neural network achieve to handle diverse tasks. In this work, two ultra-compact in-memory diffraction-based chips with integration of more than 60,000 parameters/mm 2 were fabricated, employing deep neural network model and the hard parameter sharing algorithm, to perform multifaceted classification and regression tasks, respectively. The experimental results demonstrate that these chips achieve accuracies comparable to those of electrical networks while significantly reducing the power-intensive digital computation by 90%. Our work heralds strong potential for advancing in-memory optical computing frameworks and next generation of artificial intelligence platforms.