Intra-Matrix Flexible Optical-Electronic Configuration Algorithms for Energy-Efficient Large Language Model Inference

Low-energy, low-precision inference is an important enabler for scalable deployment of large language models (LLMs). Optical-electronic computing offers high bandwidth, low latency, and potentially high energy efficiency, making it a promising approach for future LLM deployment. However, existing optical and hybrid optical-electronic inference systems typically use uniform hardware configurations, such as fixed optical power or a fixed electronic-photonic partition across layers. Fine-grained allocation of optical power and optical-electronic computation remains underexplored for photonic LLM inference. Here, we propose three flexible configuration algorithms: Flexible Optical Power Allocation (FOPA), Flexible Weak-Column Allocation (FWCA), and Joint Optical Power and Weak-Column Allocation (JOWA). Using a diagonal Hessian estimator without constructing the full Hessian matrix, these methods jointly optimize optical-electronic resources under predefined energy budgets. Evaluations on Llama 2-7B and DeepSeek-LLM-7B-Base show estimated energy savings of 40.11%–60.85% relative to FP16 inference on the NVIDIA H100 and 5.34%–38.12% relative to uniform optical-power and weak-column baselines, while maintaining an acceptable perplexity increase. Evaluations using a Tianshu-hardware-calibrated simulator show corresponding estimated energy savings of 39.91%–53.67% and 5.02%–26.76%, respectively. These results indicate that fine-grained resource allocation can improve the energy-accuracy trade-off of hybrid optical-electronic systems for LLM inference.