What is the fastest Artificial Intelligence Large Language Model (AI LLM) for generating quantum operations? To answer this, we present the first benchmarking study comparing popular and publicly available AI models tasked with creating quantum gate designs. The Wolfram Mathematica framework was used to interface with the six AI LLMs, including Google Gemini 2.0 Flash, Anthropic Claude 3 Haiku, WolframLLM Notebook Assistant For Mathematica V14.3.0.0, OpenAI ChatGPT Omni 4 Mini, Google Gemma 3 4b 1t, and DeepSeek Chat V3. Our novel study found the following: (1) Gemini 2.0 Flash is overall the fastest AI LLM of the models tested in producing average quantum gate designs at 2.66101 s, factoring in the “thinking” execution time and ServiceConnect network latencies. (2) On average, four out of the ten quantum operations that the six LLMs produced compiled in Python (40.8% success rate). (3) Quantum operations averaged approximately 21–45 Lines of Code (omitting nonsensical outliers). (4) DeepSeek Chat V3 produced the shortest code with an average of 21.6 lines. This comparison evaluates the time taken by each AI LLM platform to generate quantum operations (including ServiceConnect networking times). These findings highlight a promising horizon where publicly available Large Language Models can become fast collaborators with quantum computers, enabling rapid quantum gate synthesis and paving the way for greater interoperability between two remarkable and cutting-edge technologies.
Closser et al. (Mon,) studied this question.