This study aims to comparatively analyze the performance of three reinforcement learning algorithms-DQN, QT-Opt, and Samuel’s checkers algorithm-on the symbolic matrix multiplication task. The experiments were conducted using a customized simulation environment, MatrixMultiplyDiscoveryEnv, where each agent generates outer product-based symbolic actions to perform matrix multiplication with minimal error and computational cost. The reward function incorporates the Frobenius norm, operation count, and symbolic complexity. Based on 50,000 episodes, the QT-Opt algorithm demonstrated a highly stable reward profile, maintaining reward values close to zero throughout training. Samuel’s algorithm showed rapid early learning, improving from -300 to around -100, but exhibited fluctuations in the later stages. In contrast, DQN’s reward varied drastically, occasionally falling below -3000, indicating instability and sensitivity to environmental uncertainty. Regarding matrix error (Frobenius norm), Samuel’s algorithm minimized its error to nearly zero in early training and maintained this performance. QT-Opt also performed well but showed occasional spikes in error. In terms of operation cost, QT-Opt consistently operated within 50-100 units, showing the highest efficiency. Samuel started with costs near 300, but reduced them gradually, converging towards QT-Opt’s performance. DQN, however, showed wide and erratic cost distributions. In conclusion, QT-Opt achieved the most stable and efficient learning, particularly in continuous action domains. This paper provides a unique perspective by comparing classical and modern reinforcement learning methods within a unified experimental framework, highlighting both their historical significance and practical performance.
Özkurt et al. (Fri,) studied this question.