Formal reasoning has long been fundamental to various fields of computer science, including AI in its early days. In contrast, large language models (LLMs)—the cornerstone of modern AI—perform reasoning through autoregressive next-word prediction, without grounding their outputs in formal systems. This “informal” approach can learn world knowledge and reasoning patterns from large-scale data without rigid formalization. It offers significantly greater flexibility than traditional formal reasoning and has achieved promising results on many benchmarks. However, LLMs heavily rely on data and do not guarantee the soundness of reasoning. In this article, we highlight recent efforts to integrate modern LLMs with formal methods, an approach that seeks to harness the strengths of both paradigms. Such integration has the potential to lead to major advancements in AI-driven mathematics, formal verification, and the verifiable generation of computer systems.
Yang et al. (Tue,) studied this question.