From Message Passing to Prompting: Rethinking Graph Learning with Large Language Models

Graph Neural Networks (GNNs) and Large Language Models (LLMs) have each emerged as dominant paradigms in machine learning, excelling respectively in structured relational reasoning and language-based generalization. GNNs are uniquely suited for learning over graph-structured data by leveraging local connectivity and relational inductive biases, finding widespread application in domains such as chemistry, social network analysis, recommendation systems, and knowledge graphs. Meanwhile, LLMs, particularly those built upon the Transformer architecture, have demonstrated remarkable capabilities in understanding and generating human language, enabling a wide range of tasks from open-domain question answering to code synthesis and multi-step reasoning. As LLMs continue to scale and exhibit emergent abilities, their application to non-sequential data types, including graphs, has gained growing interest within the research community. This survey presents a comprehensive examination of the evolving relationship between GNNs and LLMs, analyzing how these two powerful yet fundamentally different approaches can be integrated to achieve more expressive, generalizable, and semantically rich models.We begin by establishing the mathematical foundations of GNNs and LLMs, discussing their respective architectures, learning paradigms, and representational capacities. We then explore recent advances that bridge the gap between graph and language modalities, including strategies for graph serialization, hybrid model design, and multimodal pretraining. Through an in-depth comparative analysis, we highlight the strengths and limitations of each paradigm, with special attention to scalability, interpretability, and task adaptability. A taxonomy of hybrid architectures is presented, alongside illustrative use cases in biomedical informatics, scientific discovery, recommendation, and knowledge-intensive natural language processing. The survey also identifies key challenges in representation alignment, training efficiency, benchmark design, and model robustness, and offers a forward-looking perspective on future directions. These include the development of graph-native language models, few-shot reasoning over structured data, causal inference across modalities, and the ethical deployment of dual-modality AI systems. Ultimately, we argue that the fusion of GNNs and LLMs represents a promising path toward more holistic and versatile machine learning frameworks, capable of bridging symbolic structure and linguistic understanding in ways that neither can achieve alone.