Recent advances in large language models (LLMs) and generative AI have demonstrated impressive capabilities across a wide range of tasks, achieving human-level performance on many academic and professional benchmarks. However, the current deep learning paradigm faces fundamental limitations that hinder the sustainable development of AI systems. These include enormous computational and environmental costs of training, the persistence of hallucinations, lack of interpretability, and catastrophic forgetting when models are sequentially updated. This work analyzes three interconnected foundational concepts essential for building more robust and sustainable AI: knowledge acquisition (learning paradigms), knowledge retention (long-term memory and model stability), and knowledge organization (structure, logic, and interpretability). We provide a structured analysis of the strengths and limitations of existing approaches — from supervised, unsupervised, and reinforcement learning to continual learning techniques, Retrieval-Augmented Generation (RAG), knowledge graphs, Chain-of-Thought prompting, and neuro-symbolic architectures. The analysis highlights that overcoming the limitations of today’s scaling-focused paradigm requires a shift toward hybrid systems that integrate neural learning with explicit, modular, and symbolically grounded knowledge representations. These findings suggest promising research directions, including the integration of RAG with knowledge graphs and the development of architectures with explicit knowledge structuring, which may form the basis for more adaptive, interpretable, and environmentally sustainable AI systems.
Lev M. Sakovykh (Sat,) studied this question.