Ranging from prediction to planning via machine learning approaches for autonomous driving: a survey

Accurately predicting the future trajectories of surrounding agents and planning a safe, comfortable trajectory for the automated vehicle (AV) are of utmost importance for autonomous driving systems (ADS). In particular, the rise of machine learning-driven interactive prediction and planning frameworks has revolutionized this field. Existing autonomous driving systems, such as modular stacks and end-to-end frameworks, remain challenges as they commonly model the prediction and planning sequentially, neglecting the dynamic reactions of surrounding agents to the planning behaviors of the AV. Instead, interactive prediction and planning frameworks benefit from bi-directional interactive modeling between the AV and surrounding agents in a joint optimization manner, circumventing the drawbacks associated with existing autonomous driving systems and boosting safety and comfortable transportation. Through gradual modeling paradigms ranging from marginal prediction, conditional prediction, and interactive prediction and planning, this paper delves into fundamental principles and the growth journey of the ultimate target for autonomous driving. Built upon this, we systematically present a comprehensive review of state-of-the-art papers, encompassing the methodology, experimental platforms, and evaluation criteria during the development of interactive prediction and planning. Additionally, this paper details critical challenges concerning scene comprehension, planning safety, robustness, generalization and deployment. Accordingly, we discuss current advancements such as hot topics of large language models (LLMs), world models, distillation strategies, reinforcement learning from human feedback (RLHF), and data balance and long-tail learning, finally proposing future promising directions to inspire continued innovation and exploration.