PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
October 20, 20250 citationsOpen Access

Perceive, Reflect and Understand Long Video: Progressive Multi-Granular Clue Exploration with Interactive Agents

View Full Paper
JLJiahua LiKWKun WeiZXZhe Xu

Key Points

  • CogniGPT efficiently identifies task-related clues in long videos, improving understanding and analysis.
  • It utilizes a Multi-Granular Perception Agent to interpret key information and a Verification-Enhanced Reflection Agent to ensure accuracy.
  • Experiments on various datasets demonstrate CogniGPT's significant advancements over traditional methods.
  • Notably, on EgoSchema, it uses only 11.2 frames to achieve high performance, showcasing reduced resource requirements.

Abstract

Long videos, characterized by temporal complexity and sparse task-relevant information, pose significant reasoning challenges for AI systems. Although various Large Language Model (LLM)-based approaches have advanced long video understanding, they still struggle to achieve both completeness and efficiency in capturing task-critical information. Inspired by human progressive visual cognition, we propose CogniGPT, a framework that leverages an interactive loop between Multi-Granular Perception Agent (MGPA) and Verification-Enhanced Reflection Agent (VERA) for efficient and reliable long video understanding. Specifically, MGPA mimics human visual divergent and focused attention to capture task-related information, while VERA verifies perceived key clues to mitigate hallucination and optimize subsequent perception strategies. Through this interactive process, CogniGPT explores a minimal set of informative and reliable task-related clues. Extensive experiments on EgoSchema, Video-MME, NExT-QA, and MovieChat datasets demonstrate CogniGPT's superiority in both accuracy and efficiency. Notably, on EgoSchema, it surpasses existing training-free methods using only 11.2 frames and achieves performance comparable to Gemini 1.5-Pro.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Li et al. (2025) studied this question.

synapsesocial.com/papers/68f5fcce8d54a28a75cf1d86https://doi.org/10.48550/arxiv.2509.24943
Ask AI
Helpful
Bookmark
Share
View Full Paper