What type of study is this?

This is a Quantitative Study study.

October 16, 2025Open Access

ViPro-2: Unsupervised State Estimation via Integrated Dynamics for Guiding Video Prediction

PTPatrick TakenakaStuttgart Media University JMJohannes MaucherUniversity of Stuttgart MHMarco F. HuberFraunhofer Institute for Manufacturing Engineering and Automation

Key Points

ViPro-2 successfully infers states from observations, improving video frame predictions in challenging contexts.
The model addresses noise in previous states, allowing accurate state estimations without full ground truth provided initially.
Improvements include unsupervised mechanisms, enhancing interactions with the observed environment and dynamical settings.
The extended Orbits dataset now includes a 3D variant, bridging the gap to real-world video prediction scenarios.

Abstract

Predicting future video frames is a challenging task with many downstream applications. Previous work has shown that procedural knowledge enables deep models for complex dynamical settings, however their model ViPro assumed a given ground truth initial symbolic state. We show that this approach led to the model learning a shortcut that does not actually connect the observed environment with the predicted symbolic state, resulting in the inability to estimate states given an observation if previous states are noisy. In this work, we add several improvements to ViPro that enables the model to correctly infer states from observations without providing a full ground truth state in the beginning. We show that this is possible in an unsupervised manner, and extend the original Orbits dataset with a 3D variant to close the gap to real world scenarios.

Ask AI

Helpful

Bookmark

View Full Paper