Efficient Structured Prediction with Latent Variables for General Graphical Models

In this paper we propose a unified framework for structured prediction with variables which includes hidden conditional random fields and latent support vector machines as special cases. We describe a local approximation for this general formulation using duality, and derive an message passing algorithm that is guaranteed to converge. We its effectiveness in the tasks of image segmentation as well as 3D scene understanding from single images, showing that our approach is to latent structured support vector machines and hidden conditional fields.