DPVOC: Dual-Prompt for Learned Variable Bitrate Omnidirectional Image Compression

With the widespread adoption of consumer electronic devices such as virtual reality (VR) headsets, panoramic cameras, and ultra-high-definition displays, omnidirectional (360°) images have become increasingly important for providing immersive user experiences. However, the high resolution and data volume of these images pose significant challenges for bandwidth-limited and resource-constrained consumer electronics. To address these challenges, based on an advanced parallel dual-branch hybrid architecture (TCM) consisting of convolutional neural networks (CNNs) and Swin Transformer, we propose a dual-prompt learned variable bitrate omnidirectional image compression framework, termed DPVOC, which utilizes distortion maps (Dmaps) and quality maps (Qmaps) as dual prompts to enable region-adaptive bit allocation and achieve efficient variable bitrate compression. Specifically, during training, to alleviate the computational burden of processing entire ERP images, we randomly crop ERP images into patches as input to the network. Considering the varying degrees of distortion redundancy across different regions of ERP patches, we introduce corresponding Dmap patches to record the local distortion levels. In the CNN branch, the patch-wise uniform Qmaps are element-wise multiplied with the Dmaps to modulate the CNN features. In the Swin Transformer branch, the uniform Qmap patches are used as prompts in the attention mechanism to guide the feature embeddings for adaptability to bitrate variations. Additionally, Dmap patches are introduced into the feedforward network (FFN) of the Swin Transformer to suppress redundant information. By incorporating fine-grained and symmetric prompts from both Qmaps and Dmaps into the encoder and decoder through the dual-branch structure, our networks can effectively adapt to diverse bitrate requirements. During inference, entire Qmaps and Dmaps are used as inputs, and their bitrate overhead is negligible. Experimental results demonstrate that DPVOC achieves superior performance in omnidirectional image compression while maintaining low computational complexity.