A Full-Reference Stereoscopic Image Quality Measurement Via Hierarchical Deep Feature Degradation Fusion

For the problem of stereoscopic image quality measurement (SIQM), it is difficult to design an efficient yet reliable full reference (FR) SIQM method due to our limited knowledge about the properties of human binocular vision. Inspired by the fact that the input visual information is hierarchically processed in our human brain, we consider different levels of distortion in an image cause individual degradations on hierarchical features, and propose to fuse the degradations on hierarchical features to facilitate the task of FR-SIQM. As one of the most classical convolutional neural network (CNN) architectures, the VGG-16 network is first applied to each view of the stereopair to build hierarchical deep feature representations based on which monocular quality estimation (MQE) and binocular quality fusion (BQF) are then performed. Specifically, the MQE stage estimates a set of layer-wise monocular quality scores by measuring the similarity between the hierarchical feature maps of the distorted monocular view and those of the reference monocular view. The BQF stage estimates a set of layer-wise binocular quality scores via a weighted average of the corresponding layer-wise monocular quality scores. The adaptive weights are determined by a modified hierarchical feature energy-based Gain-Control model. Finally, the layer-wise binocular quality scores across all layers are fused into an overall binocular quality score via regression. Experiments on three benchmark databases validate the state-of-the-art performance of our method.