Performance validation and analysis for multi-method fusion based image quality metrics in a new image database

Considering that there is no single full reference image quality assessment method that could give the best performance in all situations, some multi-method fusion metrics were proposed. Machine learning techniques are often involved in such multi-method fusion metrics so that its output would be more consistent with human visual perceptions. On the other hand, the robustness and generalization ability of these multi-method fusion metrics are questioned because of the scarce of images with mean opinion scores. In order to comprehensively validate whether or not the generalization ability of such multi-method fusion IQA metrics are satisfying, we construct a new image database which contains up to 60 reference images. The newly built image database is then used to test the generalization ability of different multi-method fusion IQA metrics. Cross database validation experiment indicates that in our new image database, the performances of all the multi-method fusion IQA metrics have no statistical significant different with some single-method IQA metrics such as FSIM and MAD. In the end, a thorough analysis is given to explain why the performance of multi-method fusion IQA framework drop significantly in cross database validation.