Meta-Learned Zero-Shot Sketch-Based Point Cloud Retrieval via Perspective-Predicted Feature Learning

In recent times, sketch-based 3D shape retrieval has emerged as a pivotal theme and garnered considerable attention within the area of cross-modal retrieval. As a prevalent 3D shape modality, the exponential growth in the quantity of 3D point clouds has boosted a substantial increase in the demand for 3D point cloud retrieval. Simultaneously, due to the absence of prior knowledge about unseen classes, transferring models learned from seen classes to tackle the data from unseen classes effectively remains a significant hurdle in cross-modal retrieval. In light of this, a novel meta-learned zero-shot sketch-based point cloud retrieval (MetaZS-SBPR) network is proposed in this paper for exploring cross-modal consistent feature representation from 2D sketches and 3D point clouds, while effectively transferring the knowledge from seen classes to unseen classes. Specifically, a perspective-predicted point cloud feature learning module is presented to capture discriminative features of point clouds from predicted perspectives, thereby mitigating the modal differences across point clouds and sketches. Additionally, a meta zero-shot retrieval strategy is introduced to investigate the knowledge transfer from seen classes to unseen classes harnessing meta-learning, thereby enabling the efficient retrieval of the point clouds from unseen classes. Experimental evaluations conducted on the ZS-SBPR benchmark dataset affirm the effectiveness of the proposed MetaZS-SBPR.