From Canonical Correlation Analysis to Self-supervised Graph Neural Networks

We introduce a conceptually simple yet effective model for self-supervised learning with graph data. It follows the previous methods that two views of an input graph through data augmentation. However, unlike methods that focus on instance-level discrimination, we optimize an feature-level objective inspired by classical Canonical Correlation. Compared with other works, our approach requires none of the mutual information estimator, additional projector, asymmetric, and most importantly, negative samples which can be costly. We show the new objective essentially 1) aims at discarding augmentation-variant by learning invariant representations, and 2) can prevent solutions by decorrelating features in different dimensions. Our analysis further provides an understanding for the new objective can be equivalently seen as an instantiation of the Information Principle under the self-supervised setting. Despite its simplicity, method performs competitively on seven public graph datasets. The code is at: https: //github. com/hengruizhang98/CCA-SSG.