Answers are grounded in this paper and cite their sources.
We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group G that acts discretely on the input and output of a standard neural network layer ϕW: M → N, we show that ϕW is equivariant with respect to G-action iff G explains the symmetries of the network parameters W. Inspired by this observation, we then propose two parameter-sharing schemes to induce the desirable symmetry on W. Our procedures for tying the parameters achieve G-equivariance and, under some conditions on the action of G, they guarantee sensitivity to all other permutation groups outside G.
No takes yet. Share an insight, caveat, or question.
Ravanbakhsh et al. (2017) studied this question.