Random features Hopfield networks generalize retrieval to previously unseen examples

It has been recently shown that a feature-learning transition happens when a Hopfield Network stores examples generated as superpositions of random features, where new attractors corresponding to such features appear in the model. In this work we reveal that the network also develops attractors corresponding to previously unseen examples generated as mixtures from the same set of features. We explain this surprising behavior in terms of spurious states of the learned features: increasing the number of stored examples beyond the feature-learning transition, the model also learns to mix the features to represent both stored and previously unseen examples. We support this claim by computing the phase diagram of the model and matching the numerical results with the spinodal lines of mixed spurious states. • The simplest associative memory is capable of “storing” previously unseen examples. • This phenomenon happens by exploiting mixed spurious states creatively. • This behavior can be predicted using the replica method from statistical physics.