The recently introduced continuous Hopfield network 9 exhibits large memorization capabilities, which manifest as attractive fixed points of its update rule—a differentiable function consisting of two linear mappings composed with the scaled softmax function. The authors of 9 provide proofs for the existence and approximate position of such attractive fixed points. For the softmax function alone, the fixed point structure has been fully characterized in earlier work 10 , from which it turns out that for sufficiently large scaling factors there are exponentially more unstable fixed points than attractive ones. In this work, we complement the findings in 9 by showing that, under natural geometric conditions on the vectors defining the continuous Hopfield network, unstable fixed points must occur, analogous to the findings in 10 . Our results show that, under these geometric conditions, continuous Hopfield networks necessarily admit additional unstable fixed points associated with higher-dimensional faces of the pattern polytope.
Hans-Peter Beise (Wed,) studied this question.