Capacity of the Hebbian-Hopfield network associative memory

In {Hop82}, Hopfield introduced a Hebbian learning rule based neural network model and suggested how it can efficiently operate as an associative memory. Studying random binary patterns, he also uncovered that, if a small fraction of errors is tolerated in the stored patterns retrieval, the capacity of the network (maximal number of memorized patterns, m) scales linearly with each pattern's size, n. Moreover, he famously predicted αc=limn→∞m/n≈ 0.14. We study this very same scenario with two famous pattern's basins of attraction: {(i)} The AGS one from {AmiGutSom85}; and {(ii)} The NLT one from {Newman88,Louk94,Louk94a,Louk97,Tal98}. Relying on the fully lifted random duality theory (fl RDT) from {Stojnicflrdt23}, we obtain the following explicit capacity characterizations on the first level of lifting: {equation} α_c(AGS,1) = ( maxδ∈ ( 0,1/2 ){1-2δ}{√2 erfinv ( 1-2δ )} - {2}{√2π} e- ( erfinv ( 1-2δ ) )^2 )^2 ≈ 0.137906 {equation} {equation} α_c(NLT,1) = erf(x)^2/2x^2-1+erf(x)^2 ≈ 0.129490, 1-erf(x)^2- {2erf(x)e-x^2}{√}+{2e-2x^2}{π}=0. {equation} A substantial numerical work gives on the second level of lifting αc(AGS,2) ≈ 0.138186 and αc(NLT,2) ≈ 0.12979, effectively uncovering a remarkably fast lifting convergence. Moreover, the obtained AGS characterizations exactly match the replica symmetry based ones of {AmiGutSom85} and the corresponding symmetry breaking ones of {SteKuh94}.