Capacity of neural networks with discrete synaptic couplings

The authors study the optimal storage capacity of neural networks with discrete local constraints on the synaptic couplings J ij . Models with such constraints include those with binary couplings J ij =+or-1 or J ij =0, 1, quantised couplings with larger synaptic range, e.g. J ij =+or-1/L, +or-2/L, . . ., +or-1 and, in the limit, continuous couplings confined to the hypercube mod J ij mod <or=1 ('box confinement'). They find that the optimal storage capacity alpha ( kappa ) is best determined by the vanishing of a suitably defined 'entropy' as calculated in the replica symmetric approximation. They also extend their results to cases with biased memories and make contact with sparse coding models.