Unsupervised Learning by Examples: On-line versus Off-line

We study both on-line and off-line unsupervised learning from p random patterns which are uniformly distributed on the N-sphere with the exception of a single symmetry breaking orientation B, along which they may be arbitrarily distributed. Supervised learning from the same kind of patterns is included as a special case. In the thermodynamic limit N→∞ with α0ex0ex=0ex0exp/N fixed we calculate the overlap R(α)0ex0ex=0ex0exB ̇ \.J/|J||B| between the unknown ``true'' B and the optimal ``Bayes'' hypothesis J with particular emphasis on the small and large {α} asymptotics and the phenomenon of retarded learning. Finally, we identify a cost function whose minimum reproduces the off-line Bayes overlap.