Journal of Statistical Physics, Vol. 50, Nos. 1/2, 1988. p. 231-257.
A neural network is called nonlinear if the introduction of new data into the synaptic efficacies has to be performed through a nonlinear operation. The original Hopfield model is linear, whereas, for instance, clipped synapses constitute a nonlinear model. Here a general theory is presented to obtain the statistical mechanics of a neural network with finitely many patterns and arbitrary (symmetric) nonlinearity. The problem is reduced to minimizing a free energy functional over all solutions of a fixed-point equation with synaptic kernel Q. The case of clipped synapses with bimodal and Gaussian probability distribution is analyzed in detail. To this end, a simple theory is developed that gives the spectrum of Q and thereby all the solutions that bifurcate from
the high-temperature phase.