Information processing in nonlinear neural networks with a finite number q of stored patterns is studied. Each network is characterized completely by its synaptic kernel Q. At low temperatures, the nonlinearity typically results in 2^(q-2)-q metastable, pure states in addition to the q retrieval states that are associated with the q stored patterns. These spurious states start appearing at a temperature T`_q, which depends on q. We give sufficient conditions to guarantee that the retrieval states bifurcate first at a critical temperature T_c and that T`_q/T_c -> 0 as q -> oo. Hence, there is a large temperature range where only the retrieval states and certain symmetric mixtures thereof exist. The latter are unstable, as they appear at T_c. For clipped synapses, the bifurcation and stability structure is analyzed in detail and shown to approach that of the (linear) Hopfield model as q -> oo. We also investigate memories that forget and indicate how forgetfulness can be explained in terms of the eigenvalue spectrum of the synaptic kernel Q.