We consider a dynamical system described by a set of random variablesN| i(t) and depending on a parameterR controlling its stability. IfR <> c the system is stable and theN i have some symmetry properties in the statistical sense (i.e., with respect to time averaging). IfR > R c the system is unstable and the nonlinear dynamics of theN i may lead to an asymptotic stationary state which does not possess the symmetries of the stable system. We show that the dynamics of symmetry breaking resembles a phase transition in the limit of many variables.