Critical properties of systems exhibiting phase transitions into phases with infinite numbers of absorbing states are studied. We analyze a non-Markovian Langevin equation recently proposed to describe the critical behavior of such systems, and also introduce and study a non-Markovian discrete model, which is argued to present the same critical features.