Reasoning about stochastic dynamical systems and planning under uncertainty has come t o play a fundamental role in A1 research and applications. The representation of such systems, in particular, of actions with stochastic effects, has accordingly been given increasing attention in recent years. In this article, we survey a number of techniques for representing stochastic processes and actions with stochastic effects using dyriilrriic Bayesian networks and influence diagrams, and briefly describe how t,hese support effective inference for tasks such as monitoring, forecasting, explanation and decision making. We also compare these techniques to several act,ion representations adopted in the classical reasoning about action and planning communities, describing how tradit,ional problems such as the frame and ramification problems are dealt with i r i stochastic settings, and how these solutions compare t o recent approaches to this problem in the classical (deterministic) literature. We argue that while stochastic dynamics introduce certain complications when it comes to such issues, for the most part, intuitions underlying classical models can be extended to the stochastic setting.