We investigate statistics of occupation times for an over-damped Brownian particle in an external force field, using a backward Fokker–Planck equation introduced by Majumdar and Comtet. For an arbitrary potential field the distribution of occupation times is expressed in terms of solutions of the corresponding first passage time problem. This general relationship between occupation times and first passage times, is valid for normal Markovian diffusion and for non-Markovian sub-diffusion, the latter modeled using the fractional Fokker–Planck equation. For binding potential fields we find in the long time limit ergodic behavior for normal diffusion, while for the fractional framework weak ergodicity breaking is found, in agreement with previous results of Bel and Barkai on the continuous time random walk on a lattice. For non-binding cases, rich physical behaviors are obtained, and classification of occupation time statistics is made possible according to whether or not the underlying random walk is recurrent and the averaged first return time to the origin is finite. Our work establishes a link between fractional calculus and ergodicity breaking.