We examine the sources of error in the histogram reweighting method for Monte Carlo data analysis. We demonstrate that, in addition to the standard statistical error which has been studied elsewhere, there are two other sources of error, one arising through correlations in the reweighted samples, and one arising from the finite range of energies sampled by a simulation of finite length. We demonstrate that while the former correction is usually negligible by comparison with statistical fluctuations, the latter may not be, and give criteria for judging the range of validity of histogram extrapolations based on the size of this latter correction.