We establish conditions for the differentiability, to any order, of the Gibbs states of classical lattice systems with arbitrary compact single-spin space and with interactions in the Dobrushin uniqueness region. The derivatives are expressed as series expansions and are shown to be continuous on the uniqueness region. We also provide a procedure for estimating the size of the derivatives. These results verify a conjecture of L. Gross and extend his results in "Absence of second-order phase transitions in the Dobrushin uniqueness region," Journal of Statistical Physics 25(1):57-72 (1981). The techniques of this paper are based on those employed by Gross.