Assuming the object surface is a scattering plane and both the reference beam and the reconstructing beam are spherical waves, we study the light wave field of object beam field reconstructing plane of off-axis digital holography in detail. The object beam, the conjugate object beam and the distribution of zero-order diffraction beam as functions of the optical system parameters are obtained. According to the characteristics of digital color holography, an optical system design method for suppressing zero-order diffraction interference is proposed. Furthermore, an optimization study has also been conducted on the wave front reconstruction algorithm to avoid interpolation error. Finally, the corresponding experimental verifications are given.