This article deals with the exciton-phonon interaction which has important effects on the properties of the charge transfer excitons (CTE) in alkali halide crystals. An interaction mechanism: the modulation of the Hamiltonian of the CTE system by the vibrations of the constituent ions, which may play an important role, is studied. The CTE-phonon interaction Hamiltonian is derived. From this interaction Hamiltonian, we have discussed the lifetime of the CTE. The linewidths calculated self-consistently under proper approximations agree semiquantitatively with the data measured by experiments in the literature. So we consider that this interaction Hamiltonian may present the important exciton-phonon interaction in alkali halide crystals.