In this paper defect structure was investigated by dielectric response of pure ZnO ceramics，binary and multi-element ZnO-Bi2O3 based varistor ceramics with the help of broad band dielectric spectroscopy. It is found that no loss peak appears at room temperature in pure ZnO ceramics，while there is one and two loss peaks in the binary and multi-element ZnO-Bi2O3 based varistor ceramics，respectively. According to relaxation polarization theory and electronic relaxation theory，it is obtained that electronic relaxation process is equivalent to relaxation polarization in dielectric spectroscopy and the characteristic peak near 105Hz at room temperature arises from relaxation process of electrons trapped by oxygen vacancies and zinc interstitials. Based on the difference in the dependence of the two loss peaks on the temperature and the atmosphere of heat treatment，aging mechanism is explained as the desorption of oxygen.