Fretting phenomena exist widely in structural engineering. In recent years, it has attracted more attention from scientists and technicians. In order to study the fretting wear in depth, we establish a new method of calculating the wear rate of material in vibratory environment. Firstly, according to the characteristics of friction pair and fretting wear process in fretting friction system, the asymmetric double potential well model is proposed and the potential energy function of the model is given. The transfer of particles between the two kinds of materials during the fretting is regarded as the motion of the particles in the two potential wells which are asymmetrical, and the particle motion equation in the potential well is established. Furthermore, considering the characteristics of the randomness, time-varying and irreversibility of particle motion in fretting friction system, a theoretical model is established by using the non-equilibrium statistical theory, which is based on the particle equation motion, combined with the Langevin equation in random theory and the Foker-Planck equation in the non-equilibrium statistical theory. The probability density distribution function of particles moving from the interior of the material to the material surface at any time is obtained. A method of calculating the wear rate is proposed by integrating the probability density distribution function. Secondly, by calculating the wear rate of the friction pair which consists of metal materials Mg and Fe, we obtain the potential energy function of the asymmetric double potential well model as the different surface energies of both materials. Furthermore, the probability density distribution function of particles moving in this friction pair is calculated. Then, the change of wear rate with wear time and width of potential well is derived, and the effect of normal force on wear rate is further analyzed. The results of calculation and analysis show that the wear rate of material decreases with the decrease of the width of the potential well in the friction pair system, decreases with the increase of wear time and increases with the increase of the normal force of the load, and the surface of the relatively soft material in the friction pair system is more likely to wear off. Finally, the conclusions of the theoretical model accord with the experimental results, illustrating the applicability of the theoretical model.