The materials with low thermal conductivity (κ) are both fundamentally interesting and technologically important in applications relevant to thermal energy conversion and thermal management, such as thermoelectric conversion devices, thermal barrier coatings, and thermal storage. Therefore, understanding the physical mechanisms of glass-like heat conduction in crystalline materials is essential for the development and design of low-κ materials. In this work, the microscopic phonon mechanism of glass-like low κ in binary simple crystal Yb₃TaO₇ with fluorite structure is investigated by using the equilibrium molecular dynamics, phonon spectral energy density, and lattice dynamics. Meanwhile, the weberite-structured Yb₃TaO₇ is also mentioned for comparison. The calculated κ indicates that fluorite Yb₃TaO₇ has a glass-like low κ while weberite Yb₃TaO₇ has a crystal κ. Such a low κ in fluorite Yb₃TaO₇ is mainly due to the large difference in interatomic force between O-Yb and O-Ta. This different atomic bonding can significantly soften the phonon mode and thus limit phonon transport. To further describe the microscopic phonon thermal conduction, the single-channel model based on the phonon gas model is first used to calculate the total κ. However, the single-channel model significantly underestimates the κ, suggesting the presence of non-normal phonons in Yb₃TaO₇. Based on this, vibrational mode decomposition is conducted throughout the entire phonon spectrum of fluorite- and weberite-type Yb₃TaO₇. It is found that most modes in fluorite Yb₃TaO₇ fall in the Ioffe–Regel regime and exhibit a strongly diffusive nature. Such diffusive modes cannot be described by the phonon gas model. Based on the decomposed phonon modes, the dual-channel model involving diffusive mode and propagating mode is used to describe the phonon thermal conduction, by which the obtained results accord well with the experimental values. The vast majority (> 90%) of heat in fluorite Yb₃TaO₇ is found to be transported by diffusive modes rather than propagating modes. Consequently, the κ of fluorite Yb₃TaO₇ increases with temperature rising, exhibiting a unique glass-like nature. In particular, contrary to conventional wisdom, the optical phonon mode in fluorite Yb₃TaO₇ plays a significant or even decisive role in thermal conduction, which could serve as a new physical factor to adjust κ in solid materials. Overall, the new understanding of the link between chemical bonding and glass-like κ can contribute to the development and design of low-κ materials.