Piezoelectric ceramics is a versatile functional material that can realize interconversion between electrical energy and mechanical energy. As the electrical properties of piezoelectric ceramics are extremely sensitive to the grain size variation, the investigation of grain size effect has attracted much attention. In this paper, the recent research progress of the grain size effect on perovskite piezoelectric ceramics, including barium titanate (BT), lead zirconate titanate (PZT), potassium sodium niobate (KNN), and sodium bismuth titanate (BNT), is comprehensively reviewed. We especially focus on topics including feasible ways of fabricating piezoelectric ceramics with the desired grain sizes, the influence of the grain size effect on piezoelectric properties, and the corresponding physical mechanisms. This review would be beneficial to understanding the influence of the grain size effect on piezoelectric properties. The review concludes with the prediction of the further investigation on the grain size effect.