Y2O3-TiO2 composite film were deposited on Si substrate at room temperature by means of radio frequency magnetron sputtering. The crystalline state and topography of the film before and after annealing were measured by XRD and AFM, and the mechanism of the film being more compact was discussed. The result revealed that the evenness and compactness of the film can be improved with increasing the sputtering power, which is due to the fact of more Y2O3 filling the pores around TiO2with raising sputtering power, which inhibited the growth of big TiO\-2 grains and improved the evenness and compactness of film. After annealing, XRD pattern indicated that the addition of Y2O3 favors the formation of rutile TiO2 film which has high-dielectric constant.