A three-dimensional model for the interaction between a sheet electron beam and the slow-wave structure of dielectric-loaded rectangular Cerenkov maser is proposed. Based on this model, the hybrid-mode dispersion equation and its analytical solution are derived by using the field-match method and the Borgnis function method. Through numerical calculations, the effects of dielectric layer thickness, beam voltage, current density, bean thickness and beam-dielectric layer gap on the linear growth rate are analyzed.