Abstract According to analysis of discrete cosine transform （DCT） coefficients of video sequence, a new DCT coefficients statistical model based on Weibull probability density function（PDF） is introduced. Compared with Laplacian PDF and Cauchy PDF, the model can describe properties of DCT of video sequences much better. In addition, according to the new statistical model and theory of entropy coding, the rate quantization （RQ） and distortion quantization （DQ） functions are proposed separately. Based on the simplified and approximate RQ and DQ which are obtained by the theory of rate distortion （RD） model and the properties of real video sequence, the novel RD model is presented. The experimental results show that the real video sequence encoded by intra or inter can be described much better by using rate distortion model based on Weibull PDF.