In recent years, with the greatly improving performance of computers, deep learning technology has shown an explosive development trend and has been widely used in various fields. In this background, the CGH generation algorithm based on deep learning provides a new idea for real-time high quality of holographic display. The convolutional neural network is the most typical network structure in deep learning algorithms, which is able to automatically extract key local features from an image and construct more complex global features through operations such as convolution, pooling and full connectivity. Convolutional neural networks have been widely used in the field of holographic displays due to their powerful feature extraction and generalization abilities. Compared with the traditional iterative algorithm, the CGH algorithm based on deep learning has a significant improvement in computing speed, but its image quality still needs to improve further. In this paper, an attention convolutional neural network based on the diffraction model of the angular spectral method is proposed to improve the generation quality of holograms while generating holograms quickly. The whole network consists of real-valued and complex-valued convolutional neural networks, the real-valued network is used for phase prediction and the complex-valued network is used to make prediction of the complex amplitude of the SLM surface, and the phase of the complex amplitude which is obtained after prediction is used for holographic coding and numerical reconstruction. An attention mechanism is embedded in the downsampling stage of the phase prediction network to improve the feature extraction capability of the whole algorithm, thus improving the quality of the generated phase-only holograms. An accurate diffraction model of the angular spectrum method is embedded in the whole network to avoid labeling of large-scale datasets, and unsupervised learning is used to train the network. The proposed algorithm is able to generate high-quality 2K holograms within 0.015s. The average peak signal-to-noise ratio of the reconstruction images is up to 32.12 dB and the average structural similarity index measure of the generated holograms is up to 0.934. Numerical simulations and optical experiments verify the feasibility and effectiveness of the proposed attentional convolutional neural network algorithm based on the diffraction model of angular spectral method, which provides a powerful help for the application of deep learning theory and algorithm in the field of real-time holographic display.