Aiming at the problems of low-resolution and poor visual quality of infrared images, a locality-constrained group sparsity based infrared image super-resolution algorithm is proposed. Firstly with considering the texture self-similarity of infrared images and group structural sparsity of atom coefficients, a locality-constrained group sparse (LCGS) model is proposed. Secondly, under LCGS and K-singular value decomposition, a pair of group structural dictionaries is learned. The dictionary pair can well capture and preserve the intrinsic geometrical manifold of low and high resolution data. Finally, the high-resolution infrared images are recovered by the high-resolution dictionary and the corresponding low-resolution group sparse coefficients. Experimental results show that the proposed method obtains excellent performance in objective evaluation and subjective visual effect.