In this paper, we propose a new method to realize both polarization-multiplexing and wavelength-multiplexing using a simple structure, which can realize hologram by the multiplexing of double wavelengths and double polarization in the visible band. Our design can reduce color cross-talk and have a higher diffraction efficiency. We design a transmission metasurface composed of simple rectangular cells. Firstly, we establish the relationship of structural parameters with the transmission phase under various incident conditions of light beams. Then we propose a fitness function that can optimize the structural parameters of the unit cell at each pixel point, which can display different images by 532 nm x-polarization and 633 nm y-polarization incident light beams respectively. Finally, finite difference time domain method is used to analyze the structure, and the holographic result fits the theoretical design very well. This work proposes using single metasurface structure to solve the problems of wavelength cross-talk appearing when using simple structures, and will have great importance in coding and anti-counterfeiting.