There are many advantages for frontally located beam smoothing elements in terminal optical system of the inertial confinement fusion （ICF） driver, but it will also exert an influence on the working state of the frequency conversion system at the same time. This paper studies the influence from the viewpoints of system integration and optimization. Based on the non-linear transmission theory of high power laser and adopting coupled-wave differential equations and perturbation theory, we calculate the third harmonic conversion efficiency, optical characteristics in near and far field as well as the shape of focal spot when CPP is laid before the frequency conversion system. Then we put forward the method for the optimization of detuning angle and thickness of the frequency conversion crystal, in order to reduce influence on ICF system when CPP is laid before the frequency conversion system. The third harmonic conversion efficiency, contrast ratio and encircled energy before and after optimization are compared and analyzed. The results show that the optimization of frequency conversion system is very effective in reducing the impact on the beam quality and the third harmonic conversion efficiency when CPP is laid before the frequency conversion system.