It is difficult to effectively shield the system generated electromagnetic pulse (SGEMP), which can significantly affect the performance of important electronic devices and infrastructure, such as low-orbit spacecraft. Numerical simulation is an essential way to study the SGEMP response. However, many previous studies ignored or simplified the effect of secondary electron emission in their models. In this paper, a three-dimensional electromagnetic particle-in-cell numerical simulation model is developed to evaluate the effect of secondary electrons on the SGEMP response of two typical structures (external SGEMP and cavity SGEMP, respectively) under different current densities (0.1–100 A/cm²) and different materials (Al, Cu and Au). A right cylinder or cylindrical cavity with a length of 100 mm is used. The photoelectrons produced by the interaction between the X-ray photon and metal are emitted from one end of the system and assumed to be monoenergetic. The photoelectron pulse follows a sine-squared distribution, and its full width at half maximum is 1 ns. Some important parameters of secondary electrons are discussed and summarized, including the emission coefficients of elastically and inelastically backscattered electrons, as well as the probability density functions of emission angles and energies. The results show that ignoring the secondary emission in the simulation model leads the peak electric field to be underestimated by twice-thrice, and the duration of electric field response by more than 10%. The oscillation frequency and the amplitude of the second peak of the tangential magnetic field are also increased, with the secondary electrons considered. Among various types of secondary electrons, backscattered electrons have a dominant effect on the change of SGEMP. The effect of true secondary electrons is about 1/5 of that of backscattered electrons. The effect of secondary electrons on SGEMP response increases with a higher atomic number of the material used in the system, mainly due to higher backscattering emission coefficient and a high ratio of high energy inelastically backscattered electrons. The secondary electrons will influence the response of the external SGEMP only when the space charge effect is strong (high X-ray fluence). While the response of the cavity SGEMP is more easily affected by the secondary electrons even at a relatively low X-ray fluence. This paper helps to better obtain the SGEMP response of a specific device under strong radiation through numerical simulation.