Two kinds of gold nanoparticles with different surface-capping groups (Au-DT and Au-DT-CTAB), 1-dodecanthiol and cetyltrimethylammonium bromide (CTAB) are prepared by a two-phase method. Structures and linear optical properties of the materials are characterized using transmission electron microscopy and ultraviolet--visible absorption spectroscopy. The effects of surface modification on nonlinear optical (NLO) performance and optical limiting behavior of gold nanoparticles are investigated using the openZ-scan technique at a laser wavelength of 532 nm. The results show that Au-DT-CTAB shows an enhanced NLO effect in comparison with Au-DT. The secondary modification with CTAB can not only improve local field enhancement effect but also make the contribution of thermoelectrons to NLO performance of gold nanoparticles on laser irradiation.