We experimentally investigate the Rydberg state excitations (RSEs) of noble gas atoms, He, Ar and Xe, in an 800-nm 50-fs strong laser field, by using the mass resolved pulsed electric field ionization method combined with the time-of-flight mass spectrometer. We measure the yields of the atomic RSE at different laser intensities and ellipticities, and compare the results with those of the nonsequential double ionization (NSDI) in strong laser fields. Our study shows that like that of NSDI, the yield of the atomic RSE increases as the atomic number increases, i.e., RSE yield trend is He Ar Xe. On the other hand, for any of the atoms, the probability of NSDI is lower than that of total RSE at the same laser intensity, which can be understood as that the yield of high energy electrons (for NSDI) is less than that of low energy electrons that can be captured into the Rydberg states. Additionally, our results show that the RSE yield strongly depends on the laser ellipticity, which is completely suppressed by a circularly polarized laser field. The dependence of RSE on laser ellipticity turns weaker as the atomic number increases, and is weaker than that of NSDI for any of the atoms. It is indicated that the atomic RSE in strong laser field can be attibuted to the capture of the low energy electrons after tunneling ionization into Rydberg states by the Coulomb potential at the end of the laser pulse.