Atomic magnetometers can achieve high magnetic-field sensitivity in the spin-exchange relaxation-free (SERF) regime, while multi-species media can provide good long-term zero-bias stability. Their underlying spin system is intrinsically nonlinear. In this work, we study the nonlinear dynamical behavior of a feedback-assisted atomic magnetometer under periodic modulation of the longitudinal bias magnetic field. In the parameter range considered here, the
87Rb electron-spin polarization remains close to a quasi-steady state, so the long-time dynamics are dominated by the
129Xe nuclear-spin polarization and are described by a simplified Bloch model. By numerically integrating the Bloch equations, we determine the stable dynamical states reached after long-time evolution. To characterize the nature of these states, we combine fast Fourier transform spectra, Poincaré sections and Lyapunov exponents. Our results show that, under periodic driving, the system exhibits three distinct states: quasi-periodic orbits, chaos, and limit cycles. The quasi-periodic state is characterized by multiple incommensurate frequencies in the spectrum. The chaotic state shows a positive Lyapunov exponent and a continuous broadband spectrum. The limit cycle state corresponds to a stable closed periodic orbit in phase space. By scanning the modulation amplitude
δ and modulation frequency \omega_\mathrmacT_2, we construct the dynamical phase diagram in the modulation-parameter space. In particular, for (\omega_0T_2, \chi/\chi_\mathrmc) = (1.3, 8), the phase diagram, is mainly composed of quasi-periodic, chaotic, and limit-cycle regions, with a small isolated limit-cycle island near the boundary between the quasi-periodic and chaotic regions. Increasing
δ can drive the system from quasi-periodic to chaos and then to a limit cycle. For comparison, when (\omega_0T_2, \chi/\chi_\mathrmc) = (1.2, 8), the quasi-periodic region shrinks markedly toward smaller
δ. Based on the phase diagram without modulation, we provide a qualitative argument for the origins of the quasi-periodic and chaotic dynamics, and note that the limit cycles with intricate trajectories emerge from the nonlinearity of the system. These results enrich the understanding of nonlinear spin dynamics in periodically modulated atomic magnetometers.