Within the isospin-dependent Brueckner framework, we investigate the contribution of three-body force (TBF) rearrangement to isospin symmetry potential as well as its momentum and density dependence.In particular, we investigate the TBF rearrangement effects on the isospin splitting of neutron and proton effective masses in neutron-rich nuclear matter. We show that the rearrangement contribution of TBF to neutron and proton single-particle potentials is repulsive and increases rapidly with increasing density and momentum. At low densities, the influence of the TBF rearrangement on symmetry potential is rather small, and the TBF rearrangement effect becomes more and more pronounced as the density rises. At high densities, the contribution of TBF rearrangement increases considerably the symmetry potential and modifies remarkably the momentum dependence of the symmetry potential. In both cases with and without including the TBF rearrangement contribution, the predicted neutron effective mass in neutron-rich matter is greater than the proton effective mass. The TBF rearrangement effect is to decrease remarkably both the proton and neutron effective masses, and reduce the magnitude of neutron-proton effective mass splitting in neutron-rich matter at high densities.