Raman spectra of the metamict and annealing recrystallization fergusonite and brocenite, found in Baiyunobo mineral deposit in China, have been measured and discussed. The spectra give three well defined characteristic band regions. The bands in the regions 840 to 600cm-1 and 340 to 280cm-1 are assigned to NbO3-4 tetrahedron stretching and bending vibrations, respectively. While the frequencies observed below 200cm-1 are due to the external modes of the NbO3-4 and lanthanide ions. The dependence of Raman frequencies on the structure and composition of the metamict and annealing recrystallization fergusonite group minerals shows that the Raman spectra are valuable for reflecting the structure distortion of the minerals. Additionally, the strong visible photoluminescence from annealing recrystallization is observed when 514.5 or 488.0nm laser line is used as excitation source, whereas metamic fergusonite group minerals exhibit no visible photoluminescence, or weak one.