Based on the Rietveld theory and Maud software, the evolution of crystalline structure of PAN-based carbon fibers was studied by investigating the different stages in the preparation of carbon fibers. The microstructure parameters of PAN fibers, oxidative fibers, and carbon fibers, including cell parameters, apparent crystal grain sizes, textural structure and micro-strain, were obtained from the named whole powder pattern fitting method for XRD result. Based on rigorus theory of crystallography, the whole powder pattern fitting treats the whole XRD powder diffraction pattern rather than individual diffraction peaks, resulting in higher credibility and precision. The results indicated that polymer chains were arranged along the axis of PAN fiber and the apparent crystallite size was about 6.4 nm. After pre-oxidation, the originally ordered structure in PAN fiber was damaged and new less-ordered cyclic ladder structure was formed in the pre-oxidized PAN fibers, while the apparent crystallite size of fiber decreased obviously. With the carbonization process, the cyclic ladder structure of per-oxidized fibers transferred into layer structure similar to that of graphite. The layers were oriented along the axis of carbonized fiber that contributed to the X-ray diffraction, which comprised the apparent crystallite grains. As the carbonization temperature increased, size of the layer structure in the direction perpendicular to fiber axis increased clearly, the preferential orientation was enhanced and the layer structure became much more ordered.