From molecular current viewpoint, demagnetizing field and process of permanent magnet, taking rectangular permanent magnets for example, the various factors that affect inhomogeneity of external magnetic field for permanent magnet are analyzed. The results indicate that the macroscopic inhomogeneity of external magnetic field for permanent magnet is closely related to space distance and shape of permanent magnet design. Demagnetizing field of permanent magnet has a complex influence on micro-inhomogeneity of external magnetic field. Technology of permanent magnet such as powder particles, orientation degree, sintering solidification, mechanical processing will affect inhomogeneity of external magnetic field of permanent magnet such as magnetic degree, symmetry, smoothness, etc.