With the rapid development of material science and industrial technology, the application of ultrasonic Lamb wave to the industrial nondestructive testing has received considerable attention due to its advantages of rapidness, high efficiency, high accuracy, and low cost. However, the multimode and dispersion problem of Lamb waves are still challenging. Multimode mixed Lamb wave signals are often present at the same excitation frequency in the actual detection. To separate dispersive multimode Lamb waves overlapped in time and frequency domains, a separation method based on dispersion compensation and fractional differential is presented. The multimode Lamb waves overlapped in time and frequency domains are first compensated by using the dispersion characteristic. Based on the dispersion compensation, the time-delay function is modeled. The function is used as a transfer function. Its inverse is considered as a dispersion compensation function. Then, the amplitude spectra of Lamb waves are divided into fractional order differentials. The parameters of each mode are extracted by using the fitting polynomial between the maximum amplitude and the differential order and that between the peak frequency and the differential order. Its amplitude spectrum is extracted based on its parameters. By combining with its phase spectrum, the individual mode is constructed after the dispersion has been recovered. Simulation and experiments are performed on a 1 mm-thick stainless steel plate. The oblique transducers with the angle of 26 and the central frequency of 3 MHz are used to excite the S1 and A1 mode overlapped Lamb wave signal in the plate. The transducers are coupled with the stainless steel plate by using the ultrasonic couplant. Simulation and experimental analysis show that the present method can not only achieve the separation of time-frequency overlapped multimode Lamb waves, but also guarantee the separation precision. The main advantage of the presented method is the combination of the dispersion compensation and the fractional differential, which solves the problem of mixing with other mode signals after the single mode dispersion has been compensated, and improves the extraction precision of each mode. Therefore, this method can be used for separating the time-frequency overlapped multimode Lamb waves. It is conducible to the signal processing of multi-mode Lamb wave dispersion.