A novel fiber sensor based compound ring laser cavity with linear variation of frequency is proposed and demonstrated experimentally. The compound ring laser cavity is comprised of a ring cavity and a straight cavity. This system can generate the beat frequency spectrum by employing an erbium doped fiber amplifier, a fiber Bragg grating is used as a sensor head and the straight cavity reflector, a π phase shifted fiber Bragg grating serves as a microwave photonic passband filter. The principle of the proposed sensor is theoretically analyzed, showing that as the displacement increases the beat frequency decreases, and there exists a linear relationship between displacement change and beat frequency shift. In experiment, it is shown that the sensor has a high sensitivity of about 86.19 kHz/mm and can achieve a good linear response (R² = 0.9973), and that the minimum monitored displacement is about 10 μm. The measurement results demonstrate that the sensor is accurate, sensitive, and the proposed sensor system has a compact and simple structure, which makes it convenient for more applications in future.