Commercial and military applications of microwave and millimeter-wave sources in aerospace, radar, navigation, and communication system require high spectrum purity and low phase-noise oscillators. The optoelectronic oscillator (OEO) emerges as an excellent low noise source that has attracted great attention in recent years. In this paper, a novel technique is proposed for single-mode selection in an optoelectronic oscillator, which uses a microwave cavity as the mode selector. It consists of a pump laser and a feedback circuit including an intensity modulator, an optical fiber delay lines, a photodetector, an amplifier, a filter and two drilling cables. The drilling cable is fabricated by drilling open holes on a coaxial cable using a drilling machine. By changing the radius of the drilling holes, the designed reflection coefficient can be obtained. By simulation, the constructed microwave resonator that consists of a filter and two drilling cables has a higher Q value and only the modes that satisfy the oscillation conditions of the loop will be selected. The basic principle is analyzed theoretically and experimentally. By comparing with traditional structure of OEO, it is shown that the novel structure can effectively improve the side-mode suppression ratio. In addition, the stability of the oscillation frequency is easier to control than the parallel structure. In this experiment, the output of a 10 GHz single-mode signal with a side-mode suppression ratio of 72 dB and a phase noise of -122 dBc/Hz@10 kHz from the carrier is obtained. Meanwhile, phase-lock techniques are used to compensate the drift of cavity length. Then the radio frequency (RF) stability of the oscillation frequency is measured using an RF spectrum analyzer, and the RF stability over 3 hours for the OEO is less than 4 Hz. This scheme has the advantages of traditional OEO with low noise since no extra active devices are needed, and it suppresses the side-mode noise also effectively. In addition, this system is promising for the development of compact, high frequency, low cost and low noise OEOs.