The physics of modulation and rf extraction of an S-band relativistic klystron oscillator is studied in this paper via experiment, theory, and simulation. It is found that the intense relativistic electron beams (IREBs) can be intensely current-modulated when the IREBs drift through three pillboxes with high coupling. After bunching in the downstream the modulated IREBs can excite high-power microwave in the triaxial cavity. These properties, which have short oscillating time, compact geometry and high beam-wave conversion efficiency, were encouraging. Using a 1 MV, 13 kA, 40 ns electron beam and a 0.9 kGs leading magnetic field, 3.5 GW radiated power was extracted in 20 ns FWHM pulses at 2.86 GHz. The efficiency was 27%, and the instantaneous bandwidth was 2%. The radiated power was 3.4 GW when the repetition rate IREBs was 20 Hz. The experimental results agree well with the simulations.