A relativistic magnetron using all cavity extraction and semi-transparent cathode has the virtues of compactness, high output power and high efficiency. The three-dimensional particle-in-cell simulations show that 1.15 GW output microwave with an efficiency about 50% can be obtained at S-band with pure TE11 mode of the fan waveguide. However, due to the fact that the output structure is composed of three detached fan waveguides, mode conversion structure in the output region is required for the convenience of practical applications. Therefore, two mode conversion structures are studied for the output mode conversion. The first structure is to widen gradually or abruptly the fan waveguide in the azimuthal direction from a given position (starting point) along the microwave transport direction. And then the three fan waveguides are connected into one coaxial waveguide. The effects of the position of the starting point on the beam-wave interaction and microwave extraction are numerically studied. For the convenience of description, we define L as the axial distance between the center of the output coupling hole and starting point. Simulation results show that for the abrupt and gradual variation case, when the length of L changes in a relatively wide region, the output power is larger than 1.0 GW in TEM mode at S-band. It is about 90% of the conventional fan waveguide with 1.15 GW. For the gradual variation case, the optimal value of L equals 10.0 cm, and the corresponding output power is beyond 1.0 GW. For the abrupt variation case, the optimal value of L equals 13.75 cm, the corresponding output power is about 1.15 GW. But in the abrupt variation case, the output power is a little more sensitive to the value of L. The second structure is to convert the fan waveguide into a rectangular waveguide. Ａcompound waveguide composed of a section of fan waveguide and a section of rectangular waveguide is designed for studying its feasibility. In the compound waveguide, the wide edges of the cross section of the rectangular waveguide are tangent to the inner and outer arc of the fan cross section respectively. And the narrow edges cross the end points of the outer arc. Simulation results show that in the compound waveguide the microwave with TE11 mode of the fan waveguide input at the inlet can be changed into the TE10 mode of the rectangular waveguide at the outlet with almost no power loss. In all, the output microwave power larger than 1.0 GW could be obtained after using the two proposed mode conversion structures. In practical applications, one could choose the relevant mode conversion structure according to the requirement.