Because non-working modes are very easy to excit in the high gain relativistic klystron amplifier, and seriously affect the beam-wave interactive action, the suppressing of non-working modes is very important. These modes can seriously degrade klystron performance and cause the pulse shortening, the normal operation of HGRKA will be affected greatly. This paper deals with the mechanism of non-working mode self-excitation by the PIC simulation, and it is obvious that the coupling between cavities is the main cause of non-working mode self-excitation. The coupling can form the positive feedback loop. The formula for starting current of non-working mode self-excitation is developed according to the coupling between cavities, and the corresponding measures are taken to avoid non-working mode self-excitation. Then the corresponding simulations and optimization are conducted. Finally non-working mode control is realized in the HGRKA experimentally when driven power is only few kilowatts. The RF output has a power of 0.98 GW and pulse width of 100 ns.