Broadband terahertz (THz) emission generated from laser induced gas plasma provides an effective tool for studying nonlinear spectrum, imaging and remote sensing. Recently, the contribution of plasma oscillation to the THz emission was revealed from the nitrogen molecules pumped by intense two-color laser pulses. Plasma oscillation contributes only to the THz emission at relatively low plasma density due to negligible plasma absorption. More generally, with the THz emission generated from the ionizing gaseous medium, the surrounding plasma is expected to play an important role in the generation process. For the THz radiation from laser filament, the plasma region is extended in the laser propagation direction, and the effect of surrounding plasma on the emitted THz spectrum needs studying. In this work, we investigate the relation between pump power and filament length from THz spectrum emitted by air filament driven by two-color laser pulse. The time domain spectrum of THz field is recorded by an electro-optic (EO) sampling technique. In our experiments, significant frequency shifts are observed as the pump power and the filament length increase, and we find that the center frequency of the THz radiation is shifted towards longer wavelength, which is the so called red-shift of the THz spectrum. This red-shift is independent of THz radiation angle. The observations are explained by the plasma absorption inside the air filament. Our theoretical model is based on three mechanisms: the ionization-induced photocurrent, the plasma current oscillation and the plasma absorption. We coherently add up all the local THz fields inside the air filament, and simultaneously consider the plasma absorption induced correction of the THz spectrum. The simulation well reproduces the experimental observation. The skin depth decreases as the plasma density increases, thus the plasma absorption dominates the red-shift process. If the skin depth is larger than the filament length, the plasma oscillation contributes to the THz spectrum dominantly, and thus leading to the blue-shift of THz spectrum. Our results indicate that for the extended filament length or higher plasma density, the combining effect of photocurrent, plasma oscillation and absorption, results in the observed low-frequency broadband THz spectrum. Our study offers a method of coherently controlling the broadband THz spectrum.