In order to obtain highly sensitive broadband organic photodetectors (OPDs) used for image sensors with the stable ability to detect three primary colors (RGB), in this paper, the spectral broadening of organic active layer based on tri-phase bulk heterojunction formed by P3HT:PCBM doped with narrow band material PBDT-TT-F which absorbs red light is investigated. The influences of PBDT-TT-F doping ratio on the morphology of active layer film and detector photoelectric properties are further analyzed. Finally, the operating mechanism of trap-assisted photoelectronic multiplication is discussed. On this basis, the detector with 350-750 nm wide spectrum is obtained where the optimum mixing ratio of P3HT:PCBM:PBDT-TT-F is 12:8:3. At a small reverse bias of 1 V, the values of responsivity and external quantum efficiency of the photodetector can reach 470, 381, 450 mA/W and 93%, 89%, 121% respectively under the illumination of three primary colors and its normalized detectivity to the RGB is close to 1012 Jones. Additionally, the maximum relative difference between each parameter and its average value is lower than 20%; the bandwidths are 5, 8, and 8 kHz respectively, which reach the imaging requirements for image sensors. The experimental results show that not only the absorption spectra of the active layer can be broadened but also the carriers collection efficiency of respective electrodes can be well maintained by adding a small quantity of spectral broadening material while keeping the microstructure of the original binary bulk heterojunction. Utilizing the reasonable combination of materials to form electron traps, photoelectronic multiplication can be realized by trap-assisted hole tunneling injection from the Al cathode into active layer, and thus improving the normalized detectivity. Moreover, in order to detect different light intensities, the hole injection barrier width should be controlled by the corresponding light intensity. The resulting OPD shows a good liner response to all three primary colors when light intensity increases from 0.1 to 10 mW/cm2. By adjusting the mixing ratio of the tri-phase materials, the stable ability to detect the primary color can be achieved. The present study paves the way for high responsivity broadband OPDs based on tri-phase bulk heterojunction.