Based on the fractal structure of strange attractor and self-affine property of time series, a method is proposed for predicting chaotic time series. The algorithm first exploits the iterative function system to track current chaotic trajectory and selects the segment which possesses the best self-affine property of the time series statistically. Then the prediction model is constructed according to attractor and coverage theorem. To illustrate the performance of the proposed model, simulations are performed on the chaotic Mackey-Glass time series, EEG signal and Lorenz chaotic system. The results show that the chaotic time series are accurately predicted, which demonstrates the effectiveness of the model.