The heavy atom phases are extremely important in solving crystal structures containing heavy atoms. Successive Fourier approximation and Karle recycling procedures starting with the heavy atom phases are two major methods of solving structures containing heavy atoms. However, difficulties may some times happen when the contribution of heavy atoms is too small to dominate sufficient number of phases or if the distribution of heavy atoms possesses a higher symmetry than that of the whole structure. It is worthy to investigate in these cases whether or not the heavy atom information can still be used to simplify the process of structure determination greatly. In this paper a method is proposed that the heavy atom phases may be used to restrict the permutation of the starting phases leading to a great reduction in number of the starting phase seta. This method incorporating with that proposed by the authors can improve greatly the efficiency of direct methods in solving structures containing heavy atoms.