Filtering technology is the key to accurate phase reconstruction in off-axis digital holography. Due to the limitations of resolution of charge coupled device (CCD) and off-axis digital holography itself, the filtering process of the step-phase objects is often accompanied by spectral loss, spectral aliasing and spectral leakage when non-integer periods are intercepted. At present, much research has been done on adaptive filtering in the frequency domain, but the above problems have not been fundamentally solved. In this work, the influence of spatial filtering on the accuracy of step-phase reconstruction is first analyzed theoretically. The analysis shows that even if the size of the filter window is equal to the sampling frequency of the CCD, the reconstructed object cannot retain all the spectral information of the object due to the limitation of the resolution power of the CCD itself. In addition, in the off-axis holographic recording process, considering the interference of zero-order terms and conjugate terms, the actual filter width is usually only 1/24 of the sampling frequency of the CCD, at which the average absolute error of the step is about 10% of the height of the step, the oscillation is relatively severe, and after further smoothing filtering, the details of the object are lost, the edge is blurred, and the tiny structure cannot be resolved. Second, according to the definition of discrete Fourier transform, the one-dimensional Fourier transform of a two-dimensional function integrates only in one direction, while the other dimension remains unchanged. When performing one-dimensional Fourier transform along the direction perpendicular to the holographic interference fringes and performing one-dimensional full-spectrum filtering, the distribution of reconstructed object light waves in the direction parallel to the fringes follows the original distribution, is not affected by the filtering, and has high accuracy. Therefore, by combining the reconstructed light waves obtained from one-dimensional full-spectrum filtering of two orthogonal off-axis holograms, an accurate two-dimensional differential phase can be obtained, which provides a basic guarantee for the accurate phase unwinding of Poisson equation. On this basis, a spectral lossless phase reconstruction algorithm based on orthogonal holography and optical experiment method is proposed. In this paper, the ideal sample simulation, including irregular shapes such as gear, circle, V, diamond, drop, hexagon and pentagram, and the corresponding experiment based on USFA1951 standard plate and silicon wafer are carried out. The AFM-calibrated average step heights of the standard plate and the silicon wafer are 100 nm and 240 nm, respectively. The experimental results show that compared with the currently widely used adaptive filter phase reconstruction, the proposed method naturally avoids spectrum loss, spectrum aliasing and spectrum leakage caused by filtering, the reconstruction accuracy is high, and it is suitable for three-dimensional contour reconstruction of any shape step object, which provides a practical way for reconstructing the high-precision phase of off-axis holography.