Spectral-domain optical coherence tomography (SD-OCT) systems generally have nonlinear sampling problems in wavenumber domain. In order to realize the precise positioning of the discrete interfaces and the high-quality reconstruction of OCT images under conventional fast Fourier transform, it is necessary to solve the precise calibration problem of the absolute wavenumber of the discrete sampling points in the spectrometer. In this work, an absolute wavenumber calibration method is proposed based on the absolute phase of the characteristic spectral line and the constraint polynomial fitting phase under precise optical path difference. In the sample arm of the SD-OCT system, the metal gauges with precise thickness difference are used to obtain the absolute phase value of the characteristic spectral line, and the phase wrapping times corresponding to the characteristic spectral line are further accurately solved. Thus, this method overcomes the 2π ambiguity of spectral phase in conventional interferometric phase methods. At the same time, combined with the polynomial fitting phase of the high signal-to-noise ratio region under window constraint, the accurate calibration of the absolute wavenumber of each sampling point is realized. Finally, comprehensive comparison between the proposed method and the traditional resampling method in terms of discrete interface positioning, axial resolution and image reconstruction quality verifies the significant advantages of this method.