Abstract: For the simulation of the problem on infinite domain, finite element method (FEM) is usually combined with transmitting formulas, and the precision of the simulation is mainly determined by the FEM methods. Spectral element method (SEM) takes advantages of both spectral method and FEM, and achieves higher precision for integration in space domain compared with FEM. This paper investigates the application of transmitting formulas in details when the interior domain uses the method of SEM. In this paper, we illustrate the use of SEM with transmitting formulas firstly. Meanwhile, the stability problem of transmitting formula is discussed in depth. The high frequency instability occurs more frequently for the interpolation in the time domain than in the space domain, and the interpolation coefficient in space domain should be in an appropriate range. Recognizing that the gamma operator is an effective measure to enhance the precision of simulation, we give a concrete method to set the value of gamma. Higher-order transmitting formulas with gamma operator may result in low-frequency drift. The paper points out that using order-reduction method is effective to get stable transmitting boundary conditions with high precision.