Abstract: To efficiently model the transient responses of a dam-reservoir system under seismic excitations, a continued fraction formulation coupling FEM formulation was proposed. Through combining the dam FEM equation and using the dam-reservoir coupling term, a transient analysis iterative algorithm was presented to obtain the transient responses of a dam-reservoir. The transient responses of a gravity dam under horizontal seismic excitations were analyzed by using the iterative algorithm. The computational efficiency comparison was made between the continued fraction formulation and the dynamic stiffness matrix and the dynamic mass matrix. Numerical example showed the accuracy and efficiency of the iterative algorithm for the transient responses of a dam-reservoir. The presented method inherits the advantages of scaled boundary finite element method (SBFEM), such as high accuracy and few discretized elements, it also avoids convolution integral, results in an efficiency improvement, and provides an approach with high efficiency for the transient analysis of a dam-reservoir.