Abstract: In order to investigate the dynamic response of a single stage gear system coupled with rotors, the vibration equations of torsional vibration of the gear-rotor system are established while the effect of gear backlash, time-varying mesh stiffness and static transmission errors are considered. Based on the bifurcation diagrams, top Lyapunov exponents spectrum, fall diagrams of FFT spectrum and pcolor of displacement-time, the periodic and aperiodic motions of the system are analyzed with varying parameters, while the characteristics of frequency and time domains of the system are discussed. The Hopf bifurcations and the phenomenon of progressive instability amplitude resulted from these transformations are obtained. Dynamic structures of the system, as well as bifurcation curves in the parameter plane, are drawn, from which the distribution areas of periodic and aperiodic motions and variety of bifurcations are described.