Abstract: The seismic response of tall buildings is influenced by the time characteristics of ground motions. However, existing ground motion selection methods inadequately account for the impact of time characteristics, leading to noticeable irrationality in the results of nonlinear response time-history analysis of tall buildings. Based on the existing two-step ground motion selection procedure, a ground motion selection method considering the impact of time and frequency characteristics of ground motions based on the convolutional neural network (CNN) was proposed. The proposed method employed the elastic response diagram in the time-domain (RDTD) to represent the impact of the time characteristics of ground motions and consider the impact of time characteristics. The CNN model was selected and trained with transfer learning technique to establish the mapping relations between the characteristics of the RDTD and the seismic response of tall buildings. The trained CNN model was then used to evaluate the impact of the time characteristics of candidate ground motions, and select ground motions for the nonlinear response time-history analysis of building structures. The proposed method was compared with existing selection techniques on six structures with varying vibration periods. The results demonstrated that the CNN-based approach enabled accurate seismic response calculations with a fewer number of ground motions, achieving comparable outcomes to those obtained with a larger set of ground motions. This substantial improvement in the rationality of nonlinear response time-history analysis had promising implications for tall building seismic assessment and design.