Abstract: The fatigue damage accumulation of asphalt materials under variable amplitude loading exhibits an obvious nonlinear characteristic. The traditional Miner’s linear fatigue damage accumulation rule fails to characterize the nonlinear fatigue damage accumulation (NLFDA) of the asphalt binders under different variable amplitude loading sequences. This study aims to develop an NLFDA model considering the loading sequence effect for accurately characterizing the effect of different loading sequences on the fatigue damage accumulation of asphalt binders. The fatigue damage of asphalt binders was characterized using dissipated pseudo strain energy (DPSE), and the fatigue damage accumulation laws of the asphalt binders under constant amplitude loading were analyzed by carrying out the stress-controlled constant amplitude fatigue tests of asphalt binders. The fatigue damage accumulation laws of the asphalt binders under variable amplitude loading were analyzed by carrying out the stress-controlled variable amplitude fatigue tests of asphalt binders under the loading sequences of low-high and high-low. The NLFDA model considering the loading sequence effect was developed using the damage equivalent criterion to investigate the effect of loading sequence on the fatigue damage accumulation. This study concludes that the fatigue damage of the asphalt binders under stress-controlled model exhibits an nonlinear evolution trend, which increase slowly initially and then increase sharply. The fatigue damage of asphalt binders under constant amplitude loading accumulates linearly obeying the Miner’s rule, and the cumulative fatigue life fractions are equal to one. The fatigue damage of asphalt binders under variable amplitude loading does not obey the Miner’s rule and accumulates nonlinearly. The accumulative fatigue life of asphalt binders under the variable amplitude loading sequences of low-high and high-low increases and decreases with the increasing first life fraction, respectively. The corresponding cumulative fatigue life fractions are greater than one and less than one, respectively. The developed NLFDA model can address the defects of Miner's rule and can effectively characterize the effect of variable amplitude loading sequence on the fatigue damage accumulation of the asphalt binders.