The transient vibration amplification characteristics of high-speed helical gears considering the back contact stiffness of gears is studied. In light of this problem, the dynamic meshing stiffness of helical gear is introduced. The parameter implies the coupling mechanism of meshing time and vibration displacement of the tooth surface, as well as the actual load contact state of the tooth surface at the high-speed transient condition. First, a 6-degree of freedom helical gear transient vibration model is established based on the dynamic meshing stiffness and backlash excitations. Then, the influences of different backlash and back sided impacts on the transient vibration characteristics are thoroughly analyzed. Finally, a helical gear transient torsional vibration test rig with enclosed power flow is built, and the transient vibration characteristics of helical gears with contact stiffness of the back teeth are verified. The research not only has great theoretical significance, but is also beneficial to the popularization and application of the helical gear transmission system in the aviation transmission and the new energy transmission system, and further improve the transient vibration and noise quality of high-speed gear system.