The nonlinear dynamic equation of vertical vibration system with two degrees of freedom is established including the time-vary stiffness and nonlinear damping. The system stability with different parameters is discussed using singular value theory, the equation and the first-order approximate solution is deduced with the aid of averaging method, and the influence of nonlinear effects is derived then. Meanwhile, numerical simulation verifies the theory results with different nonlinear parameters. A significant contribution of this study is helpful to modeling and analysis of cold rolling vertical system.