Traditional balancing procedures need test weights for calculating correction masses, which are of inefficiency and high-cost. Anovel online unbalance identification method for rotor system, which is based on integrated genetic algorithm and particle swarm optimization (GA-PSO), is proposed. The core of this method is to decompose the unbalance of rotor system into number, location, mass and phase. Theoretical unbalance responses and measured vibration are complemented in direct and inverse, and integrated GA-PSO is applied to identify the unbalance. Firstly, a new objective function is constructed by using regularization method. The sparse-representation of unbalanced vector can be acquired by using GA, then the unbalanced number can be identified. Secondly, PSO is applied to identify unbalanced locations, masses and phases, and the accuracy can be enhanced by narrowing the initial value of PSO. The simulation and experiments results show that the proposed methods can predict the rotor unbalance effectively and provide guidance for the dynamics balance without trial weights. The cost of subsequent rotor system dynamics balance can be decreased, and the efficiency can be increased.