Aimed at system identification of a small-scale unmanned helicopter in the hover condition, a novel chaotic artificial bee colony algorithm (ABC) is proposed. Due to the nonlinearity of the helicopter′s mathematical model, the helicopter′s dynamic model is converted to longitudinal and horizontal decoupled linear helicopter models based on the small disturbance theory. In addition, the system identification problem is turned into an optimization problem, the intelligent bee colony is employed to seek. The ABC algorithm can evolve in a better direction with the information exchanges among the colony and the survival of the fittest. The chaotic operator is added to the ABC algorithm to help it jump out of local optimum and improve its global search ability. The model is validated and analyzed through the actual flight data by system identification. The results show that unknown parameters can be estimated based on the proposed algorithm. Our proposed algorithm has greater accuracy than genetic and traditional algorithms.