A novel framework of cost-sensitive boosting algorithm is presented, which overcomes the drawbacks of traditional boosting algorithm which has low performance with unbalanced training dataset. A loss function is constructed, and the loss function is minimized by training decision rules. The new framework is used for rolling bearing system fault diagnosis. The comparison experiments are made with traditional Adaboost algorithm. Simulation results show that the proposed algorithm has better performance than the traditional one, when more fault dataset of rolling bearing system cannot be obtained.