Rotor effective amplitude assisted American petroleum institue （API） 617 amplitude limit criterion is proposed for finding the incipient abnormal vibration of the centrifugal compressor rotor due to the faint fault phenomenon. It improves the versatility of API amplitude limit criterion that not only for the fault recognition. According to the practical condition of the centrifugal compressor， a series of rotating speed nodes in the start and stop process are first designed under both the rigid and flexible running states. Then， a criterion is defined for choosing the coefficient of API amplitude limit. Finally， the primary decision rule with API amplitude limit criterion and the secondary decision rule with the effective amplitude are developed based on the coefficient. The slight typical faults detection of a Bently rotor test bench shows that the proposed method realizes the incipient abnormal prognosis of the rotor system while API 617 amplitude limit criterion is failure. The monitoring of rubbing induced by the rotor bending of an axial flow compressor shows that the secondary decision rule is suitable for the incipient abnormal detection due to the faint fault phenomenon. However， the primary decision rule is only applicable in fault recognition.