Aiming at the early fault detection and diagnosis of rolling bearings during its successive operations， a fault diagnosis method of rolling bearings based on graph modeling feature extraction is proposed. The signal is modeled by the short-time Fourier transform and graph theory. First， the time-frequency map of the signal is obtained by the short-time Fourier transform， extracted the spectrum map of each window. The frequency range is divided into a certain number of frequency segments， to calculate the energy of each frequency segment and to build a graph model with it. Second， the Martingale-test is used to fault detection， computing an adjacent matrix entropy value as a feature vector training support vector machine （SVM）. Finally the SVM classifer is used to identify the fault type. Two public data sets are used to validate the proposed method on the fault detection and fault diagnosis， respectively. The results show that the method can effectively detect and diagnose the bearing fault， and the effect is better than other methods.