Aiming at the poor frequency sparsity of the vibration signal from a rotating machinery, which is interfered with strong noise and represented by the conventional FFT, a compressed sensing based on the sparsity in phase space is proposed to realize the signal reconstruction with orthogonal matching pursuit (OMP). First, the frequency sparsity is improved by reconstructing the original signal in a phase space and then principal components analysis (PCA) is implemented to extract the features in the constructed space; second, a random gauss matrix and OMP are adopted respectively to compress and reconstruct the improved signal. Analysis of the simulation signal and the misalignment signal of a rotor system suggest that the proposed method can enhance the frequency sparsity of the investigated signal. Moreover, the efficient compression and the accurate reconstruction of the investigated signal have also been realized.