The detection of third-party activities near pipes based on vibration signals can effectively enhance pipeline safety. In the framework of “data collection-sample segmentation-feature extraction-recognition model training-recognition strategy”， a random forest pipeline perimeter monitoring system by vibration optical fiber is proposed . The vibration signals of pickaxe， spade， hammer and electric hammer are collected by 5.35 km φ-OTDR optical fiber， and the 85 h environmental signals are also collected for comparison. The methods of sample segmentation and feature extraction are established to train a random forest model. A strategy of space-time matrix is proposed for correcting the model result， which will reduce the false alarm by 99.59%. In the test， the recognition rate of the perimeter monitoring system is 94.87%， while the false alarm rate is only 0.013 9%， which indicates that the system performs well under common urban environmental vibrations.