Carbon fiber reinforced polymer （CFRP） has been widely used in the primary structures of aircraft. Guided-wave-based damage diagnosis has a good prospect in the field of CFRP damage monitoring， but it is confined by structural uncertainty and expertise dependence. An innovative strategy incorporating energy spectrum and siamese convolutional neural network （CNN） is proposed. The method of energy spectrum can not only eliminate the high dependence of expertise， but also collect richer information for the deep learning model. The CNN can effectively reduce the quantity of model parameters and consequently alleviate the phenomenon of over-fitting. The accuracy of damage recognition and localization achieve 88% and 85%， respectively. Furthermore， no personal expertise is involved through the entire process， thus the capability of damage recognition can be more robust.