On-destructive testing technology is widely used in the modern industry. Ultrasonic infrared thermal imaging uses ultrasonic as an excitation source and can be used to detect a variety of workpieces. However, the edge of the image is blurred and the contrast is not high, and there is also speckle noise due to thermal conduction effects and air scattering. In this paper, a method for defect detection and classification of ultrasonic infrared thermography is proposed to solve the above problems. First, the CLAHE (contrast limited adaptive histogram equalization) method is used to increase the contrast, and the Butterworth filter is used to reduce the noise. Then, the defects are detected according to the local variance of the image and their center is located by morphological processing. Experiments show that the efficiency and accuracy of the proposed method. This paper provides a convenient and productive method for defect detection and classification of ultrasonic infrared thermal imaging.