AbstractComputer vision based on machine learning theory has been widely used in the surface damage detection of concrete structures, but the characterization of internal damage in concrete still remains a challenge for researchers. Aiming at this problem, we propose a nondestructive evaluation (NDE) method to classify diverse conditions of internal damage in concrete based on short-time Fourier transform (STFT) and convolutional neural networks (CNN). The STFT converts the self-resonant vibration signals into two-dimensional time-frequency images that can be used as the input data for the CNN. The training set is fed into the CNN for feature extraction and classification, and then the testing set is brought into the trained model for verification. Both a simple case of virgin state and damaged state, as well as a complicated case covering all four internal damage states were successfully classified with an excellent recognition rate of testing samples. The key CNN hyperparameters were optimized and the classification accuracy rate of spectrum images was as high as 98.8%. Optimal data set size was also found to balance the accuracy and efficiency. The findings in this work validate the feasibility of the CNN for the detection and differentiation of invisible damage in concrete nondestructively.