AbstractDue to the importance of civil structures and infrastructures, structural safety assessment or structural health monitoring has become a basic necessity for every society. Recent developments of sensing and data acquisition systems enable civil engineers to exploit machine learning methods based on data-driven strategies for structural safety assessment and damage detection. However, the choice of an appropriate machine learning method may be problematic, particularly under some challenging issues such as the negative effects of environmental and/or operational variability (EOV), and the necessity of estimating some influential unknown elements of parametric machine learning methods called hyperparameters. Accordingly, this article focuses on three main aspects: (1) comparing various machine learning methods, (2) developing semiparametric algorithms, and (3) proposing automated algorithms for hyperparameter optimization of semiparametric and parametric machine learning methods. An innovative automated output-only approach is proposed to qualitatively and relatively predict the levels of EOV in terms of strong or weak variability. The main contributions of this article include comparing various machine learning methods, which will enable civil engineers to choose the most appropriate technique, and proposing automated approaches to hyperparameter optimization and variability level prediction. Dynamic and statistical features extracted from measured vibration data of two full-scale bridges were considered to perform the comparative studies and investigate the proposed methods. The results demonstrated that the semiparametric methods provide the best performance when their unknown parameters are determined appropriately.