AbstractRapid reconnaissance of building damage is critical for disaster response and recovery. Drones have been utilized to collect aerial images of affected areas in order to assess building damage. However, there are two challenges. First, processing many aerial images to detect and classify building damage based on a consistent standard remains laborious and complex, necessitating a new automated solution to achieve accurate building damage detection and classification. Second, drone operations during disaster response rely primarily on human operators’ experience and seldom use the obtained building damage information to optimize drone mission planning. Therefore, this study proposes a new method, which automates building damage reconnaissance with drone mission planning for disaster response operations. Specifically, a deep learning method is developed to detect and classify building damages using a newly labeled dataset consisting of 24,496 distinct instances of building damage. This deep learning method is validated, achieving 71.9% mean average precision. In addition, building damage information is modeled and integrated into mission planning, in order to optimize drones’ task assignments and route calculations. A tornado disaster in Tennessee is used as a case study to quantitatively evaluate this methodology. The present study concludes that optimal drone mission planning during disaster response can be augmented using accurate building damage information acquired from deep learning methods.