Pavement Inspection in Transport Infrastructures Using Unmanned Aerial Vehicles (UAVs)

This subsection provides an analysis of the key aspects addressed in the 13 documents selected during Phase 5. A summary of the main aspects identified in the reviewed documents is presented in Table 3, including reference number, country, pavement type (asphalt, concrete), road environment (urban, rural, airport), research focus and maturity, type of pavement distress, UAV and camera types, positioning, flight height, main data processing techniques and ground sampling distance (GSD) precision/error.

From the set of analyzed documents, the first study to demonstrate a practical application of an unmanned aerial vehicle for pavement condition evaluation was published in 2016 by Zhang et al. [46]. The study describes an early-stage application developed for an aircraft-based pavement evaluation method with the aim of assessing the technical feasibility of aerial triangulation (AT) and hyperspatial-resolution natural color aerial photography (HSR-AP). The study utilized images captured at a low altitude (5 m) using a tethered helium weather balloon to characterize pavement surface conditions. These images were captured using millimeter-scale HSR-AP, thus offering a ground sampling distance of 2 mm. Aerial triangulation, also known as Structure from Motion (SfM), was employed to process the images and estimate three-dimensional models from a sequence of two-dimensional images [46]. The results demonstrate that employing HSR-AP for image collection and using AT to process the images for generating orthophotos and millimeter-scale digital surface models (DSMs) can effectively be adopted for characterizing both the horizontal and vertical conditions of pavement surfaces. The results obtained with this technique were statistically comparable to those from conventional on-foot surveys [46], thus validating the potential for fully automating pavement distress evaluation. The pavement distresses studied included rutting, alligator cracking, and transverse cracking.

In 2017, two articles were published in the field of UAV-based pavement distress identification systems. The study referenced in [38] was carried out on asphalt pavements, with a focus on identifying the most effective learning algorithm for distress classification accuracy. On the other hand, the study referenced in [47] was conducted on concrete pavements. In [38], Pan et al. (2017) performed a study to identify the optimal learning algorithm that guarantees the highest distress classification accuracy (greater than 98%) while minimizing computational time. This study used a set of road pavement digital images in RGB channels collected through a multispectral Micro-Miniature Multiple Camera Array System (MCA) coupled to a UAV. Four common supervised learning algorithms, namely K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Artificial Neural Network (ANN), and Random Forest (RF), were employed to identify pavement distresses. The resolution of images collected with the UAV (1 pixel ≈ 13.54 × 13.54 mm of pavement area) enabled the use of the Multiresolution Segmentation (MS) algorithm integrated into the software eCognition Developer Software 9.0 to extract information about potholes and cracking from pavement images. MS identifies individual image objects based on pixel size and merges them with neighbors using the relative homogeneity criterion. This criterion combines spectral and shape criteria and is determined by a scale parameter. According to the authors, choosing an appropriate scale parameter to simultaneously extract cracks and potholes is a challenging task. The analysis of the performance of the four learning algorithms showed that the RF exhibited the best results, boasting higher classification accuracy and the shortest running time [38]. The study referenced in [47], carried out by Ersoz et al. (2017), proposes a system that combines image processing and machine learning techniques to identify and classify cracks in rigid pavement. This system consists of two main steps: the detection of crack candidates and crack classification. In the crack candidate detection step, images obtained from UAVs are segmented to separate objects with and without cracks from the background using image segmentation and enhancement techniques. The geometric properties of these objects are then collected and used to train an SVM model for distress classification [47]. To test and validate the performance of the algorithm, 109 images of rigid pavements were collected using the UAV DJI Inspire 1 Quadcopter at different heights (0.5 to 3.0 m). These images were processed, and the crack detection algorithm extracted 157 bodies, of which 124 were cracks. Subsequently, the geometric properties were determined using 80% of cases for training and 20% for testing the SVM model. The performance of the crack classification algorithm was evaluated using a confusion matrix, which revealed an accuracy of 97.0% in distinguishing crack and non-crack regions. The paper concludes that the proposed UAV-based system for monitoring rigid pavements is promising and offers a cost-effective solution compared to existing systems. The authors acknowledge some limitations, such as performance issues related to shadowy or low-resolution images, and suggest future improvements, such as incorporating more cases to enhance algorithm accuracy and addressing additional crack types.

In 2018, Pan et al. (2018) presented an application of a system composed of a multispectral camera attached to a six-rotor drone (UAV MSI) to capture images of the pavement surface. They used RF, ANN, and SVM learning algorithms to detect distress from the collected images [30]. The case study took place in the rural area of the city of Shihezi, Xinjiang, China. Given the high resolution of the pavement images collected, the Multiresolution Segmentation (MS) algorithm incorporated in the eCognition Developer software was employed to conduct the segmentation of the pavement images. The study revealed that spatial features such as texture and geometry contributed more to the accuracy of crack and pothole detection than spectral features. According to the authors, the classification data obtained can achieve better performance and require less running time when using an RF classifier with 18 trees. The overall classification accuracy for cracks, potholes, and non-distressed pavement was 98.3%. A comparative study of several resolution images showed that the spatial resolution should not exceed the minimum scale of pavement distress occurrences to be identified; otherwise, minor damages, such as low-severity cracking, can be missed during the segmentation procedure [30]. This publication can be considered an extension of the 2017 study conducted by Pan et al. [38], as both articles discuss the use of MCA cameras coupled to a UAV to capture pavement surface images and the application of machine learning algorithms (RF, ANN, and SVM) for pavement distress detection.

In 2019, two articles were published, each describing a different approach to detecting specific types of road pavement surface distress using image data collected by the UAV. The study conducted by Tan et al. (2019) proposes an automatic method for building 3D models aiming to detect road pavement distress from oblique photogrammetric images obtained by UAVs [40]. In their study, the images captured by the UAVs were processed using the standard photogrammetry software Pix4Dmapper 4.1 to build 3D models (also used by Pan et al. (2018) in [30]). The region-growing algorithm (image segmentation method) was then applied to classify the 3D model into two parts: the road pavement and the non-road-paved areas, which allowed for the exclusion of irrelevant surroundings from the pavement surface in the 3D models. Using the 3D models, the authors developed and implemented an algorithm based on the Graham scan algorithm to detect road surface distress and extract measurements such as distress length, width, and height/depth. As a result, it was determined that the approach can precisely detect areas of pavement damage with an approximate margin of error of 1 cm in height/depth measurements [40].

In the same year, Wang et al. (2019) presented a study that aimed at detecting crack junctions (transverse, longitudinal, and alligator cracks) of any type and size through the analysis of pavement surface images [39]. The proposed method was tested using a set of rigid pavement images from a public dataset, the SDNET2018 [58], and flexible pavement images collected by UAVs on the G45 highway in China. The study aimed to characterize crack distress structures from different pavement condition surface images. For this purpose, the contrast between the cracks’ junction and pavement background was improved by removing large interferences and background elements, such as lane markings, shadows, and dirt debris. Then, based on the structural characteristics of crack curves, a correlation structure index was proposed to locate possible cracks in the images. After an iterative tensor voting process, the unified ball tensor structure was used to extract actual crack junction data from the crack candidates. This is a mechanism that employs tensor-based structure characterization and propagates the structure indication to neighbors through voting. Through tensor voting, the structure of the crack junction is improved, and some interferences, such as stone mixtures and dirt debris, are gradually eliminated from the images. Thus, the ball tensor is used to uniformly characterize the crack junction of different sizes, types, intensities, or those formed by crack curves with different orientations. The experimental results demonstrated that the method could detect crack junctions with a correctness of 0.891 and a completeness of 0.887, and it can be applied to different crack types and sizes in both rigid and flexible pavements, despite different sources of noise and interference (material texture, imaging condition, etc.) [39].

In 2020, Romero-Chambi et al. (2020) developed a method for measuring the geometric characteristics of flexible road pavement potholes, including their depth, width, and volume, through 3D models generated from images obtained with a UAV and processed using software based on the Motion-MultiView Stereo (SfM-MVS) technique [19]. A case study was conducted to assess the accuracy (error evaluation) of 3D models created from images, considering variations and combinations of different flight and image capture parameters. Specifically, variations and combinations of camera view angles (vertical and oblique), image overlay rate, and flight height were tested. The results indicate that although including image capture angles other than 90° has a positive impact on the accuracy (error reduction) for measuring potholes, it does not justify the additional time spent in acquiring extra images, as the processing time nearly doubled. Additionally, no specific oblique angle or clear trend was found to ensure error reduction. Among the studied geometric characteristics, width exhibited the lowest level of error, followed by depth and volume. The study concluded that the methodology is applicable for flight heights between 10 and 15 m. For higher heights, the error level does not allow for an adequate representation of the geometric characteristics of the potholes when considering an FOV of 84° and a 20 MP image. On the other hand, for lower image capture heights, the process becomes extremely laborious as it requires manual flights, and the GPS becomes sensitive, leading to a loss of precision [19]. When comparing the approaches proposed by Romero-Chambi et al. (2020) [19] and Tan et al. (2019) [40], it is evident that the former represents a considerable improvement in this field of study, as the error levels presented are less than 1 cm.

Also in 2020, Silva et al. introduced a platform capable of detecting road pavement distress along public transport routes using UAVs and a Multi-Agent System (MAS) based on PANGEA (Platform for Automatic coNstruction of orGanizations of intElligent Agents) [8]. PANGEA was responsible for coordinating the various components of the architecture through ubiquitous computing techniques using deep learning algorithms. The platform allows the identification of pavement distresses and provides the most important results from the data collected during inspections, thus optimizing time, costs, and specialized labor. The MAS was used to coordinate and communicate with all the entities involved in this task, allowing for dynamic reconfiguration of the system. The authors emphasize that employing an MAS streamlines case study development and ensures compatibility among the different platform components. The PANGEA-based system facilitates seamless communication and information exchange between the drone and platform, adapting the work to contextual requirements. The distress identification process relied on the YOLOv4 algorithm, while communication with different agents was facilitated through RFC 149 IRC (Internet Relay Chat), a text-based real-time multi-user message exchange system known for its energy-efficient communication capabilities [8,59,60]. The use of RFC significantly enhanced UAV battery autonomy during flight, thus increasing the energy available for pavement inspection. A total of 600 images were captured, resized, and labeled, resulting in 568 labeled images. To expand the dataset, multiple versions of each image were generated using zooming techniques, effectively doubling the dataset size to 1362 images. These images were utilized for model training and evaluation, with 70% allocated for training, 20% for validation, and the remaining 10% for testing the effectiveness of the trained model. The authors did not observe any variation between the results obtained from 4K video image frames captured at 70 and 90 m height and the UAV’s operating speed between 15 and 25 km/h. The level of accuracy in distress detection was notably high, above 95%. However, although several crack detection techniques were tested and analyzed in the study, none yielded results greater than 47%. The authors attribute this lower performance to the dataset used for testing the automatic recognition process, which did not encompass European road cases and featured cracks with notably larger openings (i.e., conditions distinct from those identified in the case study), which primarily included low-severity cracking and small potholes [8].

In 2021, Nappo et al. proposed a methodology for characterizing cracks in asphalt road pavements in areas affected by landslides. This method utilizes 3D models reconstructed from UAV images. The study was conducted in the Province of Como, located in northern Italy, with the primary objective of providing a valuable tool for local authorities. This tool enables them to select and inspect road sections damaged by landslides in areas prone to such geological events. The methodology employs the use of 3D and 2D photogrammetry tools to identify, describe, and classify the severity of both longitudinal and transverse cracks. The research integrates various data sources, including satellite imagery, field surveys, and UAV images, to gather information about the road network and landslide occurrences [53]. The methodology consists of four distinct phases. In the first phase, road segments that are exposed to landslides and potentially have pavement damage are selected based on thematic maps and InSAR (Interferometric Synthetic Aperture Radar) data. Once the relevant road sections are identified, high-resolution UAV images with a resolution of 16.8 MP are collected at an altitude of 10 m using a DJI Phantom 4 Pro with an 80% image overlap. The second phase involves the use of photogrammetric techniques, specifically Structure of Motion (SfM), to create 3D models of the selected road sections. Subsequently, the point cloud obtained from the 3D model is processed to extract road surface information and analyze pavement deviations and geometric attributes in the third phase of the process. A multi-criteria binary classifier is trained to identify and evaluate the severity of pavement damage. In the final phase, an edge detection algorithm is applied to the 2D orthorectified image to identify damaged edges. These 2D edges are then overlaid with the 3D point cloud to gather additional information and determine the extent of pavement damage [53]. When applied to a practical case study, the results showed a clear correlation between different types of road damage and the presence of landslides. Longitudinal and transverse cracks were mostly observed within the landslide-affected areas, while other forms of damage, such as fatigue cracks, were found both inside and outside these landslide zones [53].

The most recent open-access documents analyzed are from 2022. Among the four selected documents, the first one, authored by Sierra et al., delves into a research study on the use of UAVs for pavement maintenance and monitoring [57]. The study aims to develop an innovative method that combines UAVs, reality modeling, and machine learning algorithms to create a digital twin of the pavement structure. This digital twin facilitates the automated detection of pavement distress. To accomplish this, UAV flights were conducted at an altitude of 60 m over a transited road, and the captured images were used to generate a comprehensive reality model of the pavement. Subsequently, machine learning algorithms, specifically CNNs, KNN, RF, and XGBoost, were applied to identify and assess pavement distress. The study shows that this UAV-based methodology, using reality modeling and machine learning, yields results comparable to traditional approaches while offering cost and time efficiency. Moreover, the researchers also discussed different UAV flight methods for data acquisition, including predetermined flight paths, manual flight control, manual still images, and manual video. Each method presented distinct advantages and disadvantages, with the selection of the most suitable approach contingent on factors such as the road network environment, desired data coverage, level of detail, and available resources. The manual still image method was deemed the most suitable for research purposes, as it provides precise data with minimal waste. Overall, the study emphasizes the considerable potential of UAVs and related technologies, such as reality models and machine learning, in enhancing the assessment and maintenance of pavement structures [57].

The article authored by Inzerillo et al. (2022) describes the use of UAVs for detecting structural cracks in concrete structures and road pavement surfaces [56]. According to the authors, the quality of images collected by UAVs can be affected by vibrations and distances, thus resulting in a loss of critical information that makes crack detection difficult. To address these challenges, the authors employed Super-Resolution Reconstruction (SRR) algorithms aimed at enhancing the resolution and precision of the captured images. In a case study conducted by the authors in Palermo, Italy, low- and high-resolution images of a road surface were collected using a DJI MAVIC 2 Pro UAV equipped with a 20 MP camera. The UAV was flown at heights of 49.7 and 51.6 m. A morpho Super-Resolution algorithm (SRa) was applied to the low-resolution image dataset to improve image quality. This dataset was then compared with high-resolution images and ground-based image sources. The results showed that the super-resolution approach significantly improved the accuracy of the 3D model, as evidenced by a reduction in root mean square error (RMSE) from over 10 cm to a range of 0 to 1.5 cm. By enhancing the quality of low-resolution images, the super-resolution approach effectively enables the detection and measurement of pavement distresses such as cracks and potholes. This, in turn, provides invaluable information for the formulation of maintenance strategies [56].

In the study performed by Schelle et al. (2022), the main objective is to propose a communication and integration system that enables safe and reliable drone flights in controlled airspace. This system addresses various challenges and risks associated with aerial surveying operations, including those related to routine maintenance, such as inspecting airport runways for damage [55]. The document highlights the challenges of integrating an unmanned aircraft system into the European national airspace and introduces the concept of U-Space, aimed at developing automated, interoperable, and sustainable solutions for Unmanned Traffic Management (UTM). To achieve this, the authors introduced a device called the Multi-Mode Transceiver (MMT), which combines Mode S transponder technology, including ADS-B (Automatic Dependent Surveillance-Broadcast), with a cellular interface. The MMT is designed to enhance the visibility and connectivity of drones in environments involving both manned Air Traffic Management (ATM) and UTM. It plays a crucial role in enabling drones to interact collaboratively with air traffic control (ATC), obtain flight authorizations, and ensure the safety of operations. The system includes various UTM services, such as flight planning management, tracking, and traffic information. The research evaluates the performance of MMT and the collaborative interface with ATC, along with a tactical collision prevention method based on energy reservation. The study provides valuable insights into the implementation and functionality of MMT in a real-world UTM environment, supporting the development of UTM systems for the safe integration of UAVs in airspace [55].

Furthermore, the article by Wróblewska et al. (2022) discusses a research methodology for evaluating pavement conditions, particularly in areas with discontinuous deformations that can affect road safety [54]. The case study was conducted in the southern province of Silesia, where an important regional road was impacted by deep mining activities, resulting in pavement surface irregularities and transverse cracks. The primary aim of the research was to assess pavement regularity and identify discontinuous linear surface deformations through the application and comparison of various measurement methods. These methods included laser profilometry, geodetic measurements (leveling and GPS positioning), and low-altitude UAV photogrammetry to determine the effectiveness of these approaches. For this purpose, a dataset comprising 233 aerial images of the road surface, collected at 60 m height with a UAV equipped with a 1-inch sensor camera with 20 MP high resolution during an 18-min flight, was used. The images were processed using photogrammetry techniques to create a 3D model of the inspected pavement surface, which enabled the identification of discontinuous linear deformations and the generation of a hypsometric map displaying elevation variations. The comparative evaluation of UAV photogrammetry results with other measurement methods, such as mobile laser profilometry, demonstrated that both methods yielded a significant number of measurement points or a high point density. The study concluded that both traditional methods, such as laser profilometry, and modern UAV-based techniques can effectively identify pavement damage. However, it highlighted that UAV-based photogrammetry offers a high level of data detail and efficiency in detecting discontinuous linear surface deformations, complementing the other methods [54].

After presenting the main aspects of the 13 selected documents, it is possible to conclude that UAV-based pavement condition assessment practices are promising. Several aspects, including their cost-effectiveness when compared to existing systems, the quality of collected data, and the potential for the development of systems for pavement distress recognition, all point to the full automation of pavement condition assessment in the near future. Most of the studies compare UAV-based approaches with traditional research methods to validate results and identify areas for improvement in UAV pavement inspection. Performance, the incorporation of additional features, different pavement conditions, and several types of distress identification are among the most discussed topics. Notably, the most recently published research (from 2020 to 2022) tends to incorporate the latest advances and technologies, such as 3D modeling [19,53,56] and image data processing techniques [8,19,53,54,56,57], benefiting from the progress made in the field since the first publications on this subject (2016) [46].

When analyzing the authors, Wang, Y., Li, Y., and Zhang, X. are identified as relevant, and there are only two articles [30,38] that share common authors, namely Pan, Y., and Zhang, X. This suggests a collaboration or shared research interests, indicating potential for future collaboration between the remaining authors. Regarding the country of origin, 31% of studies were developed in China, 46% in European countries, and 23% in other parts of the world, indicating a reasonably diverse range of contributions from different countries. It is also worth noting that only one article [47] focuses on rigid pavement, while the remaining articles study flexible pavement [8,19,30,38,39,40,46,53,54,55,56,57]. Furthermore, seven articles (approximately 54%) focus on road pavements in rural environments [8,30,38,39,46,53,54], five articles on road pavements in urban environments (approximately 38%) [40,46,47,56,57], and one on an airport environment [55], suggesting the relevance of pavement assessment for different transport infrastructures and environments.

The analysis of the selected documents underscores the fact that most of the studies are directed towards researching UAV image collection and processing techniques to obtain 3D models that represent the pavement surface for distress identification, measurement, and condition evaluation. Four articles present advanced research maturity [30,38,39,57], employing techniques to retrieve and process data on pavement surface distress to identify its characteristics efficiently and operationally, such as type of distress, dimension, and severity. However, none of the studies analyzed more than three different types of distress, usually focusing on cracks and potholes, except for publication [40], which analyzed four, namely piling up (road bulges), potholes, subsidence (road cavities), and corrugation. It is noteworthy that cracks and potholes are the most studied pavement distresses, both being examined together in over 38% of the articles [8,30,38,56,57]. Specifically, crack detection and analysis have been important research areas in pavement condition assessment using UAVs [8,30,38,39,46,47,53,56,57]. Other pavement distresses, such as discontinuous deformations [54], rutting [46,57], subsidence, pilling, and corrugation [40], are mentioned in only one or two articles, suggesting that they are relatively less studied in the context of UAV-based pavement evaluation. This is likely due to UAV-based methods being a recent and developing approach, not yet encompassing the detection of all pavement distress types, thus currently focusing on the most common and significant ones.

Quadcopters, with the DJI Phantom 4 Pro model being notably the most utilized, are prevalent in these studies [8,19,40,47,53,54,55,56,57]. The quadcopters are typically combined with cameras having resolutions ranging from 16.8 MP to 20.9 MP, enabling the capture of highly detailed aerial images. It is noteworthy that none of the articles refer to the spectral or radiometric resolution of imaging sensors. The choice of cameras is driven by the specific needs of each study, taking into account factors such as the desired resolution, georeferenced accuracy, equipment availability, and the objectives of the pavement inspection.

The selection of the positioning system primarily hinges on the required accuracy and prevailing environmental conditions. Among the articles included in this study, GPS/GNSS in UAVs emerges as the most common choice, facilitating precise georeferencing of images [19,40,53,54]. Some articles also mention the exclusive use of GPS [8,47,56,57] or GNSS [55], depending on precision needs and satellite availability.

The choice of flight height for pavement distress inspection exhibits significant variation among the analyzed documents. As depicted in Table 3, UAVs’ flight height ranges from 5 to 60 m, with some variations, such as flights as low as 0.5 m in a specific study [47]. The determination of the flight height depends on various factors, including the desired image spatial resolution, the accuracy of distress measurement, terrain characteristics, UAVs’ flight regulations, and the specific study objectives. Low-altitude flights are conducted manually and can capture finer details, while higher-altitude flights can cover a more extensive area in a single flight and be automatic. This variability underscores the flexibility of UAV survey techniques to adapt to different research contexts and the specific requirements of pavement analysis.

Regarding the autonomy of UAVs, values ranging from 20 to 30 min per battery charge are typically reported [40,53,54,57,61]. This may require interruptions during inspection; however, it is expected that autonomy will be extended to over 40 min in the near future. The recorded UAV’s speed usually varies from 3 to 5 m/s [8,40,47,53,54,55,57].

Several UAV image data processing techniques have been effectively employed to obtain image restitution, 3D models, and distress detection and classification. In terms of image restitution, the studies under scrutiny have embraced the principles of photogrammetry, such as aerial triangulation (AT) and SfM processes [19,46,53,54,56]. For the creation of 3D models, several software packages, such as Pix4Dmapper, Multi-view Stereo, or Agisoft Metashape, were used [30,40,56]. To enhance resolution, super-resolution algorithms were also used [56]. For object recognition, the eCognition Developer software and the YOLO system were adopted [8,38]. Concerning distress detection and classification (feature extraction), techniques such as the tensor voting algorithm and machine learning algorithms like CNN, KNN, ANN, SVM, RF, and XGBoost were applied [30,38,39,47,57]. Among these, SVM and RF have shown notably high levels of accuracy in detecting pavement damage, often exceeding 98%. Notably, the use of deep learning algorithms and Multi-Agent Systems (MAS) [8] suggests a more advanced approach in terms of computational intelligence.

As for the ground sampling distance (GSD) or spatial resolution of the images, some authors have reported values ranging from 0.20 cm/pixel to 1.50 cm/pixel [19,38,46,53,54]. The former was obtained from UAV data collected at a height of 5 m, and the latter at a height of 60 m. Regarding the verified distress measurement errors, they generally fall below 1 cm for UAV data collected at low altitudes (less than 5 m) [46], around 1 cm for data collected between 10 and 15 m [19,40,53], and more than 1.5 cm for greater heights above ground [54,56,57]. The analysis of the presented studies suggests that flight heights in the range of 25 to 30 m generally allow automatic flights and an acceptable level of pavement distress detection and precision, particularly for medium and high distress severity levels.