With the issuance of the ‘Guidelines for Risk Classification and Management, Safety Assessment and Monitoring of Existing Bridges’ in 2020 and subsequent revision in 2022 (hereinafter referred to as LG Bridges), analysis methodologies and inspection frequencies for bridges and viaducts on Italian roads and motorway networks have been defined at Italian level. In particular, following the definition of the Class of Attention (CoA) of the asset, obtained through a multi-level approach, the number and type of visual inspections to be carried out on each bridge annually is defined.
In the worst cases (High or Medium-High CoA), an inspection is to be carried out quarterly by qualified inspectors, who must view all the components of a single structure, reporting any defects.
In accordance with the Bridges Guidelines, these inspections include ‘an accurate photographic survey, geometric survey and survey of the main deterioration phenomena for each component of the asset… and the examination of both the extrados and the intrados of the bridge in each of their elements in order to have a complete and adequate visibility of the asset…’.
As a result, for each bridge or viaduct all structural components (i.e. beams, transverse beams, piers, abutments, supports, etc.) have to be analysed at ‘close range’, requiring an extremely costly on-site activity for each operator.
Austostrade per l’Italia manages approximately 3,000 km of motorways and, due to the particular and complex orographic structure of the national territory, has a very high number of assets (approximately 2,000 bridges and viaducts and the same number of overpasses). These works were mostly built between 1955 and 1990, and more specifically, about 75% between 1960 and 1990. Consequently, these structures have inevitably suffered the effects of time and the increase in the loads passing over them, and therefore need to be carefully analysed and inspected, in order to identify and therefore intervene on any degradation that may compromise the performance of the assets.
The inspection of bridges and viaducts using drones
In recent years, the increasingly rapid development of new digital technologies has aroused particular interest in all tools that can support infrastructure inspection activities, providing many advantages over the classical methods currently in use.
Among these, the use of Unmanned Aircraft System (UAS), commonly known as drones, can enable remote visual inspections, providing a valid and effective aid to ordinary surveillance activities. In particular, the new technologies developed in the field of photography and lidar have made it possible to combine a drone with very high resolution cameras and equally powerful lidar scanners, enabling very detailed surveys of assets to be carried out.
The use of drones allows multiple benefits over traditionally performed inspections. Firstly, time and costs associated with a single inspection are reduced, as it can be carried out much more quickly and with less effort related to the post-processing of inspection outputs. Secondly, the use of drones considerably reduces the impact on the operation of the infrastructure and therefore on the safety of the operators and inspectors themselves, especially in cases where, due to the morphology of the territory and/or the geometric characteristics of the structures, it is necessary to use special means such as lifting platforms, by-bridges, etc..
Moreover, inspections are more performing, benefiting from a better quality of output; in fact, by using drones it is possible to reach and inspect even the most remote points of the infrastructure, which are difficult to reach with normal vehicles, and to collect high-quality and user-friendly images and information, which are objective and repeatable over time, minimising the subjectivity linked to the experience and competence of the inspectors.
The ARGO 3D platform
The survey carried out on a bridge or viaduct through the use of a drone allows to obtain a large amount of data, typically unstructured, that needs to be processed and organised properly, in order to make the remote inspection of a structure possible.
To this end, Movyon, as a technological operator of the Autostrade per l’Italia group, has developed the Argo 3D platform, for the storage, categorisation and use of photographic and geometric data acquired through aerial or terrestrial surveys, enabling all the functions necessary to perform the remote inspection of an asset. ARGO 3D is part of ARGO, the extended asset management platform for monitoring and managing the life cycle of the assets.
The Argo 3D software was tested on about 30 assets on the Autostrade per l’Italia network, on which a laser-scanner and photogrammetric survey was carried out with drones and a digital model reconstructed. This model was provided to the operator to perform detailed analyses on the structural components and test the effectiveness of the remote inspection process.
Data Storage
The first step to make the data acquired in the field is to catalogue and organise it so that it can be easily accessed remotely, i.e. via the web. The Argo 3D platform provides technicians with a specially developed web platform for uploading the data collected on site, making it usable for subsequent processing.
The platform guides the cataloguing of the data according to pre-established criteria, which allow to automate both an initial validation phase of the raw data collected by the drone, and their subsequent processing, in order to produce more informative works such as, for example, the digital twin of the asset.
The reconstruction of the digital twin
The core of data processing is the production of the digital twin, through which it is possible to inspect the asset remotely, conveniently from the comfort of one’s own office. Using appropriate photogrammetry, it is possible to reconstruct the 3D model of the asset, using the photos and lidar scans collected, obtaining a true digital twin of the bridge, with very high resolution both in terms of accuracy and precision of the geometric survey, and in terms of texture.
The easy use of the 3D model with this level of resolution is enabled by modern multi-resolution 3D rendering techniques, which allow even very detailed models to be navigated with ease, enabling a complete inspection to be carried out very quickly.
The creation of the digital twin of the asset also produces another crucial piece of data to accompany the reconstruction, namely the position and direction of the camera that carried out the photographic survey.
This not only allows the user to see exactly how the photographic coverage of the asset was carried out, but also to switch between the 3D model and photos in a user-friendly manner; for example, by clicking on the 3D model it will be possible to see all the photos that have photographed the selected point, and by clicking on a photo it will be possible to see where it was taken in space.
Remote inspection and integration with BIM
The Argo 3D platform also provides inspectors with a Web application that provides, in addition to the visualisation of the digital twin of the asset, the possibility of taking measurements and entering various types of annotations directly on the 3D model, in order to identify and catalogue the defects of an asset.
The inspectors, in fact, thanks to multi-resolution rendering technology, will be able to zoom in very closely on the components of the asset, maintaining a very high degree of detail and thus allowing the identification of any defects directly on the 3D model.
Once the presence of a defect has been identified, it can be precisely located using a drawing tool that allows a polygon to be drawn on the 3D geometry, identifying the exact size of the defect itself. The traced polygon is associated with information about the type of defect detected, which is automatically stored in a specific database for subsequent analysis.
To enrich this process, the digital twin is accompanied by a BIM model of the structure, subdivided into its components, which allows semantic information to be associated with the location of the defect, i.e. the component of the bridge on which the defect was found. In this process, the operator always has the photos taken by the drone at his disposal as an aid to identifying defects.
The automatic identification of defects
The reconstruction of the digital twin is accompanied by another fundamental processing, namely the automatic identification of defects through the use of Artificial Intelligence techniques.
The use of modern networks for the semantic segmentation of images or the so-called ‘object detection’ (i.e. the identification of elements of interest by means of ‘bounding boxes’) makes it possible to suggest, when the remote inspection application is opened, a set of defects that must then be confirmed or rejected by the human operator.
The automatic defect identification software, developed and integrated into Argo 3D by Movyon, is able to recognise five classes of defects on concrete to date: cracks, exposed reinforcement, crawl spaces, efflorescence and moisture stains, while work is ongoing to increase this number. Approximately 15,000 images were used, 30 per cent of which were synthetically generated, to train and then test the artificial intelligence algorithms, which are currently able to identify defects very precisely.
By exploiting the synergy between photos and the 3D model, defects suggested by the AI will be visible both in the photos and directly in the digital twin, providing an accurate localisation of the defect in three dimensions. This ability to accurately localise defects in three dimensions makes it possible to observe and analyse the defect history and to compare the different surveys carried out over months or years, obtaining a powerful tool for predictive/forecasting analyses, to be used for optimisation in the planning of maintenance operations on assets.
Andrea Baldacci – Computer Vision Specialist at Movyon, Autostrade per l’Italia Group
David Buggiani – Head of Infrastructure Management at Movyon, Autostrade per l’Italia Group
Marzia Malavisi – Head of Structural Monitoring at Movyon, Autostrade per l’Italia Group