Optical investigation of aircraft surfaces after a lightning strike event
Conclusion of the research project "Automati, Multicopter-based Indoor Inspection of Large Surfaces" by IRT and WZL
A commercial aircraft is struck by lightning on average once a year. In order to rule out possible damage to the aircraft, the maintenance personnel must carry out a time-consuming visual inspection of the outer skin. In order to make this inspection process easier to handle, the Institute of Automatic Control (IRT) and the Chair of Production Metrology and Quality Management of WZL have developed a multicopter-based system for efficient and faster defect identification and localization on the aircraft as part of the research project "Automatic, Multicopter-based Indoor Inspection of Large Surfaces" - AMIIGO for short. This mobile unit enables simple, non-destructive inspection by digitalizing the entire surface of the aircraft using industrial camera technology. In addition to robust flight control, the overall system includes highly accurate localization and fully automatic image evaluation.
Navigation and control algorithms enable real-time automation of flight inspection
The automation of the multicopter's flight behavior is successively realized by path planning, trajectory optimization, flight control and collision avoidance in real time. The navigation and control algorithms implemented by IRT are carried out completely autonomously on the drone itself. A sensor fusion first calculates every 10 milliseconds an indoor position of the drone in the maintenance hangar accurate to a few millimetres. On the basis of the current position and a previously optimized path over the surfaces, all control commands necessary for the flight can be determined. In parallel, possible obstacles are dynamically detected with a laser scanner in order to avoid collisions during the flight.
Automated defect detection through Machine Vision and Machine Learning
WZL realizes the automated defect recognition in captured images as well as the visualization of the calculated positions of the pinhead-sized defects. Classic algorithms of machine vision and modern methods of machine learning are used. In concrete terms, the identification of maintenance-relevant surface defects in the location-indexed image data is carried out by a hybrid of a classic corner recognition algorithm and a convolutional neural network. Thus identified defects are made available to the maintenance personnel in the form of an interactive "defect map". This enables the maintenance personnel to estimate the necessity and scope of maintenance and to plan this maintenance depending on the position and type of defects.
For the localization of the identified defects on the aircraft surface, the image data are referenced synchronously at the time of acquisition with position data from the sensor fusion. The virtual projection of the defect position on the aircraft model surface is based on the position date of the image acquisition and the metrologically determined position of the real aircraft in the spanned coordinate system.
The system, which was developed within two years, was successfully demonstrated on a Boeing 737-500 during project completion thanks to the support of Lufthansa Technik AG in Hamburg. The project was sponsored by the German Federation of Industrial Research Associations "Otto von Guericke" e. V. (AiF) with the support of the German Research Association for Measurement, Control and Systems Engineering e. V. (DFMRS) from Bremen. In addition to DFMRS, WZL and IRT of RWTH Aachen University, APODIUS GmbH, Automated Precision Europe GmbH, Faserinsti-tut Bremen e. V., Five Robots GmbH, Interdisciplinary Imaging & Vision Institute Aachen e. V., Lufthansa Technik AG, Nikon Metrology GmbH, SCISYS Deutschland GmbH and SPECTAIR Group GmbH were represented on the project committee.