Research Project AMIIGO - Interim Results




work Phone
+49 241 80 25830


Surface inspection of aircrafts using automated multi-lens systems and neural networks to generate defect maps

  Research team is holding model of a multicopter Copyright: © WZL The research team (f. l.): A. Kluge-Wilkes (WZL), R. Pugliese (IRT), M. Sorg (DFMRS), A. Buckhorst (WZL), T. Konrad (IRT), Dr. S. Recher (SCISYS Deutschland GmbH) and Dr.-Ing. R. Zweigel (IRT)

After the kick-off in October 2017, interim results were presented in the research project "Automatic, multi-copter-based indoor inspection of large surfaces" - AMIIGO for short. The aim of the project is to reduce the effort required for non-destructive inspections of large surfaces by using automated multi-copter systems. For the defined application - lightning strike inspection on aircraft (e.g. Airbus A380) - the systems move camera and measurement technology over the aircraft surface. Pictures of the object surface are taken while the position information is stored during shooting. The recorded data sets are searched algorithmically for defects caused by lightning strikes. Defects found are visually processed in so-called defect maps on 3D models of the aircraft.

The status of the project was discussed at the committee meeting accompanying the project at the end of May. The results refer to the necessary path and trajectory planning for the automatic flight of the multi-copter systems as well as the algorithmic analysis of the generated data material. In addition, the further methodical procedure for implementing the overall systems was defined.

For trajectory planning, the first step is to abstract the envelope to be flown on the basis of a 3D model of the aircraft to be inspected. This envelope is located at a constant distance of 2m from the object surface. Negligible details are generalized as flat surfaces. The enveloping surface is then divided into equidistant support points, which represent target positions for the multi-copter system. In order to achieve an energy-efficient flight, the shortest path containing all support points is calculated. The dynamic properties of the system, such as accelerations and speeds, are supplemented in a next step, resulting in a trajectory.

The identification of lightning damage in the collected image and position data is realized by a hybrid approach developed at the WZL, consisting of a classical corner detection algorithm in combination with a neural network. The neural network is a so-called Convolutional Neural Network (CNN), which is primarily suitable for the analysis of image and audio data. To train the net, existing and manually evaluated pictures of typical fault patterns as a result of lightning strike were used. If damage is identified by the hybrid method, the pixel position of the error is then calculated. From known boundary conditions, such as distances between camera and aircraft surface at the time of image acquisition and the measured position of the multi-copter system, the actual location information of the identified damage to the aircraft can be determined via suitable coordinate transformations. All location information is processed and then visualized for the planning process on the basis of the 3D model used at the beginning.

The project is supported by the Federation of Industrial Research Associations "Otto von Guericke" e. V. (AiF) and organized by the German Research Association for Measurement, Control and Automation and System Technology e.V. (DFMRS). Within two years, the Institute for Automatic Control IRT and the Chair of Production Metrology and Quality Management at the Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University will work hand in hand to develop an autonomous system for defect detection on aircraft. IRT will focus its research mainly on multi-copter automation, while the Laboratory for Machine Tools and Production Engineering will be responsible for measurement technology, project management and the networking and communication of interfaces.

In addition to the DFMRS, the Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University and the Institute for Automatic Control of RWTH Aachen University, the project committee also includes APODIUS GmbH, Automated Precision Europe GmbH, Faserinstitut Bremen e.V., Five Robots GmbH, Interdisciplinary Imaging & Vision Insitute Aachen e.V., Lufthansa Technik AG, Nikon GmbH, SCISYS Deutschland GmbH and SPECTAIR Group GmbH.