Industrial Reinforcement Learning for the Quality Control of Bulk Metal Forming Processes




Tobias Müller



+49 241 80 28211



New Research Project "IRLeQuM" for Process Optimization and Quality Improvement in Bulk Metal Forming Processes starts at WZL

  Copyright: © Mubea Tailor Rolled Blanks GmbH Visualization of a flexible rolling process

Instabilities due to external influencing variables, unknown interdependencies between process parameters or product quality characteristics often lead to scrap in bulk metal forming processes despite existing process controls. Current control concepts are based on implicit operator knowledge and automated control of individual process parameters. However, new holistic approaches are being investigated to further objectify control and reduce deviations. Quality control loops are a means of overarching compensation for quality deviations. In combination with machine learning approaches, in this case reinforcement learning and transfer learning, they offer the potential to reduce scrap. This is achieved by automatically adapting the plant parameters when instabilities occur.

The aim of the research project "IRLeQuM" is therefore to develop a method based on reinforcement and transfer learning for the implementation of novel controllers in quality control loops of bulk metal forming processes.

To enable reinforcement learning-based control, the necessary IT infrastructure is first defined and implemented. Such a control system offers the advantages that, on the one hand, all quality-relevant information, such as process parameters, environmental conditions or raw material properties, can be included in the control system. On the other hand, the implicit operator knowledge of the control can be made permanently usable.


In order to reduce the learning time of the reinforcement learning algorithm and to save resources, it is not trained directly on the real process, but on a stochastic process simulation. The knowledge gained from the simulation is then transferred to the quality control loop of the control loop by means of transfer learning. The result of the research project will be a quality control of bulk metal forming processes, which controls processes automatically, comprehensively and in real time and optimizes the quality of the processes. The improved process quality will in turn increase the quality of the products and reduce scrap.

The research project "IRLeQuM", with a project duration of three years, started on June 1, 2021 and is being carried out in collaboration with the Chair of Production Metrology and Quality Management, the Chair of Manufacturing Technology (both from the Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University) as well as the companies Mubea Tailor Rolled Blanks GmbH (consortium leader), Eichsfelder Schraubenwerke GmbH, IconPro GmbH, Schomäcker Federnwerk GmbH, Quality Automation GmbH and the associated partners MAWI GmbH and Schuler Pressen GmbH.

Funding Notice:
The research and development project "IRLeQuM" is funded by the Federal Ministry of Education and Research (BMBF) in the program "Future of Value Creation - Research on Production, Services and Work" (funding code 02P20A073) and is supervised by the Project Management Agency Karlsruhe (PTKA). The responsibility for the content of this publication lies with the authors.