Fraunhofer Institute for Production Systems and Design Technology
The Best of Both Worlds
The AI detects welding process characteristics such as the number of spatters, the diameter of the arc and the length of the weld pool. This enables in-process control to be performed during production.
Human intelligence vs. artificial intelligence – they both have their strengths and weaknesses. Take, for example, optical inspection and quality assurance in manufacturing: Humans find it easier to understand correlations, such as those between quality characteristics of a product and various process parameters in manufacturing. Artificial intelligence, on the other hand, especially deep learning, has significant advantages in terms of scalability and reliability in the field of industrial image and signal processing. However, it has not yet been widely deployed in industry because training AI requires large labeled data sets, and these are often expensive and difficult to obtain. In addition, small and medium-sized companies often lack the necessary in-house AI expertise.
The situation at hand calls for a hybrid approach that combines the advantages of humans and AI: an intelligent assistance system that supports workers in processes such as product ramp-up, calibrations, process adjustments or reworking, always suggesting the most resource-efficient path. Which, in turn, learns from the experiential knowledge and decisions of the human actors along the value creation processes. Over time, it becomes able to independently recognize new product characteristics and link and evaluate these with process characteristics in order to control the production facilities more resourceefficiently.
A team of scientists at IWF TU Berlin is getting closer to this product of wishful thinking. They are developing hybrid systems that combine the efficiency of automated plants with the flexibility of humans. This hybrid intelligence allows human and artificial intelligence to play together in such a way that they can learn from each other and thus achieve better results. Together, they can handle complex manufacturing tasks such as quality assurance. The approach is adaptable and versatile, and is therefore particularly useful for small and medium-sized companies, for which low volumes and frequent changes in production processes have not previously allowed full automation. The project is funded by the BMBF initiative »Learning Production Technology – Use of Artificial Intelligence (AI) in Production (ProLern)« as part of the German government's hightech strategy.
The scientists are looking at the scalability of hybrid intelligence in the form of AI cloud services to assist in-process control and adaptive control of production equipment. To train the AI, users of the assistance system should be able to provide feedback about how their decisions on various process parameters affect product quality via simple »plug-and-play«.
»As a result, we will hopefully soon be able to provide companies with access to hybrid intelligence that recognizes and characterizes workpieces while they are still being manufactured. The AI will then suggest measures to compensate for quality deviations and adapt processes to new product characteristics – all that in the most resource- efficient way, based on a comparably low initial investment and hardly any in-house AI expertise,« says Dr.-Ing. Soner Emec, who is leading the project at IWF. By moving the computational operations to cloud servers operated by project partner PSI, companies do not need to have highperformance computers on site to train the AI models and use the AI assistance services.
The highlight and at the same time the challenge of the research project is to teach machines to arrive at good decisions even with a poor data basis, just like humans can. In order to achieve this, the researchers must further develop the human-AI interaction in such a way that, on the one hand, mutual understanding arises in the context of manufacturing. On the other hand, in the medium-term the AI must perform »self-learning correlation« between new quality characteristics of products and process characteristics. In other words, the AI must be able to establish correlations between causes and effects for a wide range of parameters.
Initial project results are promising: For example, the developed AI is already able to infer process quality and stability via image-based determination of the number of spatters and weld pool geometry during arc buildup welding, and to suggest possible reasons in the event of instability.
On the other hand, by directly integrating user feedback, the AI also learns to interpret new process signals and features like a human. This is because the assistance system contains software tools that not only facilitate data manipulation and data stream management for operators, but also serve to actively train the AI. For example, they assist with labeling and augmenting data and transparently display the detected features – such as surface texture, spatter count or weld pool area – via processspecific metrics. These key figures in turn support monitoring of the process and workpiece.
The image, process and sensor data streams as well as feedback from users are analyzed during production via the »AI cognitionsupporting assistance system« (KI-kognitionsunterstützendes Assistenzsystem, KIKA) module, and the results are comprehensibly transmitted in real time to the users in the plant and to the machine control software. A special feature of the IWF approach is that the developed KIKA module with its assistance tools can be integrated into the data stream management environment Kafka, which enables real-time monitoring of several thousand sensor data streams as well as video data sequences. To facilitate operation and achieve further scaling, the team is collaborating with the Berlin Institute for the Foundations of Learning and Data (BIFOLD) and the German Research Center for Artificial Intelligence (Deutsches Forschungszentrum für Künstliche Intelligenz, DKFI). Together, the researchers are integrating the AI methods and required frameworks into the data stream management environment »NebulaStream«, which is being developed by BIFOLD.
Examples:
3D Metal Printing for Large-Volume Components
The challenge: When printing largevolume metallic components, the process parameters must be controlled adaptively in order to be able to guarantee the required process stability during production and the resulting component quality, even under the influence of unknown disturbance variables. Each of the numerous materials that can be processed, such as mild steel, CrNi steel, tool steel, aluminum or titanium, requires different buildup strategies and process parameters that are also geometry-dependent. For example, buildup strategies for thin-walled structures are usually not transferable to thick-walled structures. Unknown disturbance variables include, for example, varying ambient conditions (humidity, ambient temperature, etc.) and fluctuating wire quality (diameter variance, residues of drawing greases, etc.). These can lead to disproportionately strong slag formation and potential slag inclusions in the component core, but also to bonding defects, porosity, ignition defects or other defect patterns that can only be detected by inline video sequence analysis during production.
The approach: The KIKA module supports workers in making reliable statements about component quality and necessary decisions about process configuration, even under the influence of unknown disturbance variables.
The vision: A completely self-learning and autonomous process control based on a plug-and-play approach. There is no longer a need for trained technical staff. The system simply receives the CAD file and the desired material as well as necessary quality criteria to be met as input, and selects the build strategy completely autonomously. It performs slicing as well as path optimization for particularly complex parts of the component, and produces the component under continuous control. Irregularities can be detected, classified and evaluated independently by the KIKA module, which finally leads to the output of a quality protocol of the manufactured component under evaluation of the required quality criteria. The system setup enables continuous improvements in terms of process stability and component quality and independently learns correlations between setting parameters and required output parameters. Thus, with each component production, the system continues to improve itself until ultimately 100 percent defect-free components are produced.
Drug Printing for Personalized Medicine
The challenge: Using additive manufacturing, it is possible to supply patients with the drugs they require flexibly, locally and individually starting with a batch size of one. However, the challenge lies in patient-specific dosing the various active ingredients with different properties in nanoliters. This requires reproducible and highly precise application of the active ingredients by the technology used. To check the manufacturing process, offline samples are currently taken and analyzed for their active ingredient content in a cost-intensive wet-chemical process. However, wet-chemical analysis for the end customers with small batch sizes is not profitable. To overcome this obstacle to pharmacological production, a 100 percent process control must be ensured.
The approach: The KIKA module is to track each drop, determine the physical properties during the drop flight and adaptively control the print head nozzle. The image data will be used to derive the physical properties such as volume, velocity, trajectory, shape, bubble formation, center of gravity, and roundness of the droplets. Furthermore, it is determined whether only a single droplet was ejected or whether so-called satellite formation occurred. In addition, geometry is recorded at the meniscus, which represents the fluid curvature at the nozzle.
The vision: Based on the image data, the KIKA module adaptively calibrates the process parameters and regulates them during production. This would optimize the process flow under different conditions and with different active ingredients, while increasing process stability and maintaining complete process control.
Thereby, a system is created that can infer process characteristics from visual product features, which are hardly measurable. As a result, the process can be controlled either automatically or manually when operating the machine to make sure that quality deviations are compensated for during production and new product characteristics are efficiently adapted. In this way, KIKA enables machine operators without AI expertise to easily configure AI applications for feature recognition from sensor data streams and image sequences.
The developed system is currently being tested under real industrial conditions in two additive manufacturing scenarios to prove its resource efficiency potential: – Printing of large-volume metallic components by arc buildup welding (Wire Arc Additive Manufacturing, WAAM) at the project partner GEFERTEC GmbH – Personalized medicines by so-called »Drop-on-Demand (DoD)« processes at the project partner DiHeSys GmbH.
The use cases were chosen from the field of additive manufacturing, because the value creation here is shifting more and more towards the end customer due to cheaper and more efficient 3D printing technology. In the near future, the KIKA module could be used on site by assemblers to manufacture components in the workshop or by pharmacists to produce medicines in the pharmacy. However, they often lack the many years of process- and material-specific expertise needed to set up the configurations required to achieve the highest possible product quality. This expertise may soon be substituted by machine intelligence. Using the hybrid approach of IWF TU Berlin, equipment manufacturers can adapt the machine capabilities of the equipment and the operator interfaces to enable fully automated calibration and printing through a simple understanding between machines and humans.