Machine vision combined with artificial intelligence makes it possible: the technology automatically inspects welded connections in body shells and identifies anomalies. The Spanish automation specialist DGH has developed such an application for the automotive industry.

Machine vision combined with artificial intelligence makes it possible: the technology automatically inspects welded connections in body shells and identifies anomalies. The Spanish automation specialist DGH has developed such an application for the automotive industry. It improves the consistency, speed, reliability, and accuracy of the entire inspection process - and does so completely autonomously. The machine vision solution was developed by DGH using the software products MVTec HALCON and the Deep Learning Tool from the Munich-based company MVTec.

High quality standards are required in automotive production. This naturally also applies to welding processes on the body-in-white (BIW). The importance of body stability is self-explanatory. More exciting is the question of how the high quality of welded connections can be ensured - automatically and seamlessly. The challenge is that many different defects can occur which impair the quality of the bodywork. For example, cracks, incomplete weld seams and irregular welding patterns must be precisely identified. DGH tackled exactly this challenge. The Spanish company, which has its main headquarter in Valladolid and recently has been integrated in GROUPE ADF, supports a wide range of industry segments with innovative solutions for process automation. The result is an inspection system that automatically captures images of welded joints. These are then immediately checked by the MVTec HALCON AI algorithms and DGH machine vision software. It sends the results - OK or NOK - to the PLC. This controls how to proceed with the bodywork accordingly. MVTec HALCON is the standard software for machine vision from MVTec. The Munich-based family business has been developing hardware-independent machine vision software for industrial applications since it was founded in 1996 and is one of the technology leaders in this field - partly because the company offers various powerful deep learning algorithms.

Deep learning in production: optical inspection of welded connections
Deep learning is a type of artificial intelligence. In machine vision, deep learning enables the implementation of more and more applications, including those that were previously not possible. In addition, the performance of existing applications can be significantly improved. DGH has also taken advantage of these developments. On behalf of a large French automotive group, DGH's team of experts developed an automated system for inspecting welded connections between metal parts for inert gas welding (MIG welding) processes. "Previously, the inspection was always carried out by long-serving employees. This is because it is not always easy to recognize whether the quality of the welded connection from the different processes is OK. When implementing the new system, we therefore incorporated the experience of such employees. We have trained the underlying deep learning networks with their knowledge. The required robust recognition rates are only possible using deep learning," explains Guillermo Martín, Innovation & Technology Director at DGH. The primary aim of the implementation was to achieve a very high-quality standard for all weld seams. In addition, the new, autonomous quality inspection was intended to bring the fundamental advantages of automation to bear. Namely, greater speed, reliability, accuracy and clear consistency in decision-making - in contrast to the subjectivity of human decision-making.

DGH achieves clean process integration of machine vision
Implementing such a system involved several challenges. "It was clear to us that we had to implement the system based on machine vision. Sensors or classic 2D/3D vision systems fail due to the complexity of the weld seams. The first challenge was therefore to develop a viable solution and reliably detect the different types of defects. Additionally, the second challenge involved transferring the expertise of experienced employees into the deep learning application. Finally, the third challenge was to carry out the inspection processes in a short time. The reason for this is the tight cycle times," explains Martín. The system that has now been implemented at the French car manufacturer works as follows: When a body arrives at the inspection station, the PLC triggers various inspection processes. When the station receives a trigger, the attached 2D cameras take photos of the welded connections individually or one after the other and transmit them via GigE Vision protocol to the machine vision software, where they are processed. The system checks whether anomalies can be detected around the weld seams. It can reliably inspect different weld joints, seams, and spots created during various welding processes. The data is then sent to the PLC and the corresponding results are visualized on a screen. The inspection application was developed by DGH on an industrial PC. The system continuously monitors communication with the production plant's PLC and several 2D cameras. The MVTec HALCON machine vision software forms the heart of the setup.

Deep learning methods from MVTec
Two deep learning methods are used from the machine vision software to reliably detect the defects. Firstly, "instance segmentation" is used to localize the relevant area, i.e. the weld seam, on the images captured. This deep learning technology is able to assign a class to different, trained objects with pixel precision. In the next step, "anomaly detection" is used. Deep learning-based anomaly detection enables automated surface inspection and accurately detects deviations, i.e. defects, of any kind. "Anomaly detection had two decisive advantages for us: On the one hand, the detection rates are very high and robust. On the other hand, training the underlying neural networks was simple. This is because mainly "good images" of the welded joints, i.e., images of weld seams without defects, were required to train the deep learning networks. The Anomaly Detection Network is trained using only good images. Availability of "defect images" is not a requirement for Anomaly Detection. However, few defect images can help find an optimal threshold value to differentiate between good and defect weld seams. This threshold value is applied on the anomaly score, which is the output of the Anomaly Detection network. Determination of the threshold is not a part of the training. We therefore only needed a small number of images. This is very practical, as good images are available quickly and easily. Large number of images showing defects are much more difficult to organize, not to mention the fact that it is impossible to obtain images of all possible defects. This is where deep learning has a clear advantage," explains Guillermo Martín. In images of weld seams that differ from the trained images, the anomalies or defects are reliably detected. The size of the delta between OK and NOK is determined by the so-called threshold value. The threshold value is a parameter within deep learning methods that regulates the value up to which the image to be checked may deviate from the trained "good image". The user can freely set this parameter and therefore can bring transparency into the "black box" of artificial intelligence decision-making.

Important preparation for deep learning: labeling the images and training the neural networks
Deep learning technology requires the neural networks to be trained with images before operation. These images must first be labeled for training. DGH used MVTec's Deep Learning Tool for this preliminary work. With this free of cost tool, image data can be easily labeled and then comfortably trained. To do this, DGH first collected images of weld seams. The knowledge of the employees was also used here. They checked each image to ensure that mainly "good images" were used for training. An incorrectly used "bad image" would falsify the results of the training. The "good images" are then loaded into the Deep Learning Tool and labeled there specifically for the instance segmentation technology. The "Smart Label Tool" is available for this purpose. All the user has to do is select the welded joint area with a mouse click, and the "Smart Label Tool" automatically labels the welded joint. This ensures that the Deep Learning Tool then only trains on the basis of the relevant areas of the image. The car manufacturer's employees were also involved in this step. They knew which areas within the image contained important information about the welded joint and how large the corresponding frame around the welded joint should be. After the images are labeled, a split is created. This involves splitting the image data set, usually in a ratio of 50 percent for training, 25 percent for validation, and another 25 percent for testing. Training, validation, and testing are carried out easily and conveniently in the Deep Learning Tool at the touch of a button. The trained model is then saved and loaded in HALCON for inference. This is possible due to the seamless compatibility between the Deep Learning Tool and HALCON. The software is now ready for operation.

Machine vision software as a core component of the inspection system
"We at DGH have been working with MVTec for over ten years and are therefore familiar with their powerful tools and algorithms. That's why we decided to trust MVTec HALCON for this project as well," reveals Guillermo Martín. The challenges in terms of training and speed due to the tight cycle times have already been mentioned. There was also another requirement for the machine vision software: the environment is difficult due to reflective metal surfaces and the different lighting conditions.

DGH was able to overcome all challenges and deliver a system with the desired high quality. "In the beginning of 2024, the first system was put into operation at the car manufacturer's plant. Once it ran successfully, we received a new request from the same manufacturer in April 2024 to implement a second system for inspecting welded joints," says a delighted Guillermo Martín. In particular, the goals of reducing the dependence on skilled workers for quality inspection processes in times of a labor shortage and subsequently increasing the degree of automation were achieved. Automation based on machine vision and artificial intelligence has significantly reduced errors and ensured consistent and reliable detection of welding defects. For this reason, Guillermo Martín also believes that despite the large number of machine vision systems already in use in various industries and manufacturing processes, there is still potential for growth - for example for deep learning solutions in particularly demanding and complex applications.

About MVTec Software GmbH
MVTec is a leading manufacturer of standard software for machine vision. MVTec products are used in a wide range of industries, such as semiconductor and electronics manufacturing, battery production, agriculture and food, as well as logistics. They enable applications like surface inspection, optical quality control, robot guidance, identification, measurement, classification, and more. By providing modern technologies such as 3D vision, deep learning, and embedded vision, software by MVTec also enables new automation solutions for the Industrial Internet of Things aka Industry 4.0. With locations in Germany, the USA, France, Benelux, China, Taiwan, and South Korea as well as an established network of international distributors, MVTec is represented in more than 35 countries worldwide. www.mvtec.com