Machine vision technologies based on CNNs and deep learning can be used profitably in a wide range of different industrial sectors and applications, allowing the automation and significant acceleration of inspection processes in the electronics industry, for example.
Machine Vision Technology Update
Johannes Hiltner | MVTec Software GmbH
Tell us a little bit about MVTec and the products and services you provide.
MVTec is the developer and vendor of the general purpose machine vision software products HALCON and MERLIC. The company was founded in November 1996 as a spin-off of the Technical University of Munich and the Bavarian Research Center for Knowledge-Based Systems (FORWISS). Today, it is a leading international manufacturer of software for machine vision used in all demanding areas of imaging like the semi-conductor industry, inspection, optical quality control, metrology, medicine, or surveillance. In particular, software by MVTec enables new automation solutions in settings of the Industrial Internet of Things.
MVTec HALCON is the comprehensive standard software for machine vision with an integrated development environment (HDevelop) that is used worldwide. HALCON is optimized for the needs of OEMs and system integrators and allows engineers to set up their own solutions for a specific machine vision task. It enables cost savings and improved time to market. HALCON’s flexible architecture facilitates rapid development of any kind of machine vision application. The software provides outstanding performance and a comprehensive support of multi-core platforms, special instruction sets like AVX2 and NEON, as well as GPU acceleration. It serves all industries, with a library used in hundreds of thousands of installations in all areas of imaging like blob analysis, morphology, matching, measuring, and identification. The software provides the latest state-of-the-art machine vision technologies, such as comprehensive 3D vision and deep learning algorithms.
MERLIC is a powerful, all-in-one machine vision software product that enables users to quickly build and integrate complete solutions. Not a single line of code needs to be written whilst working with MERLIC. The individual tools are named after the tasks concerned, describing these in a language which the user can easily understand: if the user wants to measure something, he clicks on “Measure”; if he wants to count the number of objects in an image, he needs to click on “Count”. MERLIC also grants easy access to all the elements of the machine vision periphery and offers seamless PLC connectivity – enabling ideal integration into the production environment.
Furthermore, MVTec builds customized software solutions for machine vision – from consultancy, studies, and prototypes up to integrated products. Software solutions can be based on standard PC or embedded hardware (e.g., Arm®-based systems). MVTec has also an excellent background in processing images of different kinds, including 3D, infrared, hyperspectral, and X-ray images.
Machine & Deep learning technologies are real trendy topics in the machine vision market. Could you explain why?
Digitalization is having a revolutionary impact on industrial production, with automation becoming ever more widespread as part of the Industrial Internet of Things (IIoT, also known as Industry 4.0). A wide range of machines and robots is performing more and more tasks in every-day production. On assembly lines, new compact and mobile robots, such as collaborative robots (cobots), often work side by side with their human colleagues. Improving the safety and efficiency of these processes necessitates additional, state-of-the-art technologies. Machine vision – whose techniques and processes have become crucial in automating and expediting production processes – plays a central role here, allowing diverse objects to be identified, allocated and reliably handled along the entire value chain.
.jpg)
To further improve the identification process and tune it to the needs of flexible, networked IIoT processes, machine vision is beginning to incorporate methods employed in artificial intelligence (AI). The catchphrases here are machine learning as part of AI, deep learning as the most relevant method within machine learning, and convolutional neural networks (CNNs) as the most popular deep-learning architectures. All these technologies have a key facet in common: their ability to comprehensively analyze and evaluate huge volumes of data (Big Data) for training many different classes; the upshot of this is far greater efficiency in distinguishing between different objects. Increasingly, these data are being generated within the IIoT in the form of digital image information or, alternatively, as data from scanners, sensors, and other process components.
Which industries will benefit from these technologies the most? What are use-cases that you anticipate?
Machine vision technologies based on CNNs and deep learning can be used profitably in a wide range of different industrial sectors and applications, allowing the automation and significant acceleration of inspection processes in the electronics industry, for example. The efficiency of detecting any conceivable product defect can thus be greatly improved using self-learning methods – as described above. Even the minutest of scratches or cracks in circuit boards, semiconductors, and other components can be reliably identified, enabling automated removal of the relevant parts. The food and beverage industry also benefits from deep-learning technologies. Sub-standard fruits and vegetables, for example, can be identified more accurately before they are packaged or further processed. The processes are also being used in automotive engineering – an industry renowned for its very high level of automation; in this sector, self-learning algorithms are used to perfectly identify minuscule paint defects invisible to the naked eye. Another important application area is pharmaceuticals, where pills which may have very similar external characteristics contain very dissimilar active substances. Deep learning and CNNs let the drugs be identified, inspected, and distinguished from one another with a very high degree of accuracy, thereby ensuring they are matched with the correct blister packs.
What challenges do companies face when they plan to implement deep learning technologies?
Due to their complexity, AI-based technologies such as deep learning require highly-qualified development staff. A large number of sample images is generally required for the training process before an object can be recognized. Some classes of objects may need as many as 100,000 comparison images to achieve acceptable recognition rates. Even when the necessary sample data are available, the training process is extremely time-consuming. The programming work needed to identify different defect classes during fault inspection is usually also extremely complex. All of this necessitates a highly skilled and suitably trained workforce, and most companies simply don’t have such resources at their disposal. As already mentioned, however, that was yesterday’s status quo: our new software release provides a remedy.
How can modern software solutions support the integration of deep-learning technologies?
Today’s state-of-the-art vision solutions – which also already include numerous deep learning functions – can help. With the most-recent version 17.12 of the off-the-shelf software solution MVTec HALCON, companies can train convolutional neural networks (CNN) themselves, without having to invest heavily in time and money. The reason for this is the software’s underlying strategy: it comes with two networks that are optimally pre-trained for industrial use – one, optimized for speed, the other, for maximum recognition rates. Consequently, the training process only needs a few sample images that are supplied by the customer and thus tailored to the customers' exact applications; the result is a neural network that can be precisely adapted to the customer’s unique demands. With an easy and systematic means of classifying new image data, corporate users – which generally lack in-depth AI expertise – can now dramatically cut the programming work involved, saving large amounts of time and money. Companies’ existing machine vision personnel can train the network without any hitches arising along the line.
In which way do companies benefit from these solutions?
The time-saving potential becomes clear with the use case of defect recognition. The appearance of defects, such as minute scratches on electronic components, can never be described accurately beforehand. As a result, manually developing an algorithm to detect every conceivable fault based on sample images would be highly complex. Experts would have to view hundreds of thousands of images by hand and then use their observations to develop an algorithm describing the defect as precisely as possible. This would simply be too time-consuming.
CNNs and deep learning technologies, by contrast, can autonomously learn specific defect characteristics, and accurately define the associated problem classes. Using these techniques, the technology only needs about 300 to 500 sample images per class to train, verify, and thereby precisely detect the various defect types. The process takes no more than a few hours. It not only minimizes the amount of time needed, but also significantly improves the recognition rates compared to manually programmed defect classes. Self-learning algorithms thus help slash false negative rates, while at the same time eliminating the inefficiently high error rates associated with manual programming.
Is Deep Learning the general cure for all machine vision use cases? If not so, please explain why?
Deep learning won’t revolutionize the machine vision industry – it’s definitely not the Holy Grail that will solve all our problems. It’s simply a classifier. There are so many applications in the machine vision sector that are not based on classification: data code reading, for example, contour-based object localization (shape-based matching), or 3D matching technologies – all areas in which we don’t currently see deep learning as a great benefit. These will continue to be used in more conventional algorithms for the next 10 to 20 years. At present, we regard deep learning as a powerful tool, but with a clear focus on classification problems.
Deep learning-based classification is a technology which needs a huge amount of computing power. Even if we have a very high-performance implementation with GPU support, this can be “way over-the-top” for some applications: when a simple application is involved, such as distinguishing objects based on the form of their region, other conventional techniques are much faster as they are also highly optimized for this task.
Take a look into the future 5 or 10 years, what developments in Machine Vision, Deep Learning are we seeing and how is MVTec involved in this development of the industry?
We’re sure that machine-vision technology will remain just as relevant and fascinating for process automation looking ahead. As a technological leader, MVTec has for a long time focused on hot topics such as embedded vision, the convergence of automation and machine vision (vision integration), as well as new technologies such as deep learning. Just recently, for example, we demonstrated in a technological proof that extensive deep learning functions are also possible using low-cost embedded boards based on the NVIDIA Pascal architecture: to this end, we successfully tested the deep learning inference of HALCON 17.12 on NVIDIA Jetson boards with 64-bit Arm® processors, enabling a significant acceleration of deep-learning processes on an embedded device.
Other areas which we’ll continue to focus on are simplifying industrial image processing and improving our technology’s accessibility to an ever-wider target group. Our MERLIC software – which offers users intuitive, ergonomic tools for all routine vision tasks for a quick, simple and robust compilation and immediate integration of end-to-end machine-vision applications – perfects this approach. This will also have a bearing on the further developments of our deep learning-based functions.
Our main aims are to eliminate complexity and the need for expert knowledge, and to provide a higher abstraction level for deep learning. Over the next few years, MVTec will continue to work on deep learning and provide high-level interfaces. Our hope is to improve the technology to such an extent that our users will no longer need hundreds, but only tens of sample images per class.
Deep learning is still a young technology; although it is complex, it offers significant investment opportunities. That is our mission for the next few years – a mission to make this complex technology accessible to a wider spectrum of users.
The content & opinions in this article are the author’s and do not necessarily represent the views of RoboticsTomorrow
Comments (0)
This post does not have any comments. Be the first to leave a comment below.
Featured Product
