Systems integration for machine vision solutions – Driving application success with current and future technologies

✕

A tried-and-true axiom regarding “systems integration” in industrial automation says that for any project or application it is the point at which “someone has to ‘make it work.’” One might reasonably expand on that by adding that the application also has to “work” correctly and reliably. Depending on your discipline, industry, and the size or scope of the application, systems integration may be approached in a wide variety of ways. However, no matter what the project is, or whether you are working on a system for internal use or providing integration as a service, success comes from thorough planning and competent execution. Integration for machine vision and support for related technologies admittedly sometimes may be a more constrained task or subset of a larger project. Nonetheless, in the rapidly changing and expanding landscape of imaging hardware components and software solutions, the job of systems integration is as important as ever regardless of project scope. Here are some of the key elements of integration in this context and how they help ensure success now and in the future, when utilizing both mature and cutting-edge vision technologies.

The starting point - it’s not about technology

A common critical mistake is to design a solution around a technology instead of designing based on the needs of the application. The problem with this approach is that the solution becomes constrained by that prejudgment, and the final system may either not be able to support the application requirements or may end up being more costly or take more time than necessary. A typical scenario is that your company or end-user customer presents an application and states that the solution must utilize AI, or a smart camera, or a cobot, or some other specific technology. Or, as the integrator you immediately start considering certain components before digging deeply into the needs and nuances of the project, or perhaps one relies on a vendor recommendation after only a cursory explanation of the proposed system. Where then does one start?

The most effective way to approach any application in machine vision is to gain a complete understanding of the entire project and the requirements that will make the solution valuable to the overall process. Only then can one confidently identify the correct solutions architecture and the components that absolutely will deliver the desired results. This initial step sometimes is called an application analysis which results in a system specification. Analyze the needs of the existing operation, process, or automation and gather all the details relevant to the targeted application.  Informally, application analysis can be said to be an exercise in asking a lot of questions and knowing exactly which questions to ask.  Perform the analysis without consideration of any machine vision technology to avoid influencing the analysis causing you to miss critical information that could ultimately lead to a better system design.   Define the targeted parts thoroughly and identify variations and special characteristics.  Examine the entire production process and commercial considerations. Then develop a specification and select the components which best serve the project.

Technology considerations – some things never change

Not to minimize the potential value of new and emerging technologies, it is important to understand that with mature and proven components, machine vision has long provided a core set of capabilities that benefit industrial automation in key use categories. Generally described, these application categories are inspection, measurement, guidance/location, and identification.  

General inspection is perhaps the most widely implemented application in machine vision.  Typical implementation cases include the detection of defects and verification of assembly. Measurement with machine vision is the visual gauging of features.  While more involved than general inspection, measurement applications can deliver high precision in-process metrology.  Location and related guidance applications use imaging that is calibrated and referenced to world coordinates; importantly used in vision guided robotics (VGR).  Identification describes machine vision use cases in code symbology or character reading, and applications which require differentiation of objects or features.

No matter what the use cases, the driver behind successful implementations has always been the acquisition of a repeatable and reliable image of the part or features being examined. Appropriate imaging of the key objects or features contributes more than 85% to the success of the application.  Proper imager selection along with knowledgeable and creative design of optics and illumination is a fundamental prerequisite for integrating machine vision systems in most cases.  The goal in the design of imaging for any machine vision task is to 1) create contrast between the features that are important to the inspection and the background, and 2) eliminate or overcome the effects of ambient light from other undesirable sources.   If image contrast is good and consistent over expected part or process variations, the resulting image processing and analysis will produce more accurate overall results and will be much more reliable. Despite occasional product or technology claims, if a feature or defect is too small, too blurry, or is indistinguishable from the surrounding background or confusing features, then no processing technology or software approach will be reliable in detecting it. The good news is that some newer technologies show capabilities to either improve imaging capabilities or overcome some of the subjectivity in image analysis. Seek these out when standard imaging approaches can’t be used but verify capability prior to specification.

Technology considerations – current or cutting-edge

The appeal of working with and implementing cutting-edge technologies can be seductively compelling; particularly given the sometimes-disproportionate attention and hype provided for the most recent and unique components or software. A key “trap” in machine vision integration is to try the latest components only to learn they were not well-indicated for the task. To be clear: it is very important to keep up with, learn, and even evaluate the latest developments within the vision marketplace, but in doing so, be able to identify what applications these new offerings are most well suited for then specify them in those projects.

Improving on the “status-quo”

Observably, most machine vision applications in any of the use case categories still are being successfully implemented using 2D imaging, off the shelf lighting and optics, and standard software or smart cameras. The very good news though is that these more mature components continue to evolve to provide better performance, higher quality and reliability, and greatly expanded capabilities.

Cameras continue to make use of the latest sensors to deliver higher image resolutions and greater frame rates. Camera interfaces have been evolving also, and driven by advances in general computer technologies, 10-, 25-, 50- and even 100GigE connectivity are delivering up to hundreds of frames per second, even at ultra-high pixel counts. Other interfaces like CameraLInk and CoaXpress continue to thrive as well for demanding imaging designs. Smart cameras are following the same growth model in terms of ongoing enhancements. A notable inclusion is the addition of deep learning capability in various forms becoming available in many smart cameras, either as an optional tool or as the dedicated processing mode of the device.

Some other “standard” technologies experiencing rapid advancement worth noting include:

3D Imaging:  components that capture 3D information for a field of view have become a standard in the market.  Performance and range of capability continues to grow for a variety of applications including high-accuracy surface analysis, 3D part and feature measurement and inspection using 3D image data instead of the usual 2D grayscale information, and VGR applications.

Hyperspectral and Multispectral Analysis: a very mature technology but often totally misunderstood in the machine vision realm, these cameras can do discrete spectral analysis of materials or even chemical composition online.  Some vendors are more actively targeting machine vision integrators where a unique need can be fulfilled but be sure to carefully evaluate the suitability of the technology.

Line Scan Imaging: Many industrial applications demand high speed imaging and processing.  Line scanning is a common technique to achieve higher imaging rates with high resolution, but integration of these components can be more complex than common area imaging applications.

Advances in illumination and optics: prominent among these are the availability of higher intensity LEDs with broader, more consistent spectral output, and non-visible wavelengths such as short-wave infra-red (SWIR). Lighting manufacturers are becoming increasingly creative in providing specialty products that enable certain types of processing, i.e. computational imaging and multi-color, multi-directional illumination. Lenses for focused uses are a recent development and facilitate better imaging in some applications.

Choosing cutting-edge

The important decision when moving to any new technology is whether the learning curve of design and implementation warrants the benefit provided over the “tried and true.” From a practical viewpoint though, most new offerings are in the form of useful and innovative advances to existing technologies; the truly novel product introduction is relatively rare.

For those definitively new technologies, choose wisely and test exhaustively. In recent years we’ve seen things ranging from deep learning to neuromorphic (event-based) imaging sensors. In the case of the former, deep learning settled into the most reasonable implementation, providing a useful tool for certain applications in machine vision. The latter, a camera sensor that images changes in a scene has struggled so far to find broad suitable use cases. Bottom line; be informed but diligent in choosing the correct system design as opposed to the most cutting-edge.

Technology considerations – ease of use

Finally, the consideration for ease of use. There is no argument that the system delivered to the end user must be uncomplicated in its execution. Ease of use in machine vision components means being simple to configure. Ease of use in some cases has also come to mean making the system itself perform a task with minimal design, integration, configuration, or programming, and these two uses of the term do overlap sometimes.

Simple machine vision systems or components dedicated to a singular task have indeed embraced and succeeded at ease of use. A simple application like code reading might be achieved with minimal configuration of a component.

Machine vision components and software for general purpose use, where multiple tools and algorithms must be available to address a variety of applications, also increasingly offer a configurable environment that reduces or eliminates the need for programming in code. While this “ease of use” is valuable in the integration process, it does not take the place of a competent machine vision engineer who must still understand the application and select the best approach to acquisition and processing of the image to attain the required results. Even the much touted and so-called simplicity of deep learning tools requires skill and knowledge and detailed execution for successful implementation.

However, the other spectrum of ease of use has enjoyed recent advances as seen in the availability of complete, standalone solutions for well-targeted tasks. A prominent example is that of robot guidance using machine vision, particularly in applications for random bin picking or guided palletizing and de-palletizing.

In conclusion, fundamentally “systems integration” is the challenging process of bringing together diverse and disparate components and sub-systems and making them function as a single unified system.  Machine vision integration is most successful when approached as a very broad engineering project task, following guidelines for execution and design that make the best use of the available technologies.