1 December 2006

Vision in hostile environments

With computer processing a given, the key issue is sensor survival.

By Walt Pastorius and Nicholas Sheble

There are numerous process monitoring jobs for vision sensors in hostile manufacturing environments, such as arc and spot welding, molten metal pouring, wood processing, and electronics.

When vision sensors lie right in the manufacturing process, rather than after the fact, instant data to detect process change becomes available for closed loop control and instant response.

Each in-process monitoring environment has its own specific needs regarding sensor performance specifications, environmental factors, operator interface requirements, and different industrial practices.

Specially designed sensor environmental protection for hostile in-process applications is a major issue. Considerable development effort on packaging vision sensors to survive harsh environments has transpired over the years.

Specific environments encountered include high temperature-both of the sensor environments and the surface monitored, including molten metal, high electromagnetic fields, and metal splatter found near weld guns, dirty environments, and saw mill applications operating in high ambient light levels.

The validation of sensor performance in hostile environments is necessary and important, as are results outlining the values users find after implementing vision sensing in processes. Depending on individual applications, these cover a spectrum from quality and/or productivity increases, faster response to process changes, and cost reductions.

We'll look at examples from automotive assembly, foundry, and the wood processing industries.

Dramatic evolution

Machine vision systems for industrial applications first began appearing in the 1970s. Most of the early systems rested on custom designed sensor technology, were relatively complex, and were relatively fragile, largely due to components.

Initial systems were 2-D, with 3-D sensing technology evolving roughly 10 years later. Early implementations in manufacturing provided 100% inspection capability on finished components, monitoring every part produced, taking advantage of the fast, non-contact nature of vision sensing.

While this required sensors to operate in the factory environment, the applications were remote from the more hostile manufacturing process.

Over time, vision systems have dramatically improved in terms of performance, accuracy, robustness, size, and cost effectiveness. Components of vision sensors (cameras, lasers, LED's, and other items) have evolved in to fully, solid-state rugged devices, available with ever-increasing performance and lower cost.

Computational devices, required for image processing algorithms, have similarly dramatically evolved. Much of this evolution has followed developments in the consumer market, such as digital cameras, CD players, and PCs.

As vision sensors have become more robust, vision sensors have become an integral part of the process, with sensor data often directly controlling the process in an adaptive closed loop manner, providing optimization of the process.

Today, these applications involve placing sensors in hostile environments, including molten metal pouring, welding, wood processing, and many others. Sensor packaging has also improved to survive these hostile environments.

Automotive assembly

Monitoring of dimensions of automotive bodies and subassemblies was one of the first applications using large numbers of 3-D vision sensors to come to the factory floor.

Prior to use of vision sensors, dimensional monitoring occurred on a sampling basis, using coordinated measuring machines and offline measuring devices. Vision systems provided ability to monitor 100% of production rather than a small sample, flagging process deviations quickly and finding parts with random deviations.

For many years, these systems were end of line stations, with many sensors mounted on a frame in an idle station. A typical installation like a final inspection station for a body in white-an assembled car body, before adding doors, hood, and deck lid, and prior to painting-might have 80 to 100 sensors.

Final inspection stations usually come with a computer-based controller, which collects and displays data and data-derived statistical information, and communicates with line controls and the plant network.

End-of-line or completed component inspection provides valuable capability to monitor dimensions of assemblies and identify any parts outside of tolerances.

However, such stations do not generally assist the user to identify the cause of variation, such as what station or sub-component in the assembly process created unacceptable conditions.

To address these issues, the next step in development of vision sensors for assembly monitoring provided sensors for in-process monitoring.

Most automotive sheet metal assembly operations involve spot welding, an environment with weld splatter, smoke, high electromagnetic fields induced by welding currents, and sometimes water when cooling hoses for the weld guns rupture.

To meet desires to mount sensors throughout the assembly line to provide in-process monitoring, sensor design and packaging has improved to allow sensors to mount in the assembly weld tools, right next to the weld guns. This allows sensing to spread out to critical locations throughout an assembly line for rapid feedback and root cause analysis of process variations. A typical in-tool 3-D vision sensor sits inside a sealed, watertight housing that prevents damage when water lines rupture.

Sensor electronics make use of electromagnetic protection from fields induced by welding currents. A further feature of the sensor is a removable protective shield on the front of the sensor. The shield protects the sensor's glass window from weld splatter and requires only infrequent changing when pitting eventually occurs.

When sensors mount in active assembly stations, studies of the process become easier. Sensors can operate in a continuous data-gathering mode to obtain data of actual location movements during the process. Since data acquisition is automatic, and results read out in essentially real time, such studies can transpire quickly and easily.

This ability to take and display process information automatically, right in the line, facilitates process development and process improvement studies, and it is particularly valuable when setting up a new tool or assembly line.

The in-tool sensors operate from a cost effective Pentium PC platform, with ability to connect multiple distributed sensor locations to a single PC. A Windows-based operator interface to display data and control system operation, as well as generate statistical reports is part of the rig.

Data can also transmit over a network to plant information systems, statistical analysis systems, or other PCs for custom report generation or other special analysis.

Molten metal pouring

Level control of molten metal during pouring can provide significant benefits in terms of reduced cost, improved throughput and quality, and removing operators from a dangerous environment.

The pouring environment is extremely hot, smoky, and often includes splatter of molten material, a major challenge for any sensor technology. Laser 3-D sensors have met this challenge since the 1980s.

Locating the sensor away from the molten surface solves part of the environmental issue. Temperature, both from ambient conditions as well as radiation from the surface, can not affect the guts of the mechanism as sensor packaging, housing the sensor in an enclosure, with water, and/or air-cooling of the sensor head at work.

Optical issues also are important-to obtain a clean image of the laser spot without "seeing" the energy emitted by the surface. Sensors specifically designed for molten metal measurement combine both optical filtering for the laser wavelength and electronic filtering to obtain a clear image of the laser spot on the molten metal surface. Today, laser line sensors for level control applications are working well in providing more information for better accuracy and reliability.

Implementation of closed loop pouring control requires not only sensing capability but also control software and mechanical devices to control physically the molten metal flow, fully integrated.

Laser based pouring control reduces waste by eliminating over pouring, reduces scrap by preventing short pours, improves quality, and often improves throughput by optimizing the pouring cycle.

Wood products align logs

Processing trees into lumber is a complicated endeavor because every tree has a different shape, making optimization of recovery difficult. Optimization, however, provides best use of limited ecologically valuable natural resources.

Over the last 25 years, vision sensing in wood processing has evolved, in conjunction with advances in computing power as well as material handling, to increased yield, reduce waste, and provide better end-product quality.

Today, 3-D laser sensors monitor the geometry of materials at each step in the process, determine the best geometry of cutting for desired yield, and provide offsets to the saws for each individual cut.

Applications throughout the wood mill include log and cant optimization, board edgers and trimmers, veneer peeling, and lumber sorting, stress grading, and measuring the thickness of panels and boards. Automatic defect detection and classification of boards using non-contact laser measurement combined with other techniques is now installing at an increasing number of planer mills. The latest sensors for lumber applications combine 20 color sensing for defects with 3-D laser sensing in a single sensor package, simplifying implementation and footprint.

Ambient conditions in a lumber mill, such as temperature and ambient lighting, are typically uncontrolled. Processing operations cause large amounts of sawdust and other contaminants in the atmosphere.

Through sensor packaging such as sealed enclosures and air purging to keep dust from accumulating on sensor windows, these problems are no longer problems.

Since sensor data serves in closed loop control, sensors must be robust and reliable. In addition, effective contouring of logs or boards requires typically thousands of data points from random geometry surfaces, which can have widely varying surface conditions and colors, all collected in a small number of seconds.

Dynamic automatic light control is necessary to deal with widely varying light levels from varying surface conditions. A typical sensor implementation has multiple sensors measuring each board passing dynamically on a conveyor.

Today, 3-D scanning works in a variety of lumber processing operations, with growing demand for measurement accuracy and speed, and day today consistency in combination with higher production rates, which have made mechanical and manual optimization methods uncompetitive.

Implementation of vision sensing and related process optimization can increase process yields by 5% to 15% and even more, improving economics and addressing environmental concerns.

Working on the road gang

Inspection of road and runway surfaces for profile, roughness, rutting, cracks, and macro texture provides the ability to predict maintenance requirements, enhance safety, and monitor wear, as well as providing a database for optimizing construction methods.

Manual and mechanical methods are time consuming, labor intensive, and disrupt traffic flow. Since 1979, 3-D laser sensors have provided an ideal way to collect information from roads and runways. Typically, a number of laser sensors sit mounted on the front of a test vehicle, which then drives over the road surface at highway speed, without disrupting traffic.

Sensors for road monitoring encounter a variety of challenges, including widely varying ambient lighting (sunlight to dark shadows), rapidly changing surface texture, color and brightness, dirt, rain, and widely varying temperature conditions.

The laser sensors used for road inspection typically have sealed sensor housings to protect from moisture, large standoff to clear rough roads, and spot sizes down to 0.2mm to provide detailed surface resolution.

Data rates provide sampling intervals as low as every 0.4mm at vehicle speeds of 90 km/hr. Sensors are equipped with fast, broad, dynamic range gain control to deal with varying road surface conditions.

Today, thousands of laser sensors work daily for road authorities and contractors to survey the condition of pavement in many parts of the world. The result is improved maintenance, better roads, fewer accidents, and lower costs for motorists.

Vision sensors have found a broad range of applications in hostile environments in a variety of industries, as these examples show.

The key issues are proper sensor design and packaging to insure sensors can survive in extreme temperatures, next to welding and molten metal pouring operations, and even mount on outdoor vehicles. These capabilities provide many opportunities to apply sensors in process monitoring, closed loop process optimization, and inspection. They improve quality and productivity.


Walt Pastorius, Ph.D., (walt.pastorius@sensorsthatsee.com) is a technical adviser at LMI Technologies Inc. Nicholas Sheble (nsheble@isa.org) is senior technical editor for InTech magazine.


  • As vision sensors became more robust, they became an integral part of the process, with the data often directly controlling the process in an adaptive closed loop manner.
  • Monitoring the dimensions of automotive bodies and subassemblies was one of the first applications using large numbers of 3-D vision sensors.
  • Data can also transmit over a network to plant information systems, statistical analysis systems, or other PCs for custom report generation or other special analysis.


Magic and technology become one

A 3-D scanner analyzes a real-world object or environment to collect data on its shape (and sometimes color).

Using collected data, algorithms, and computer processing, one can then construct digital, 3-D models that work in a wide variety of industrial and commercial applications.

The 3-D scanner creates a point cloud of geometric samples on the surface of the subject. They are analogous to cameras in that they have a cone-like field of view, and like cameras, they can only collect information about surfaces that are visible to them.

While a camera collects color information about surfaces within its field of view, 3-D scanners collect distance information about surfaces within its field of view.

The "picture" produced by a 3-D scanner describes the distance to a surface at each point in the picture.

If a spherical coordinate system is defined in which the scanner is the origin and the vector out from the front of the scanner is j=0 and q=0, then each point in the picture is associated with j and q.

Together with distance, which corresponds to the r component, these spherical coordinates fully describe the 3-D position of each point in the picture, in a local coordinate system relative to the scanner.

Two types of 3-D scanners are contact and non-contact. Non-contact 3-D scanners come in two flavors: active scanners and passive scanners.

Active scanners, like those in this article, emit some kind of radiation and detect its reflection in order to probe an object or environment. Possible types of radiation used include light, ultrasound, or x-ray.

The time-of-flight 3-D laser scanner is an active scanner that uses laser light to probe the subject. At the heart of this type of scanner is a time-of-flight laser range finder.

The laser range finder finds the distance of a surface by timing the round-trip time of a pulse of light. A laser emits a pulse of light and the amount of time before the reflected light reaches a detector registers.

Since the speed of light-c-is a known, the round-trip time determines the travel distance of the light, which is twice the distance between the scanner and the surface.

If t is the round-trip time, then the distance is equal to (c × t)/2.

The accuracy of a time-of-flight 3-D laser scanner depends on how precisely we can measure the time: 3.3 picoseconds is the approximate time for light to travel 1 millimeter.

The laser range finder only detects the distance of one point in its direction of view. Typical time-of-flight 3-D laser scanners can measure the distance of 10,000-100,000 points every second.

The triangulation 3-D laser scanner is also an active scanner that uses laser light to probe the environment. With respect to time-of-flight 3-D laser scanner, the triangulation laser shines a laser on the subject and uses a camera to look for the location of the laser dot.

Depending on how far away the laser strikes a surface, the laser dot appears at different places in the camera's field of view. This technique is triangulation because the laser dot, the camera, and the laser form a triangle.

The length of one side of the triangle, the distance between the camera and the laser emitter is a known. The angle of the laser emitter corner is also a known. The angle of the camera corner can be determined by looking at the location of the laser dot in the camera's field of view.

These three pieces of information fully determine the shape and size of the triangle and give the location of the laser dot corner of the triangle.

In most cases, a laser stripe, instead of a single laser dot, sweeps across the object to speed up the acquisition process.

The principle that serves for these applications and others is the same. Software runs on a PC or an embedded system, and it controls the process and connects to a scanner card.



Solid state: Pertaining to circuits where signals pass through solid semiconductor material such as transistors and diodes as opposed to vacuum tubes where signals pass through a vacuum

EMI is electromagnetic interference, and its affect on signal transmission or reception comes from the radiation of electrical and magnetic fields. Electric and magnetic force fields surround moving electric charges, near weld guns.

Spherical coordinate system is a coordinate system for representing geometric figures in three dimensions using three coordinates, (ρ, φ, θ), where ρ represents the radial distance of a point from a fixed origin, φ represents the zenith angle from the positive z-axis, and θ represents the azimuth angle from the positive x-axis.



Eye on Machine Vision: Lower-cost systems attract manufacturers' attention

Thermal imaging sensors: New safety measures

Seeing it my way: Machine vision is making more and more sense ... now.