Is Vision Guided Pick and Place Really That Scary?
Entry posted by Batu Sipka
2,393 views
Let’s say you are trying to implement a pick and place application with your robot. Industrial robots are amazing in terms of going to the place they were told to go. But what if that place we told them to go changes constantly and we don’t know where the part is going to be next time around. That’s when we use machine vision’s help to guide our robot to the right pick location.
The general idea is that a vision system needs to be looking at the potential pick locations, and tell the robot where to go and pick up the next part.
I’m sure a lot of you would agree that communication is the key to success. That is no different in this case. If two people are speaking different languages that conversation is not going to work great.
In digital cameras, there is a sensor that collects the light from the outside world and converts it into electricity. The sensor has “points” (or you can call it a grid) on it that are called pixels. The images we obtain from the camera are represented in these pixels.
Robots on the other hand, have coordinate systems. And they usually get represented in meters or millimeters. Two different languages...
The whole magic is to be able to know where the camera is located relative to the robot end of arm tooling. Camera can either be mounted at:
-
The end of arm tooling
-
A stationary location
If the camera is mounted at the end of arm, we need to know the location of the camera relative to our gripper. At this stage, we only need to know the relative location in terms of two dimensions. The third dimension, we usually can control. For example, if the camera is mounted 3 inches in X and 1.5 inches in Y away from the end of arm tooling, and our part is in the middle of camera’s field of view, the robot needs to move -3 inches in X and -1.5 inches in Y (on its end of arm tooling’s coordinate system) to be able to grab the part.
Wait a second, how about the Z? In the robot program, I always have a set location to take my picture before the pick so I know how far the robot end of arm tool is relative to my parts in terms of Z.
But what if the part is not in the center of the field of view of the camera, the camera needs to report the part’s location somehow to the robot right? Yes, and that’s when the calibration comes into play.
Basic idea is to calibrate the camera’s pixel readings into robot coordinates. Since most of the time, the robots work in real world coordinates (mm or m), you can use the built in calibration function on your camera software if it has one. These routines usually require some type of a grid with known size squares or circles so the camera can do math to convert it’s pixels to real world coordinates.
I usually don’t do that. I would have to make sure my axes are aligned perfectly and would have to take lens distortion into consideration somehow. I make a randomly marked paper or plate (see the plate below with holes in it) and use that as the calibration grid. Here are some simple steps and tips to get the calibration process working;
-
Jog the robot to the location where the picture is going to be taken and take a picture. Make sure all the markers on the plate are visible on the image and the top of the plate is the same distance away from the camera as the top of the part you are eventually going to pick up.
-
Note all the locations of the markers in pixels from the camera software. You will need to implement some tools in the vision system to be able to get location data from these markers.
-
Jog the robot to each of these markers and note down the locations in robot coordinates.
-
All you need to do now is to match the pixel location of the marker and the robot location of the marker and do math!
Cognex Insight cameras have a built vision tool called N-Point calibration to do this very easily. You select the markers on the plate and write down the corresponding robot coordinate in a table. The software takes care of the rest and your location tools (like PatMax) will report back in robot coordinates now.
Almost the same process when the camera is mounted on a stationary location. You just need to know where the camera is located relative to the arm. Only other thing to be careful about now is that the camera needs to be farther away from the pick area so that the robot arm can swing in and grab the part without hitting the camera.
Understanding the basics, it’s not so scary to do a vision guided pick and place anymore. Does it still scare you? If so leave a comment below or reach out to us at https://www.gibsonengineering.com/.
Disclaimer: This blog post is valid for using 2D cameras for doing a pick and place of the same type of part. Depending on the parts and the hardware used, some additional steps might need to be taken.
1 Comment
Recommended Comments