Checking for Object Correctness

by Francisco Barranco, Aleksandrs Ecins, Ching Teo

One of the goals of the project was to design and implement a system with the capacity of detecting mistakes while actions are being performed. In this way, the system corrects on the fly and avoids wasting materials, accidents, or low-quality products. A system like that is interesting in many different applications as for example, manufacturing. Additionally, this step is crucial specially if we think in a future improved system that can learn by imitation.

The 3D recognition of the objects (in our case the plank, cut plank pieces, or the two plank pieces aligned or assembled) is essential for checking the correctness of the actions (in fact, of the products of the actions) at different points in this project.

We set up five different points in which we are checking the correctness of the product based on the geometrical constraints. We combine the 2D and 3D information to do it, segmenting the planes of the planks using the PCL library [1] running RANSAC fitting routines for planes.


At this point, we check that the result of the action is a right line (orthogonal to the sides of the plank) and that it is drawn close to the center of the plank.


In this case, the system checks that the two cut pieces have similar sizes. In the figure, we show the value of the perimeter of both pieces. We use perimeters relationship as a similirity criterion.


After sawing the big plank we get two smaller planks. Then, the subject aligns both pieces to be orthogonal.


Here our system checks if the screw has been screwed in the center of the plank. We compute the centroid using the 3D data information and estimate the distance from the hand to the screw. The values are also displayed in the figure.


Finally, we also check wether the nail is close to the corner of our frame. In the figure we saw the estimation of the distance from the hand to the corner.


[1] PCL Point Cloud Library