Is automated bin picking finally real?

With the lowest unemployment in nearly 50 years, the need for automated bin picking is urgent. In the US, where 38% of the manufacturing labor force moves parts between bins and manufacturing machines, 500,000 jobs remain unfilled.  The automation industry is trying to meet that need: every automation trade show adds a few new companies that claim to have finally solved the puzzle of bin picking.  The 2018 Automatica show in Munich, for example, had no fewer than 14 bin picking demonstrations.   But even at large manufacturers, few bin picking stations can be found, and at SMEs, the number is close to zero. Why is the adoption rate so low, when the need is so great and so many vendors claim to offer a solution?

A complex problem

The simple answer is that for the most part, automated bin picking is only a partially solved problem.  Picking randomly positioned parts from a bin and placing them precisely into a machine is a simple task for a human, but a daunting task for robots.  Robots must be able to grasp parts in an infinite number of orientations and reach deep into the corners of the bin- all while avoiding collisions with the bin, other parts, or the work cell itself.

Currently, bin picking is fully automatable only with a huge system integration project, one that requires multiple advanced technologies to work together. These include:

• A 3D model of the part, the bin, the robot end effector, the placement target, and any environmental obstacles
• A model of one or more ways to pick up the part with the end effector and deposit it at the placement target
• A 3D sensor to map the bin
• Image analysis software to locate each part and potential obstacles in the bin
• Path planning software to find a collision-free route from the part’s pick-up point to the placement target
• Robot control software to maneuver the robot, end effector, and part along the route

There are commercial bin-picking systems that include some of these components, and address a subset of bin picking challenges. Usually, these systems combine a 3D sensor with image analysis software that runs on a separate computer.  A robotics expert is expected to integrate the sensor, separate computer, separate software, and robot controller, and then write a program to retrieve the location of each part and figure out how to get it to the placement target.  Creating a general path planning algorithm for an infinite variation in part orientation is a near-impossible task.  At best, weeks to months of experimentation and tuning yields a specialized algorithm with unreliable performance. (see Figure 1).

Because bin picking is so complex, people have attempted to apply Deep Learning techniques. So far, the results have been disappointing, at least for industrial use.  AI is very effective for applications like image classification and voice recognition, where lower accuracy is acceptable.  But AI simply doesn’t have the reliability and accuracy that manufacturers need to replace human operators for bin picking.

Meeting the needs of small manufacturers

With the large integration and programming effort required, it’s no surprise that most real-world bin picking deployments are found at large, sophisticated manufacturers like automotive OEMs.  But most bins and manufacturing machines are found at SMEs (69% of the industrial labor force worldwide).  SMEs have the greatest labor shortages but the least capital and expertise required to create a bin picking system.

A truly universal bin picking solution needs to be usable by non-experts, configurable in a few hours, provide sophisticated path planning that works with little or no tuning and be cost-effective for SMEs.

Energid is democratizing automated bin picking by providing a complete pre-integrated system, with all the complexity managed by the underlying software.  All the system components (sensor, imaging, and path planning software and co-processor) are pre-configured and plug into the robot controller with no setup required.  This reduces installation time to mere hours.

All programming is integrated with the UR cobot’s teach pendant through the UR+ platform, enabling bin picking actions to be freely mixed with standard robot commands using the same programming interface. Training time is minimal. Setup and programming are guided by a series of wizards that mimic the training a human operator would have to go through, e.g. how to pick up a part, where to put it down, obstacles to avoid, etc.   

Automated path planning is key to ease of use and reliability

Planning a unique collision-free path for each part in the bin to the placement target is by far the trickiest challenge for bin picking, and is often left to the ingenuity of an automation expert.  Path planning is the main determinant of system reliability, and if not done well will result in collisions, parts left in the bin or dropped, and missed targets.

Energid’s bin picking solution leverages Actin, the intelligent path planning and motion control software that has been proven in mission-critical applications like NASA spacecraft, autonomous robots on oil drilling platforms, surgical applications, and industrial automation.  Behind the scenes and in real-time, Actin automates the path planning for each part, improving performance and reliability, without the need for user-defined waypoints and manual tuning.

Finally, a bin picking solution for SMEs

So, can we declare that the bin picking problem has been solved once and for all, and manual machine tending is a thing of the past?  Not quite, since there will always be applications that are just too complex for automation, and will require human operators for the foreseeable future.  However, advances in cost-effective cobots, 3D imaging, and intelligent motion control software have aligned to enable a new generation of bin picking solutions that can handle many more of today’s machine tending applications, and enable manufacturers to shift their limited resources to higher-value tasks.  More importantly, bin picking is now accessible and cost-effective for SMEs.  Finally, a bin picking solution for the rest of us.