Skip to main content
A robotic gripper with suction cups is holding a small cardboard box while being adjusted by a person.

Bin picking: A real challenge for robots

Ines Ugalde Diaz | Siemens Inventors of the Year | Newcomer

There are tasks that are easy for humans but really difficult for robots, and bin picking is one of those tasks. For example, robots need to be able to pick a single part out of a bin containing an unsorted, chaotically arranged pile of individual parts like a pile of parcels.

Ines Ugalde Diaz and her team in Berkeley (U.S.) have been developing robot control systems for more than six years and have made over 40 inventions so far. Ines was honored as Inventor of the Year 2024 in the Newcomer category for a bin-picking solution that can be flexibly modified for different robot gripping tools. This particular invention was selected because it serves as the seed of the next generation of Siemens SIMATIC Robot Pick AI. Pick AI has been featured at numerous trade shows, including this year’s Hannover fair in Germany in the spring of 2024.

Gripping by vacuum

A person is standing in front of a whiteboard covered with diagrams and handwritten notes related to 3D object recognition and ranking.

Many robots grip by creating a vacuum. At the end of their arm – what’s called the end effector – robots employ one or more suction cups to exert a vacuum onto the object to be gripped. To do this, the suction cups have to land on an optimally, flat, smooth, and non-porous surface. There are many variants of suction pads in different sizes and shapes, with a rectangular or square base and with one or more suction cups.

“Our new invention facilitates the use of suction grippers in variable sizes and arrangements. It's up to the customer to decide what they want,” says Ines. “That's actually a very unique aspect of our product that differentiates us from all the competitors.”

Seeing through the chaos

In the chaos of a bin, the individual parts lie on top of each other in random spatial orientations. The robot first has to recognize the structure in this chaos: It needs to identify where a part begins and where it ends and understand how it’s positioned in order to find a place where it can get a good grip. The necessary data – the current view of the box – is provided by a 3D camera. AI algorithms that have been trained to differentiate the individual objects in these images lay the groundwork for controlling the robot arm.

Enhancing standard software

The awarded invention builds on the basis of scene understanding, in particular distinguishing the bin and the individual objects. This is usually referred to as “instance segmentation,” a standard problem in machine learning and computer vision. There are already many pre-trained models for this task that have benefited from training on millions of examples. They’re good but they’re too general, and they make mistakes in bin-picking scenarios. Ines and her group improved the performance of these standard solutions to satisfactory levels by giving them an extra training on specific data sets: for example, real-world data from existing Pick AI robotic cells.

A person is working at a desk with two computer monitors displaying code and a robotic arm visible in the foreground.

Data-sharing for a better performance

A hand is pointing at a computer screen showing a colorful digital image with a green checkmark indicating successful processing.

“Our solution is intended to work at any customer site, because we trained it on vast data sets from both synthetic and real-world sources,” says Ines. “It does really well on boxes, bottles, and bags, and also if the part comes wrapped in plastic foil. In principle, customers shouldn’t see any operational degradation, but if they do, they can allow the models to learn from their failures. This is part of our strategy for the product. We’re preparing to ingest large-scale data sets from fleets of robots into the Siemens cloud, which has the ability to leverage a failure case to make our product better – not just for one customer, but for all customers who agree on data-sharing.” The improvements not only lead to performance gains but also to revealing new use cases. “We’ve shown that with minimal effort we can modify the software to take on completely novel use cases, including robotic depalletizing,” Ines explains. Pick AI is at the core of a newly deployed robotic depalletizing cell at one of Siemens productive warehouse in Erlangen, Germany. Thanks to the data, the team is pushing the frontiers of robotic manipulation.