These Self-Learning Robot Arms Teach Each Other How to Pick Up Unfamiliar Objects

GIF: YouTube

It’s all but certain that human workers in factories and fulfillment centers are a dying breed. The last nail in their coffin might be a new robotic arm from a company called RightHand Robotics that’s not only able to teach itself how to pick up objects it’s never handled before, but it can also share what it learns with other robot arms around the world.

One of the biggest challenges of automating a fulfillment center, such as those operated by Amazon around the world, is the staggering variety of products a robot arm needs to be able to pick up and move. Robots in factories are typically purpose-built to handle a single part or tool, but robots collecting products to fill online orders might handle thousands of different objects, all differently sized and shaped, throughout the day. And having to manually teach a robot how to handle every last one makes it less cost effective.

To ensure its new RightPick system can continuously adapt to new products all the time, RightHand Robotics developed a multi-fingered gripper with both an extending suction tool in the middle, and a camera that’s able to analyze objects and determine the best strategy for grasping and holding any object. Images from the camera are instantly processed by an algorithm developed by RightHand that tells the gripper what combination of fingers it should use, and if activating the suction tool is necessary. The system can also take advantage of machine learning techniques to automatically refine and tweak that algorithm as it encounters and learns to handle unfamiliar products.


The new skills that one RightPick system learns on the job can also help improve RightPick setups at other factories and fulfillment centers, as the robot arms remain connected to a cloud server at all time, sharing their collective knowledge to help improve each one’s capabilities. This connectivity also allows the company’s engineers and developers to remotely connect to one of the arms if it’s having an especially hard time adapting to a new object, and needs to be taught how to properly handle it.

For the time being, RightHand Robotics is still testing, refining, and working towards perfecting its RightPick system. But the real benefit here is that it doesn’t necessarily have to be perfect before being deployed somewhere. We’ve probably all fibbed on our resumes a little to get hired, and then learned the new skills we needed to be successful at a new job while working it. That’s the same thing the RightPick system can do, making it more valuable to a company as time goes on.

[RightHand Robotics via MIT Technology Review]

Share This Story

Get our newsletter



It has a very long way to go. I noticed the demo only demonstrated it picking up bottle type or smooth objects and it was really slow and awkward compared to a skilled worker.