Robots to learn more human-like behaviour without a real-world set-up

While Dactyl doesnt need humans to input knowledge to teach it, the way in which it learns is certainly human-like. As humans learn from failure and through trial and error, Dactyl uses reinforcement learning to become smarter.

The Shadow Robot Company, experts at grasping and manipulation for robotic hands, have teamed up with OpenAI, a San Francisco-based AI research company, for the second time to prove that agents trained in simulation can effectively perform human-like tasks without real-world replication of training.

Shadows Dexterous Hand manipulates physical objects with unprecedented human-like dexterity under the command of OpenAIs robotic system called Dactyl. Dactyl is trained entirely in simulation using the MuJoCo physics engine. This simulation is only a rough approximation of the real robotic set-up but through domain randomization (altering aspects of the environment like its physics and visual appearance), Dactyl can digitally learn how to solve real-world tasks without needing accurate modelling of the real-world or demonstrations by humans.
While Dactyl doesnt need humans to input knowledge to teach it, the way in which it learns is certainly human-like. As humans learn from failure and through trial and error, Dactyl uses reinforcement learning to become smarter. After learning the techniques - in this case, reorienting several kinds of objects - Dactyl can then transfer its knowledge to govern the Shadow Hand to perform goal-orientated tasks effectively, without the fuss of fine-tuning.
The Shadow Hand is designed to mimic the human hand, from its overall size being that of a human male to a total of 24 joints, to having the range of movement of the human hand. Using OpenAIs Dactyl software it is seen to grasp and manipulate objects with state-of-the-art precision using grasps observed in humans, such as tripod (a grip that uses the thumb, index finger, and middle finger), prismatic (a grip in which the thumb and finger oppose each other), and tip pinch grip. It also picked up Dactyls learnings to twist and toss the object into the desired position.
After you place an object - such as a cube or a block - in the palm of the Shadow Dexterous Hand, you ask Dactyl to move it around; for example, to rotate the cube to put a new face on top. Dactyls network is told the coordinates of the fingertips and uses images from three regular cameras positioned around Shadows Hand. OpenAI tested how whether Dactyl could achieve 50 rotations before it dropped the object or failed to find a movement. The idea is to provide the system with a variety of experiences rather than maximizing realism.
Why is this exciting? Although the first humanoid hands were developed decades ago, using them to manipulate objects effectively has been a long-standing challenge in robotic control. Unlike other problems such as locomotion, progress on dextrous manipulation using traditional robotics approaches has been slow, and current techniques remain limited in their ability to manipulate objects in real-life settings. Now, using Shadows Dexterous Hand, which is the closest resemblance to the human hand available on the market and a great tool for AI testing, OpenAI were able to show successful application of reinforcement learning into the real world.

Rich Walker, Managing Director of the Shadow Robot Company says: "Its been a great pleasure to work with OpenAI again. We have shown robots can achieve amazing levels of dexterity in such an elegant way and were keen to further demonstrate the capabilities of deep learning algorithms in solving real-world robotic challenges."
Touching on the topic of the Shadow Hand, the team at OpenAI state: "Modern-day robots are typically designed for specific tasks in constrained settings and are largely unable to utilize complex end-effectors. In contrast, people are able to perform a wide range of dexterous manipulation tasks in a diverse set of environments, making the human hand a grounded source of inspiration for research into robotic manipulation."

Helpful Materials:

OpenAI video: https://www.youtube.com/watch?v=jwSbzNHGflM
OpenAI blog: https://blog.openai.com/learning-dexterity/
Images: https://blog.openai.com/learning-dexterity/ (Please credit OpenAI)

About Shadow:
The Shadow Robot Company, based in London, is one of the longest-running robotics companies in the UK, developing dexterous robotics manipulation technologies ("hands for robots") and using their advanced grasping and manipulation software to solve real-world problems. Shadow was founded in 1987 by a group of robotics enthusiasts and for the last ten years, Shadow has delivered exciting engineering projects for clients with outstanding work in Mechanical, Electronic, Software and Pneumatic Engineering. Shadow's collaborations range from theatre projects with Leeds University to working with both NASA and ESA (European Space Agency) and universities throughout the world.
Shadow will be attending ROSCon and are exhibiting at IROS in Madrid, Spain (29th Sept and 1st Oct 2018)

Featured Product

Robotmaster® 2024

Robotmaster® 2024

Program multi-robot cells and automatically solve robotic errors with ease. Hypertherm Associates announces a new version to its robotic programming software. Robotmaster 2024 addresses key market trends including the support for programming multiple robots in a single work cell and the demand for automatic trajectory optimization and robotic error correction.