New deep learning technique paves path to pizza-making robots

This article is part of our coverage of the latest in AI research.

For humans, working with deformable objects is not significantly more difficult than handling rigid objects. We learn naturally to shape them, fold them, and manipulate them in different ways and still recognize them.

But for robots and artificial intelligence systems, manipulating deformable objects present a huge challenge. Consider the series of steps that a robot must take to shape a ball of dough into pizza crusts. It must keep track of the dough as it changes shape, and at the same time, it must choose the right tool for each step of the work. These are challenging tasks for current AI systems, which are more stable in handling rigid-body objects, which have more predictable states.

Greetings humanoids

Subscribe now for a weekly recap of our favorite AI stories

Now, a new deep learning technique developed by researchers at MIT, Carnegie Mellon University, and the University of California at San Diego, shows promise to make robotics systems more stable in handling deformable objects. Called DiffSkill, the technique uses deep neural networks to learn simple skills and a planning module for combining the skills to solve tasks that require multiple steps and tools.

Handling deformable objects with reinforcement learning and deep learning

If an AI system wants to handle an object, it has to be able to detect and define its state and predict how it will look in the future. This is a problem that has been largely solved for rigid objects. With a good set of training examples, a deep neural network will be able to detect a rigid object from different angles. However, when it comes to deformable objects, the space of possible states becomes much more complicated.

“For rigid objects, we can describe its state with six numbers: Three numbers for its XYZ coordinates and another three numbers for its orientation,” Xingyu Lin, Ph.D. student at CMU and lead author of the DiffSkill paper, told TechTalks.

“However, deformable bodies, such as the dough or fabrics, have infinite degrees of freedom, making it much more difficult to describe their states precisely. Furthermore, the ways they deform are also harder to model in a mathematical way compared to rigid bodies.”

The development of differentiable physics simulators enabled the application of gradient-based methods to solve deformable object manipulation tasks. This is in contrast to the traditional reinforcement learning approach that tries to learn the dynamics of the environment and objects through pure trial-and-error interactions.

DiffSkill was inspired by PlasticineLab, a differentiable physics simulator that was presented at the ICLR conference in 2021. PlasticineLab showed that differentiable simulators can help short-horizon tasks.

PlasticineLab is a differentiable physics-based simulator for deformable objects. It is suitable for training gradient-based models.