The mechanical chef reviewed a total of 16 videos and was able to determine the correct recipe in 93% of cases. Additionally, it was able to recognize variations in the recipe and the demonstration of a new ninth salad, which it added to its culinary book and reproduced.
Thanks to the work of researchers from the University of Cambridge, the dream of many people to have a home robot chef may soon come true. The research team managed to teach a robot to cook dishes by simply watching videos of humans preparing them.
“We wanted to see if we could teach a robot chef to learn in the same step-by-step way as humans do, identifying ingredients and how they come together in a dish,” said Grzegorz Sockacki from the Engineering Department at Cambridge, the lead author of the study.
To test the idea, the scientists created eight simple salad recipes and recorded videos of their preparation. They then trained the robot using a publicly available neural network that was pre-programmed to identify various objects, including fruits and vegetables used in salads.
The robot chef analyzed each frame of the video using computer vision techniques. It was able to identify not only different objects such as knives and ingredients but also the actions of the human demonstrator.
The recipes in the videos were transformed into vectors, and by performing mathematical correlations, the robot recognized the similarity between the demonstration and the vector. As a result, it was able to determine which recipe was being prepared.
The mechanical chef reviewed a total of 16 videos and was able to determine the correct recipe in 93% of cases. Additionally, it was able to recognize variations in the recipe and the demonstration of a new ninth salad, which it added to its culinary book and reproduced.
Currently, for the success of the robot, it requires recipes that are not too complex. It is also not yet possible to dream of showing the robot food preparation videos from TikTok, as it is challenging to track the fast-paced food videos.
“But as these robot chefs become better and faster at identifying ingredients in food videos, they could use platforms like YouTube to learn a wide range of recipes,” said Sockacki.
The research group acknowledges that the capabilities of the robot chef are still limited, and many challenges need to be overcome. However, they have demonstrated that a robot can gradually learn to cook by observing humans, which allows for easier and more cost-effective deployment of robot chefs.
