ETH Zurich’s PuppetMaster Robot Can Manipulate Marionettes

ETH Zurich engineers have designed a robotic system capable of animating puppets or marionettes, and although that sounds incredible enough, it’s not the end-goal for the platform. According to a recently released paper entitled: PuppetMaster: Robotic Animation of Marionettes, “Our long term goal is to enable robots to manipulate various types of complex physical systems — clothing, soft parcels in warehouses or stores, flexible sheets and cables in hospitals or on construction sites, plush toys or bedding in our homes, etc. — as skillfully as humans do.”

The PuppetMaster system was built around ABB’s Yumi IRB-14000 collaborate robot, which itself, was created as an assembly robot for individualized products, making it a great choice in ETH Zurich’s endeavor. The engineers outfitted the robot with a predictive control model that accounts for the puppet being manipulated and the kinematics of the robotic puppeteer. The algorithm employs a physics-based simulator that helps the robot determine how the puppet will move based on its movements and corresponding motions.

To animate a puppet, all the PuppetMaster needs to know is its design and the target motion in how its actuated. While moving the doll in real-time, the robot is simulating its motions while optimizing those movements to get the puppet to actuate the way it was intended. The engineers have validated their PuppetMaster system using a series of experiments conducted on several marionettes and their motions- including a bird-shaped puppet, bounding puppy, slithering snake, and flying dragon, which it did quite well, even for a robot. The engineers still have a long way to go before their system can manipulate marionettes in the same manner as professional puppeteer’s, but it’s a good step in the right direction.