Interact in Virtual Reality with Self-Assembling Robots

When it comes to virtual reality, the problem of displaying the environment has largely been solved. Headsets will certainly continue to advance, but they’re already very functional. The real challenge with virtual reality is how to interact with the intangible objects you come across in the world you’re exploring. The question is how to represent those in-world objects, which can come in any size or shape, in a way that they can be physically handled.

One interesting solution from Stanford’s SHAPE Lab is to use self-assembling robots to represent virtual objects. Each individual robot is a simple small block that’s capable of driving around on a flat surface, like a desktop. Magnets on each side of the blocks allow them to be mechanically connect together. The robots can then arrange themselves into rough approximations of the item you’re handling in the virtual world.

While the resulting objects themselves aren’t particularly detailed or refined, they’re close enough to provide some interaction that would otherwise be unobtainable. When you’re done with one object, the robots can rearrange themselves into whatever is needed next. At its current stage, the system is far from ready for primetime. But, smaller and more sophisticated robotic blocks could make the technology more practical.