Play Rock, Paper, Scissors on Your Raspberry Pi with Computer Vision

You already know how to play the classic schoolyard game Rock, Paper, Scissors — the rules are incredibly simple. Because those rules are so simple, it’s easy to program a Rock, Paper, Scissors game if you use buttons or commands to select which move to play. The computer can then choose a move at random, and compare it to yours. That’s what Julien de la Bruère-Terreault and his son started with, but then they took it many steps further by giving the computer the ability to recognize your actual hand gestures.

Julien’s project uses the OpenCV computer vision software running on a Raspberry Pi 3 Model B. A Raspberry Pi Camera Module V2 is pointed at a platform where you place your hand and form a gesture for whichever move you want to play. A machine learning algorithm then compares that to gestures it was trained on to determine your move, and a Python program determines whether that move beats the computer’s.

The biggest challenge for this project was accurately recognizing which gesture a hand was making. For that to work smoothly, the platform was 3D printed in green material, so that it could act as a green screen for isolating the hand. The image is then processed in four different ways to ensure a strong match with the training images. That methodology gives accurate results, and the machine learning system could also be used to expand the game into the infamous Rock, Paper, Scissors, Lizard, Spock from The Big Bang Theory.