Use Machine Learning to Play an Entire 88-Key Piano with Just 8 Buttons

The piano is one of the most popular musical instruments in the world, but that doesn’t mean it’s easy to play. A beginner pianist can easily spend months practicing just to learn to play a few simple tunes. However, while playing the piano is certainly an art, it’s also a science governed by well-documented music theory. There is always room for individual expression, but the patterns that sound good to humans are well known. That means those patterns can be deciphered with machine learning, and that can be used to play a piano with just eight buttons.

Piano Genie was created by Chris Donahue, Ian Simon, and Sander Dieleman of the TensorFlow Magenta team. Magenta is a research project with the goal of using the power of Google’s TensorFlow machine learning platform to create music and art. In the case of Piano Genie, they’re using TensorFlow to build an “Intelligent Musical Interface” that dramatically simplifies how a musician plays the piano. With Piano Genie, the musician can tickle all 88 ivories of a standard piano by pressing just eight arcade-style buttons.

The machine learning system was trained on a collection of about 1400 performances from the International Piano-e-Competition. TensorFlow analyzed that data set to determine the sorts of patterns, like scales and keys, that we enjoy listening to. It then matched the basic structure of those patterns to the eight buttons. So, when the musician pushes the eight buttons in sequence, Piano Genie is able to determine that they’re intending to play a rising scale. Far more complex improvised pieces are possible, because Piano Genie is always trying to correlate the musician’s button presses with musical patterns that it knows are pleasing.