Carnegie Mellon Researchers Mod Smartwatch to Sense Hand Activity

We already use our smartphones and watches to keep tabs on what our bodies are doing, whether it’s a fitness, medical, or sleep-tracking app, but who knew we needed to keep track of what our hands are doing? Researchers from Carnegie Mellon University’s Human-Computer Interaction Institute were able to use a standard smartwatch (which model is unknown) to identify a myriad of different hand movements and correlate them to specific activities — including using a mouse, playing piano, and even brushing hair.

The researchers were able to tap into the watch’s operating system, made a few adjustments, and used its accelerometer to recognize hand motions. They were also able to (in some cases) identify bio-acoustic signatures associated with 25 different hand motions at about a 95% accuracy. While their breakthrough may sound like a privacy advocates worst nightmare, the researchers envision smartwatches as a base platform for the body to capture everyday activities. Apps could be made ‘smarter’ by knowing what our bodies are doing at any given task.

The researchers tested their system using 50 volunteers, each wearing a modified smartwatch for 1,000 hours of daily activities. The watches recorded hand motions, orientation, and bio-acoustic data, while participants described what their hands were doing during each movement. More than 80 hand activities were labeled, giving the researchers a rich data set to draw upon to identify actions. As it stands, the smartwatch is only capable of identifying activities while being worn on your dominant arm for the system to work, however that may change in the future at some point.