This System Uses Encoded Light for Millimeter-Level Finger Tracking

If you want to track how a person, or a person’s hand is moving, there are several techniques including using an accelerometer or a vision system. Researchers at the UC Irvine and the FX Palo Alto Laboratory have taken things to a whole level with Lift — a visible light-enabled tracking technique that can localize 10 fingers simultaneously with an average accuracy of 1.7 millimeters, and an average refresh rate of 84Hz.

This incredible speed/accuracy is accomplished via a DLP projector that places a pattern of coded images onto a surface. Sensors placed on each of the user’s finger send light intensity signals to an Arduino MKR1000 mounted on each wrist, which decodes this data into positional information.

Lift can currently work with normal “non-smart” gadgets in two dimensions, but there is consideration of using this method with multiple projectors and even inertial sensors to further enhance its capabilities.

By projecting encoded visible patterns onto an object’s surface (e.g. paper, display, or table), and localizing the user’s fingers with light sensors, Lift offers users a richer interactive space than the device’s existing interfaces. Additionally, everyday objects can be augmented by attaching sensor units onto their surface to accept multi-touch gesture input.

Intrigued? You can read more about the project in the team’s published paper here.