Researchers demonstrate Virtual Reality (VR) finger tracking tool FMKit that recognizes air-drawn passwords and PINs
When playing games or visiting a Virtual Reality (VR) website/video, many times you are presented with a credentials login window. Up to now, you had no option but to remove your Virtual Reality headset and type in the username and password/PIN physically. But now thanks to this new FMKit tool, you can air-draw your login details and the website/gaming portal will allow you to enter without removing your VR kit. If the FMKit is commercialized and used by VR kit makers like Leap Motion and Oculus Quest VR headset, you will no longer have to remove your VR headsets to log into your favorite VR channel or game.
The viability of FMKit was demonstrated by its makers, the researchers from Arizona State University. The research team of Duo Lu, Linzhen Luo, Dijiang Huang, and Yezhou Yang have made it possible for VR headsets enabled with FMKit to precisely track individual finger motions, as well as recognizing in-air handwriting. In layman terms, it means you can draw your username and password or PIN/Passcode in the air while using the FMKit enabled VR headset and it will recognize what you have drawn to let you enter the login screen.
The ASU researchers work could usher in a revolution in the AR/VR space which has been stumbling due to lack of innovation. It now enables technology to track individual finger’s path to be recorded in 3D space. It then compares against four data sets of handwriting samples to comprehend what the user has air-drawn. FMKit tool can be used to identify individual users, securely authenticate users by password, and create text input as an alternative to typing, speaking, or selecting words with a handheld controller.
According to the researchers, the FMKit currently supports two types of input devices — a Leap Motion controller that works at 110 scans per second, and a custom inertia-measuring data glove that works at 50 scans per second.
FMKit uses Python modules to gather, preprocess, and visualize scanned signals. According to the researchers, the FMKit can correctly identify the air-drawn PINs and figures with 93% accuracy on Leap Motion gear and nearly 96% accuracy with the customized glove. For passwords and anything containing words, Leap Motion’s gear identifies words accurately 87.4% of the time.
The team has open-sourced their entire FMKit Python module on GitHub. If you are interested in AR/VR technology, you can join the group of collaborators to fine-tune FMKit and make it work on other VR headsets. The ASU research team will be demonstrating FMKit at the online CVPR 2020 Workshop on Computer Vision for Augmented and Virtual Reality to be held later.