There are currently 4 main interactions and controllers in virtual and augmented reality systems: VR controllers, haptic gloves, and keyboards (virtual and physical).
Virtual keyboards are slow and input bandwidth is very limited. Controllers work great for games and other interactions but are severely limited in text input. Physical keyboards are faster than virtual ones but require a keyboard to be brought around, adding to clunkiness, and haptic gloves are great for natural interaction but also suffer from limited text inputs and lack the speed of input or button pressing of VR controllers.
Based on this context, ControlXR is a new set of VR gloves that combine all four of these interactions into one, universal controller which has high text input bandwidth, can act as VR controller, and in addition as haptic feedback gloves that can feel dozens of different virtual materials.
We use piezoelectric sensors and actuators on the tips of the finger, and through the computer vision of the VR headset we determine the keys being typed.
Exploring different novel interface of keyboards within VR and through haptic feedback, we tested different keyboard designs throughout the semester.
The Haptic glove function of ControlXR is functional: when touching virtual objects, the glove gives haptic feedback on the correct fingers. To implement this, we used Unity's XR Interaction Toolkits new library XR Hands. The Haptic feedback is customized based on different frequencies, amplitudes, and waveforms.
Currently our team of 6 is working on designing a custom PCB to drive piezos using Boreas haptic actuators, and reverse engineering the BOS1901 kit to be controlled using an ESP32 microcontroller. Now, the team is focusing on improving a novel typing interface that integrates this piezoelectric feedback.
To enable wireless communication, we successfully connected an ESP32 to a VR headset with two-way communication, allowing any custom message to be sent back and forth for driving and sensing.
Early glove prototype with PCB powering piezoelectric actuators when touching virtual objects.
Custom PCB design to communicate between Quest and piezo-electric actuators
Implementation of VR controller
Development of waveform based tactile sensations
Improvement and extension of range of tasks that can be done with the gloves