For a long long time, gamers have had an exclusive opinion on controllers in a virtual world. Well, mostly because not much else needed controllers, right? Actually, that’s not the case. We use controllers everyday when it comes to real life. For instance, door handles, keyboards, touchscreens, elevator buttons, car steering and gear shift – there are a bunch of them everywhere you look and you are used to them.
When it comes to VR, this very familiarity translates into a jarring lack of immersion. When you’re driving a sleek sportscar in game, but actually holding two universal joysticks shaped nowhere close to what it’s doing in game, you can’t help but feel disconnected from the entire experience. Compared to the realism in the rest of the experience, this is really a pity. Or is it?
Experimental devices and technology from the world of tech is trying to solve the lack of immersion that universal hardware controllers cause by bringing in newer methods to control your VR experience that are more analogous to your daily interactions with objects. And there are quite a few that show a lot of promise.
Special Mention – Realistic genre-specific controllers
Before we get started, we would like to make it clear that genre-specific physical controllers are not to be entirely discarded. The PlayStation VR Aim controller has seen a fair amount of popularity. With its integrations with VR content like the Brookhaven experiment, it could very well define the future of VR shooters. The same goes for other genre specific controllers available from third-party providers. If only it weren’t too darn expensive for one to go after a different controller for every genre they like to play…
You’re wearing one
To start off, if you’re experiencing VR, it’s highly likely that you are already hooked onto a bunch of controllers (not the handheld ones). We are talking about the actual headset. The configuration in most of the powerful VR headsets makes your own head a controller. With that clear, let us look at some of the efforts that want to take it one step further.
And no, we are not talking about waving a wand or a stick in front of your face to do stuff in game. What we are talking about here is using your hands naturally IRL and those actions being represented in VR. And so far, Leap Motion has been the ‘one that was promised’ in this case. Their popular VR developer kit and the community of over 200k developers makes it one of the most widely used gesture controllers out there. And as one can see, they have the results to show for it.
But this shouldn’t make you think that Leap Motion is the only company out there that is getting this right. eyeSight’s computer vision solution and Myo’s muscle movement detecting band have also been available for a while. But some of the most exciting ones under development are:
- Hapto: Glove like controller that provides a sensation of touch while allowing simple sliding gestures.
- VicoVR: Bluetooth enabled full body tracking for iOS and Android based VR.
- Pebbles: Acquired by Oculus along with Nimbles VR, both these technologies use depth sensing to bring realistic gesture control to VR.
Overall, gesture looks like the most realistic alternative VR control method right now. Especially with tech like Microsoft Research’s NormalTouch and TextureTouch showing promising results in the labs right now.
Tracking your gaze in VR is an interesting way to control your experience from the get go – Mainly because VR does already account for your gaze. As in, when you look at something without moving your head, you are still looking at it in a pretty realistic way. But, the experience could be improved. For instance, take the example of a medical tutorial in VR. The program asks you to look at the femur of a skeleton in front of you. With the current state of the tech, if your head is pointed in the general direction you will get it right. But with proper eye-tracking the game will be aware if you’re looking at the femur or the hip-bone. Which is crucial if we are looking at training doctors and surgeons in VR anytime soon in the future. Additionally, things like intent-prediction, dynamic DoF and foveated rendering possible.
Major companies like Google, Microsoft and Apple have filed their patents for their own eye tracking solutions. Microsoft even announced native eye-tracking support for Windows 10 pretty recently. But should we expect them to bring it into potential VR offerings of their own? By all means.
The most promising name in smaller companies involved in VR eye tracking is Tobii. With their VR eye tracking devkit for the HTC Vive released earlier this year, called the VR4, they are compatible with all types of Head-mounted displays and offer one of the most popular eye-tracking dev kits out there.
With smaller companies like Pupil Labs and Eyefluence also having demoed their own eye-tracking solutions, compatible with multiple platforms including VR, in the recent past, there is a lot of promise in this area. If only the software to utilise eye-tracking methods also existed beyond their own dev-kits.
Taking things one step further, what if you could just tell a VR experience what to do with your thoughts? Sounds like the perfect solution to the immersivity problem, doesn’t it? Turns out that is not as sci-fi as you might have assumed. Neurable has achieved significant progress with their brain-computer interface and recently provided a very convincing demonstration at SIGGRAPH 2017. As a part of the demo, one could choose an object out of a collection by just thinking about it. Of course, there’s an initial calibration and other tests involved.
Check out their announcement here. With their SDK, Unity developers can integrate brain activity input into any VR experience they are working on. And how will your standard headset, say the HTC Vive, detect this input? Simply replace the elastic straps at the back with the straps provided by Neurable. Yup, that simple.
So, which one?
That’s…kinda hard to say yet. Yeah, yeah, we know that we are almost back where we started. But things are much better than before. From a problem that looked like it had no solution, now we have the problem of plenty and just have to figure out which one works. And like it has always been in the world of gaming, it is upto you to try these multiple types of controllers when (and if) they come out and give the final verdict on what works best!