Let's talk about Virtual Reality. It's been in the news lately as several recent attempts have overcome some of the early technical difficulties with the screen real estate. Specifically I am responding to some incredibly positive reviews to the HTC Vive, a a VR headset being made in conjunction with noted software and games company Valve, but the Oculus Rift has also been in the news, especially since it was bought out last March by Facebook. Virtual reality, or having your entire field of view replaced by a virtually created environment, seems closer now than it ever has been. Unfortunately, Virtual Reality will be a non-starter in our lifetime.
It has nothing to do with the device itself, and there are exceptions, but those exceptions all fall into one damningly narrow category: games without interesting human characters. Your typical war shooter might have a decent experience in virtual reality, but I predict (as a decent example) Skyrim-esque games will plunge off a cliff straight into Uncanny Valley, and remain mired there for the foreseeable future--at least, by my guess, the next fifty years. An alarming prediction? Let's break it down.
The core of the problem is that making an immersive experience is incredibly tough, not merely technically, but socially. The "Uncanny Valley" problem has been raised numerous times since video games started to get out of the 2D plane and people began to attempt immersion. The Uncanny Valley problem, for those who haven't heard the term, means that although we can understand and empathize with cartoonish and ultra-realistic depictions of characters, there exists an alarming middle ground where the partially-realistic character strikes us as wrong, and we can no longer empathize with it. There are a lot of parts to it, and the game industry has done a decent job of working around it, but it's far from a solved problem, especially when you are entirely replacing a person's sense of reality instead of just putting it on a screen in front of them.
First and foremost, Artificial Intelligence is an unsolved problem and will be for well more than fifty years. We're not even talking about full, hard AI, which is beyond even Ph.D. research theses at this point; currently, almost any non-human-controlled entity in any video game is nothing more than a script meant to do nothing more than to plausibly avoid doing more work on the subject.
For example, many video game characters will look at you immediately when you approach and continue staring at you (politely, if the modelers are good, or unnervingly otherwise) either indefinitely, until you leave, or until some other action is triggered and they walk away. Because in a typical console or PC games, your viewpoint is straight ahead from your torso, if they are in your line of sight, it is to be assumed you are trying to interact with them, and this shows that they are paying attention to you.
In most of these cases, and in far more cases where AI is handled more simply, non-player characters drop everything else they're doing simply because you're in the area, because the script cannot handle the enormous number of variables that a player adds to a situation just by walking in. Do you plan to talk? Interact with objects? Interact with characters? Stand in someone's way? Do something suspicious? Try to enter an area you shouldn't? Attack people? Lay down on the ground? Because there is no such thing as AI, NPCs can only react to specific triggers, and can only take predetermined actions. This is a cute limitation of "flat" (non-VR) games, but it makes immersion impossible.
A more egregious example of this is "Creepy Watson", which serves admirably to highlight what I mean about modern games only attempting to disguise the lack of AI, not actually create it. The game is Sherlock Holmes, and Holmes' omnipresent sidekick Watson has not a single shred of movement code nor a single walking animation. Instead, if at any time you take your eyes off him and are far enough away, he will teleport to you, facing you. The culmination of this is dear Watson always showing up to stare at you, silent, motionless, in ways that defy physics, common sense, or social graces.
All of this is an attempt to avoid doing difficult AI work, for which there are no libraries or prefabricated code. Most if not all video games that makes use of complicated pathing reinvent that particular wheel to some extent, and if the game involves submerging people in a seemingly living world, your pathing and character interactions are going to be complicated indeed.
AI aside, realistically modelling arbitrary human movements is not quite a solved problem, either. Oh, it's been done, but it's only slightly below Ph.D. thesis level, and again, it's far from production code that can be easily dropped into a project. That means that walking on uneven terrain, handling arbitrary objects, and so on, all of it has to be scripted in advance, or else fudged. That's not too bad, until you start thinking about living in and being surrounded in a world where the actions of everyone around you are scripted, or where people aren't really walking on the ground and things happen only approximately the way they should. It can be disguised, but until they are solved problems, every game designer will have to consciously make efforts to do so.
Okay, so, games that require AI are out. What about multiplayer? Motion capture is not such a foregone conclusion; you can capture people's arms and legs and hands and relay that information to the character model. Only, as human beings, we wish for and expect face-time. We watch people's eyes and facial expressions. And unfortunately, in order to experience VR, you just put a giant unwieldy piece of plastic in the middle of your face, which means that it just became significantly harder to monitor facial expressions--maybe even impossible. If someone was incredibly dedicated, they could put a camera to watch your eyes inside the mask, and another to watch your lips and jaw, but even with that, the muscles in your face are now fighting against the mask pressing against it, meaning that even an accurate representation of your face is going to be biased.
Again, you CAN make multiplayer games, with as detailed or simple of character model tracking as your budget (and the consumer's gadgetry budget) allows, but any sense that you are immersed in the world is going to depend on things seeming natural, or at least consistent. VR can give you a true sense of being surrounded by a less-real world, but... is it worth the expense and hassle of putting on a VR headset to move the camera with your head? The only "Killer App" of VR is full immersion, and that's in the details.
Those details are stickier than any of us would like, and it's not going to change anytime soon. Would that it were otherwise, but it's not.