The Right Touch: On Controllers, Touchscreens, and VR

Critical Distance’s latest bi-monthly topic for the Blogs of the Round Table is as follows:

Joysticks. Keyboards and mice. Mashing a controller with your fist. Touching. Poking. Waggling. Wiggling. Moving your head around a virtual reality world. Directing an arc of your own urine. The ways in which we can interact with games have changed from simple electrical switches into much more complex and nuanced forms. We can even adapt and alter controls for people who have difficulty using traditional methods.

Some of these methods work, and some don’t. Most of us we be familiar with complaints about the Wii’s “waggle” controls, the thumb-numbing frustration of virtual buttons on a touchscreen device, or the gyroscopic motions that ruin the 3D bit of the Nintendo 3DS.

How do we move forward with controls in games? Are the old ways the best, or a barrier to entry? Are you looking forward to playing Farmville on the Facebook-ulus Rift?

A few months back I talked about the difficulty in having a “Citizen Kane of videogames“, in a post that was responding to what GB ‘Doc’ Burford thought that game might look like (1, 2).  He made a number of great points – and, indeed, his ultimate premise that the Citizen Kane of games will be a first person shooter now looks awfully prescient considering what’s happened with the Oculus Rift in recent weeks – but I felt that the ultimate problem with a game achieving that sort of status came down to the control scheme.

Citizen Kane is a movie that can be experienced by anyone with access to the movie (and the senses with which to properly perceive it).  I’m not sure what the “Citizen Kane” of music might be, but let’s maybe, for the sake of argument, call it The Beatles’ “Sgt. Pepper” – if you have the proper equipment with which to play the album, and a set of working ears, you can experience it as it was meant to be experienced.  Same thing goes for books (and my personal CK of books, “Infinite Jest”) – all you need is a copy of the book and the ability to read.

The problem with games, however, is that it’s not just the game that you need in order to properly experience it – you also need to be familiar with how to play it.  You need to be trained – by experience or instruction – on how to interface with the work in a way that is distinct from the passive consumption of other media.   Gamers, as a general rule, have had their entire lives to get familiar with gamepads and/or mice and keyboards, and so even if a new game is introducing a wildly different control scheme than anything that’s come before it, gamers will have a natural advantage in terms of becoming acclimated to that scheme.  They’re not hunting and pecking for the buttons; their fingers are already where they need to be, and moving and interacting with the digital world soon becomes second nature.  Many non-gamers, meanwhile, will take one look at a modern game controller, perceive it as an obstacle and pre-emptively surrender.

The Wii made great strides in attracting non-gamers to the medium; having a 1:1 relationship with what was happening on screen made a lot of intuitive sense.  I have vivid memories of my in-laws, in town to take help look after my wife after her foot surgery, standing around in my living room playing golf and bowling – and my wife, immobilized in a chair, kicking all of our asses, actually.

The Kinect was supposed to make this even easier and more intuitive by completely removing the need for a physical device.  Of course, the original Kinect camera wasn’t necessarily as precise as it needed to be, and I hear anecdotal evidence that the Xbox One’s Kinect, while certainly superior to the previous incarnation, still has problems here and there accurately tracking physical movement.  The larger issue, though, is that nobody really likes running around and jumping and waving their arms all that much in service of a videogame, and it would appear that most games these days have taken the hint and aren’t really utilizing motion controls all that much, if at all.

Touch-screen devices – smartphones and tablets – have made huge inroads in terms of getting non-gamers involved in gaming.  For the most part, controls for touch-screen devices are incredibly intuitive, and more and more “non-gamers” are playing games on these devices.  Virtual buttons and d-pads are tricky to get the hang of, of course, but most of the people I see on the subway are playing Candy Crush Saga, and so they don’t necessarily deal with the same frustrations that others do.

Indeed, even my one-year-old son has some understanding of how tablets work; my wife and I have downloaded some drawing apps for him, and he intuitively gets that he can touch the screen and things will happen in direct response to his touch.  (Likewise, my mother, who claims to be totally in the dark about technology, has no problem playing Solitaire on her iPad.)

Still, the next big thing appears to be VR technology.  Facebook’s already-infamous $2B acquisition of the Oculus Rift, plus the sudden reveal (and positive reviews) of Sony’s Project Morpheus, seem to indicate that VR is not a fad (at least, not in the way that 3D now appears to be).  People who try it are instant converts (or, alternately, instantly nauseous).  But the converts seem convinced that VR is the real deal, and considering the pedigree of the recent hires at Oculus, one is tempted to believe them.

And yet, will strapping a silly-looking helmet onto one’s head really appeal to the non-gamer?  Is that how we will experience the Citizen Kane of games?  Fellow BoRT writer Matt Leslie at Lesmocon writes:

The main problem I can see with VR is that it’s overwhelmingly antisocial. I can’t picture a world where two or more people would sit around in a living room all with these things strapped on, nor would you let other people “watch” you play it. You can argue that its strengths would be in single player games anyway, but a single player game that you’re basically forced to play alone is not going to be the new hotness. Videogames have become highly social and are not something near-exclusively played by little kids and closeted nerds any more, so any advance in the technology needs to acknowledge that.

His point is a good one, though it should be noted that if there is any company that can figure out how to make VR less anti-social, it’d probably be Facebook, right?

Ultimately, the reason why I keep viewing this topic with the Citizen Kane qualifier is that, for all intents and purposes, whether and how a creative work can be accessed matters in terms of keeping the medium moving forward.  Gamers have adapted to new controllers before, and will continue to adapt – even though it seems that the Xbox360 controller more or less set the standard as far as handheld controllers go (and as evidenced by the iterative similarities of the XBO and PS4 controllers).

The Wii and the Kinect were novel approaches to attracting a wider audience into gaming, but, sadly, they became novelties themselves because there was no truly great game that took full advantage of what those control systems had to offer.  It would appear that VR has much wider potential in terms of immersion, and one can create a wider variety of games for it – but it’s also going to be expensive, and silly-looking, and the whole nausea thing can’t be ignored.

In my opinion, touch screen controls might very well be the key towards making games easier to play for the non-gamer, since so many non-gamers are already using touch screens and have an intuitive grasp of how they work.  This is also why the Steam controller is so interesting to me, personally speaking – of all the new controller ideas, the Steam controller’s touch pads seem like they might be the most capable – and the most intuitive.

And so if the ultimate goal of controller design is to get more people involved, then it seems to me that they’ll respond easier to concepts they already understand.

NOTE: Because of WordPress weirdness, I can’t include the little widget that allows you to see other submissions for this BoRT topic.  But I encourage everyone to check out the BoRT page to see further submissions as they roll in.