Nevertheless, consistent advances in both graphics software and hardware have been propelling ever more realistic graphics into our homes. Movies and gaming were the vanguard for realistic graphics. In only a couple decades we went from Pong to Fallout 3. Yet, we still haven’t ditched the joystick, mouse, and keyboard.
Speech recognition was supposed to make keyboards obsolete, but it isn’t as practical as once thought and the technology has been surprisingly stagnant. Yet, more elegant interfaces have been creeping their way into the consumer marketplace; unfortunately, I think infrastructural inertia will prevent movies and gaming from becoming the vanguard for advancement of these technologies.
Porn will be the vanguard, and from there it will make its way into other markets. If you read Wired, you may have come across one of the weirder technological developments of the 21st century: Teledildonics. These machines are basically interactive sex toys which can be activated remotely by a partner over the Internet. Clearly, it is the porn industry which is the most creative and enthusiastic in embracing such new sensory technologies.
When it comes to immersion, the keyboard, mouse, and joystick simply do not cut it. Yet, why would the developers of Spore think it is worth the effort to, say, make an electronic glove that can allow users to feel virtual textures? There is only one industry which is enthusiastic enough to accomplish such a feat.
Before we talk about what’s possible, let’s take a look at some of the interfaces which are already on the market.
What is already here?
Head-mounted Displays
When people think of VR, the first thing they think about is head-mounted displays. These things are really cool. They encompass your entire field of vision and can provide you with the experience of a screen that is hundreds of inches tall for a fraction of the price. Furthermore, they can provide stereoscopic 3D. Remember those 3D goggles you could wear to watch old 3D movies? Well if you have a game and/or a graphics card that is set up for stereoscopic rendering, with today’s graphics, a head-mounted display can give you an experience that will completely knock your socks off.
It is just tricky to find games, software, and/or hardware which support stereoscopic vision. OpenGL has great stereoscopic support, and if you use linux you can get this plugin which will automatically convert all games for you. For Windows users, you just have to hope your game supports it, or if you are lucky find less efficient and badly supported hardware-based stereoscopic drivers.
The best head-mounted display on the market today is eMagin’s Z800. It provides a gyroscope to detect head movement and a stereoscopic OLED display. OLED is a new technology which is superior to LCD in every way. It is thinner, lower power, flexible, brighter, and more responsive. The only limiting feature is the price. Since these things are new, they are still pretty pricey even though they are produced using very conventional means. This visor is a bit old, created back in 2005. Expect the next generation of OLED head-mounted displays to be arriving shortly, and they will likely be half the price and twice the resolution. Though, currently you can pick one up new for $1,200.
Brain-Computer Interfaces
Now this is the future of gaming. There are currently a couple different devices on the market now which can record your brainwaves using electrodes and use them for gaming purposes. Yet, what everyone is looking forward to is Emotiv’s EPOC headset, which will have much greater functionality. It’s current release date is summer of 2009. It’s target price is only $299 and contains over 20 electrodes. Using this data it can detect your facial expressions, emotions, and certain thought processes. It can track your excitement level during a game and can allow you to manipulate objects on the screen using only your mind. Don’t believe me? Check out this video.
We can only speculate about what will be capable in the future with this technology, but clearly we should expect more electrodes and a greater refinement of the whole experience. As we learn more about the brain, we can apply these concepts towards more creating more realistic virtual experiences.
Right now, this technology requires a certain amount of training in order to tailor the device to an individual user’s quirks, much like how speech recognition technology works. I expect this process to be refined over time, and expect more emotions to be detectable. Currently, one can levitate virtual objects purely by thinking of the act of lifting, and the same can be done with rotating, pushing, pulling, and vanishing. It is also possible to change the virtual weather just by thinking about it. I expect more actions to be possible as time goes by.
Eventually, there may come a time when this technology improves the productivity of mainstream computer users. At that point, we may be manipulating data by pure thought. Also, I am confident it is also possible to compose music this way, lowering the barriers to entry into the world of music, and allowing even those with the most modest of musical talent to compose beautiful symphonies. The same could be true of painting, though in a virtual environment it could be much more fun. Imagine painting with clouds or fireworks.
VR Gloves
Those Nintendo Wiis have been selling like hotcakes. Sure they are small and cheap, but the main reason is the Wiimote. It is a small cylindrical handheld controller with a built-in accelerometer that tracks movement. It also uses optical tracking. This allows for the player to swing virtual lightsabers, bowl virtual bowling balls, and so forth.
In the VR world there have been devices like this for awhile, and one of them is the P5. This device is similar to the wii-mote but allows even more precise control since it is a glove and can track the movement of all the fingers. Unfortunately, it uses a clunky optical tracking system rather than a webcam. Nevertheless, it allows for a surprising degree of control. If you wanted, you could even play a virtual piano on a laptop or netbook during a plane trip.
Speech Recognition
Speech Recognition is mostly here, it has just been underwhelming as far as productivity is concerned. Our friend Ray Kurzweil was one of the pioneers of this field. It has been great for the blind, and other disabled users, but it hasn’t added much benefit for the rest of us.
Yet, there is huge untapped potential for speech recognition in gaming and VR environments. Imagine casting spells with your voice (“abracadabra”) or engaging in spoken dialogue with NPCs. Of course a simple voice chat setup could potentially be used effectively in multiplayer games with serious role players, but more often it completely ruins the immersion. (“Where’s the Cheetos?!”)
3D Sound
3D sound is already a reality. In fact, speakers are so good these days that even modestly priced ones produce sound indistinguishable from the real thing. Those who have surround sound speakers already know how much better movies can be with sound coming from many directions. In 3D environments, this is even cooler since the sound changes direction as your character does. The computer’s 3D sound software, like OpenAL, does most of the work and is implemented in most games. You don’t need an expensive 7 speaker setup to experience this. Just get a good pair of surround sound headphones and you’re set.
What needs work?
First and foremost, all of this technology must (and will) become cheaper in order for mainstream adoption to occur. This is the only way for these technologies to mature. Yet, even if all of the above technologies were perfectly implemented right now we would still have a few sensory inputs left out to dry. Our senses simply send electrical signals to our brain, and we have always been able to find ways to trick them one way or another.
Taste and Smell Emulation
There has been some development in this area, but as far as I know nothing will be hitting the market soon. This is one area that I expect pornography to pioneer the new technologies. Pheromones, tastes, and smells are a big part of the sexual experience for many. However, I think almost anyone would be interested in tasting virtual cake or smelling virtual roses. We just wouldn’t be willing to pay the big bucks to do so. The users of porn, on the other hand, have reliably shown interest in shelling out the big bucks. Unfortunately, I think there wont be much demand for this until the other senses have been thoroughly emulated and this is the only one remaining.
Touch Emulation
Feeling surfaces and textures in virtual reality is a long-held dream, but only the most basic of systems can currently be bought, and they are ridiculously expensive. It is over $4000 for a deluxe VR haptics glove. That glove combines all the features of the P5 above, but also has the ability to feel pressure when virtual objects are touched.
There is nothing on the market currently that lets you feel textures, but work is under development. For full immersion, such a haptics system would have to be expanded to a full bodysuit, not just a glove. Remember the movie AI? This will be the hardest of all to implement, but I have faith in the determination of the virtual sex industry.
Final Thoughts.
This technology is rapidly approaching a point where the boundaries between the virtual and the real will be demolished forever, and this will present tough ethical challenges to society. It will also produce a whole lot of fun, and may even boost productivity. All of this must be considered. There is a lot to be hopeful about, but it is imperative that we critically examine the the social consequences.
Would this sort of VR positively or negatively impact people’s conception of reality? What effects will it have on children? Will this change the nature of classrooms?
Will this make telecommuting more common? Could this actually reduce the need for transportation, lowering the cost of living and our ecological footprint?
When these sorts of technologies become integrated into something like Second Life, will we ever leave? Will this stifle progress or enhance creativity? I have already given examples of the wealth of new avenues for creativity.
Lawrence Lessig makes the point that John Phillip Sousa bemoaned the 20th century’s abandonment of streetcorner singing and individual creativity, yet Lessig argues the 21st century is lowering the barriers to entry into music and other creative fields. With Creative Commons, online music streaming, file sharing, and open source music software more and more music is being freely produced, shared, and remixed. The 21st century is shaping up to be filled with amateur creativity.
What effects will this have on our social lives and political institutions? There has already been quite a bit of virtual activism, and a whole lot of virtual tyranny.(DRM) Is virtual deviancy acceptable? Only a few quixotic anti-videogame crusaders want to ban virtual violence, but what about virtual pedophilia?
All our values will be challenged, but some may be strengthened. I have already argued that these technologies will actually enhance the viability of monogamy in the short term, though the definition of monogamy itself will likely change given the new customizable nature of our personal reality.
There is great potential for this to actually improve social relationships. The clunkiness of our 2D social networks will become painfully apparent once Augmented Reality and fully immersive VR is commonplace. We will be able to interact with others in a more natural way.
The anonymity inherent in virtual worlds will present new avenues for freedom of expression, but also new avenues for destructive behavior. Trolls are already a problem on the Internet, but imagine 3D trolls. Those of you who play MMORPGs don’t have to imagine this, but as the immersion grows so does the potential for annoying and hurtful behavior. Granted, we will be able to block trolls and customize whatever else we wish, but I think the downsides are already becoming obvious. What sort of spam and scams will these environments make possible? What about virtual rape?
Furthermore, what sort of psychological effects will this unlimited power of customization create? We are already experiencing a trend of cyberbalkanization. We tend to view information that already agrees with our viewpoints, and as our choice of media sources grows, intellectually isolated groups of people spring up who from birth have only been exposed to one ideological viewpoint. VR might take this to a whole new and disturbing level. Of course the “good ol days” when there were only a few papers and three news networks to choose from weren’t so great either. Perhaps VR will instead foster a more connected global community based on shared values, and allow all the diverse groups of people to peacefully live out their individual fantasies virtually. Be it religious fundamentalists or pedophiles.
In Neil Postman’s book Technopoly, he convincingly argued that technology changes our values in ways that extend deductively from the function of these tools. It is worthwhile to consider how to keep what is good about our current values as best we can as we move forward.