Tuesday, May 2, 2017

The war in heaven: a three-dimensional VR culture clash


Virtual reality is a weird collaboration between several different fields / industries, and each practice brings its own assumptions and baggage. When I go to VR events and conferences away from the influence of games, I often feel bewildered and confused by the different language and norms.

One time I was at a businessy VR event near Wall Street and some tech-biz guy said he had a "VR-ready industrial robot" deployed in a warehouse, and would any angels [investors] be interested in taking a tour? I was so confused (what the hell is a VR-ready industrial robot?) so I joked aloud, "is it a sex robot?" -- but no one else reacted at all, they just continued with the conversation and asked him about market caps or something, as if I didn't even say anything! Not even a pity smirk or furrowed brow or roll of the eyes!

This is part of a larger skirmish about language, and about how to talk about VR. The PC manufacturers have adopted "VR-ready" as a euphemism for "expensive gamer computer"... In tech, you have to be able to refer to an investor as an "angel" without laughing and bursting into flames. These are kind of trivial examples that don't seriously impede communication, but I do think they hint at how we're trying to shape cultures and norms for VR. There's not even agreement about whether to call VR "VR" -- maybe VR is just a subset of "AR" (augmented reality), but really those are both just subsets of "MR" (mixed reality) -- or maybe let's unify under the recent umbrella term "XR" (x-reality?)... terminology and labels and language matters.

And since angels are involved, I guess this is a war in heaven.



One useful flashpoint in the VR culture clash is the debate around 360 video. Basically: film finds them to be a useful stepping stone while they wait for volumetric tech to get better, game developers usually hate them and think they're the absolutely worst, and tech doesn't care either way as long as they get to profit from it.

The common critique against 360 video, as inherited from video games, might go like this: VR is best when it's interactive, and "strong interaction" involves a wide expressive range of user input with a wide range of meaningful novel responses. By this standard, most 360 video is weakly interactive, since it takes only one input (rotational head tracking) and outputs only one possible outcome (a linear video that always unfolds the same way.)

The average 360 video is such a non-interactive that it makes the interactive drama parody scene in Francois Truffaut's Fahrenheit 451 adaptation look like Dwarf Fortress! ... But watch that scene closely: we're supposed to look down on the two TV actors as (gay?) domestic men fretting about silly bourgeois house arrangements. Truffaut's "weak-minded housewife" character, of course, is completely captivated by this bad "interactive" mixed-reality-TV drama about nothing, while the Enlightened Rebellious Intellectual husband rolls his eyes. You can't separate this critique of interactive drama from its lazy reliance on 1960s gender tropes -- the scene relies on them in order to "sting." Similarly, it's impossible to ignore the politics latent in a seemingly intellectual "apolitical" formal critique of anything -- and especially impossible to ignore how gender works in games, tech, or film.

So when I confessed on a stage at Versions, "I kind of hate 360 video," I watched the front row of the audience -- a row of mostly women -- shift in their seats and harden. With that statement, I was basically attacking their careers and artistic practice. It was really convenient that I had all this formal intellectual argumentation for why this VR research by women is suddenly "wrong" for "the medium"... I totally accept how my statement might have came-off as yet another man telling women artists what to do. Ironically, my critique comes from the first wave of VR theory spearheaded by women: Janet Murray's ideas of immersion, Marie Laure-Ryan's analysis of narrative as virtual reality, and Brenda Laurel's research in smart costumes, all rely on theories of performance and active participation. (And like, Brenda Laurel was at Versions too, and she dislikes 360 video too! I wish she'd back me up on this!!)

But anyway. This is why I'm trying to be more constructive than "360 video sucks" -- because even if I think it's ultimately a dead-end for the form, I also realize that it's a domain where a lot of women work -- and honestly, the average 360 short is still more culturally relevant and conceptually stronger than 99% of the current Steam VR catalog.

So my current thinking is that there are currently 3 general camps for VR (games, tech, film) and 3 general dimensions to VR (real-time, real-world. and immersive.) Here's a handy diagram:
(click to enlarge diagram)
Three possible dimensions / qualities to VR:
  • "real-time": live, dynamic, responsive to user input in real-time, cares about "improvising" new content or arrangements for every person or every play
  • "real-world": uses cameras or others sources to pipe-in data from the real-world, cares about social relevance and connection to reality
  • "immersive": cares about integrity / consistency / coherence of the simulated diegetic world, cares a lot about suspension of disbelief
Three camps of VR, with what I see as their voices / strengths / weaknesses:
  • GAMES: real-time + immersive = VR as simulation, world
    "We did all the shader and engine research. You're welcome. Also, kids these days don't use Facebook or watch 360 videos about refugees -- they play Minecraft."
    Did all the initial tech research, has the closest ties to the GPU industry. Focuses on deep formal explorations of interactivity and craft, but that depth also results in very narrow conceptual vision. Still suffering from Stockholm syndrome of being held hostage by conservative gamers; thinks civilization always needs another robot zombie shooting gallery game as long as it has a new hat. Obsessed with solving locomotion, as if mouse-look is intuitive and not exclusionary. Games understand players, but not people.
  • FILM: immersive + real-world = VR as experience, performance
    "What's the point of deep games and markets if people don't give a shit about any of it? Great job with the Kinect and Google Glass, by the way."
    The oldest of the three industries here, has already weathered huge technological shifts before. Culturally relevant and conceptually experimental, more accessible and inclusive than other camps. Wants to make big claims about impact, but doesn't do the homework of thinking rigorously about structuring interaction or participation. Home of the empathy machine cult, which misconstrues montage theory as psychology instead of aesthetics. Film understands viewers, but not people.
  • Tech: real-world + real-time = VR as network, market
    "Depth and cachet are useless if there's no way to use it easily. And who's going to study and manage your audience? Good luck paying the rent with your little experiments."
    Building all the VR manufacturing, network infrastructure, and markets, praying the bet will pay-off in 5-10 years as a new industry that rivals the smartphone era. That long-shot long-view certainly helped tech in the past, but right now, stuff like social VR is often embarrassing, incoherent, and tone-deaf -- which is maybe what happens when you think VR is just a stop-gap until some sudden AR-AI singularity in the near-distant future. Tech understands user data, but not people.
As I think about all this, it strikes me: why isn't there a camp that's just about everything... real-time, real-world, and immersive? Well, no one is stopping you!

These aren't supposed to be rigid definitions that lock you into a warring faction. You don't have to swear your fealty to games or tech to betray film, etc. Rather, I'm just trying to articulate and identify the generalized "values" that I think each field is trying to impose on VR.

Or should I say AR?... or MR... or XR.... ?