An artwork has a point of view, a perspective informed by the artist who created it.
So does a virtual world. But when you’re immersed in someone else’s vision with a headset on your face, the designer’s intentions, attitudes and biases can be much less obvious than when you’re looking at a painting or sculpture from a distance.
That’s the kind of insight Yale art students and faculty bring to Blended Reality, a cross-curricular applied research program through which they create interactive experiences using virtual reality, augmented reality and 3D printing tools. Yale is one of about 20 colleges participating in the HP/Educause Campus of the Future project investigating the use of this technology in higher education.
Interdisciplinary student and professor teams at Yale have developed projects that include using motion capture and artificial intelligence to generate dance choreography, converting museum exhibits into detailed digital replicas, and making an app that uses augmented reality to simulate injuries on the mannequins medical students use for training.
The perspectives and skills of art and humanities students have been critical to the success of these efforts, says Justin Berry, faculty member at the Yale Center for Collaborative Arts and Media and principal investigator for the HP Blended Reality grant.
If you leave VR and AR in the hands of technologists alone, “you lose something really important. You’re trying to answer questions and you’re not asking them,” Berry says. “One of the things I’m interested in is getting people to ask questions so that we can get better answers.”
Art and humanities students have made observations that Berry calls simple but powerful. Their feedback, both positive and negative, can inform how educators approach the possibility of introducing blended reality tools into their classrooms.
The first step? Anticipate a range of reactions.
“If you have this impression that this is just this delightful thing, that you’re going to put it in the room and everyone is going to jump on it, you might be surprised that half the room might not be willing to put it on their face,” Berry says. “I think the dynamic it creates in the classroom is complicated. How these emerging technologies are going to translate or transform into a classroom experience, we have to think these things through from top to bottom so that they’re meaningful and effective.”
For example, Yale art student Valentina Zamfirescu was initially skeptical of incorporating VR into her work.
“She said virtual reality is an inherently violent medium,” Berry says. “You’re sticking a device on your face, and that device is shoving input into your eyes and ears so you can only see and hear something that someone else has designed. That’s someone you don’t know, you don’t have direct access to, and you don’t know what they did or why they did it. You’re basically submitting yourself to someone else’s vision of reality.”
Zamfirescu’s critique might apply, for example, to the immersive sound and video experiences created by Yale language instructor Dinny Risri Aletheiani to show her students what it’s like to visit a traditional early morning market or encounter afternoon street performers in Indonesia. Aletheiani selected and edited her footage to present specific scenarios and convey contexts in which students might find themselves speaking and hearing Indonesian.
Students have had strong reactions to the experience of slipping into the world their professor created.
“When they are in the busyness of the street and the market, they feel pressure. They actually have a bodily reaction,” Aletheiani says. “It makes them think differently and feel differently.”
Not every student arrives in Aletheiani’s class familiar and comfortable with using a VR headset, she says. Indeed, the tools most commonly used to create blended reality experiences can be divisive. That’s one of the motivations behind “Clamshell Controller,” a project that aims to design a new kind of device that offers different ways to participate in blended reality settings.
“If you look at almost all current controllers, they are gun-based. You pull a trigger to interact,” says Lance Chantiles-Wertz, a member of the Clamshell research team who recently graduated with degrees in film and mechanical engineering. “The idea is to create a platform to allow new kinds of experiences.”
To get a wide range of perspectives on how students would like to interact with virtual worlds, the Clamshell researchers held open design sessions during which they offered participants craft supplies like clay and wire to model possible controllers.
Working with people of varied backgrounds has yielded practical advice about matters such as the level of user difficulty appropriate for non-technical experts, Chantiles-Wertz says. It has also pushed his team to stretch their imaginations when presented with a new idea.
“There’s no reason to say no to it at all,” Chantiles-Wertz says. “Would it be possible? Why not try it?”
