Holocaust Survivor Holograms Give History New Depth | KCET
Holocaust Survivor Holograms Give History New Depth
When you meet Pinchas Gutter, he's ready to talk. Seated comfortably in an armchair, the Holocaust survivor makes eye contact and lets you have the first move.
"Hello, how are you?" he asks.
"Good. How are you doing?"
"Not too bad."
While this may sound like an average interaction with someone you've never met, it is far from it. Gutter isn't physically present. In fact, he's probably at home in Toronto. This conversation is with a 3-D recording of Gutter recounting his experiences in the Holocaust: surviving the Warsaw Ghetto, a death march and six Nazi concentration camps.
As the remaining Holocaust survivors like Gutter reach their twilight years, so do the last living testimonies to the genocide. New Dimensions in Testimony, a new project from the University of Southern California's Shoah Foundation and Institute for Creative Technologies answers the question, "What if you could ask a Holocaust survivor anything, even after they've died?"
The Shoah Foundation already had thousands of recorded Holocaust survivor testimonies in its Visual History Archive when it was approached by concept developer Heather Maio of Conscience Display with a fresh approach: What if we made the testimonies interactive?
"Being able to talk to a Holocaust survivor is something that stays with you forever," NDT Project Manager Kia Hays said. "We're getting to a place where that's not going to be a reality anymore."
Currently on exhibition at Illinois Holocaust Museum and Education Center near Chicago, NDT allows visitors to have a virtual conversation with Gutter, who will appear through a faux 3-D presentation technique called a Pepper's ghost, an approach used in Disney's Haunted Mansion ride.
To start a conversation with Gutter, museum visitors hold a mouse's button down and speak into a nearby microphone. Gutter has prepared responses to thousands of questions about his life before, during and after the Holocaust. He even will tell a joke and sing songs, if you ask.
When you ask, "What do you remember from the Holocaust?" Gutter responds with a memory about a police officer who saved his life.
After being assigned to the policeman's work group, the official realized he knew Gutter's family. He was in need of a nurse for his wife, who was very sick, and Gutter fit the bill. Later, on the day many of the people in the camp were shot, the policeman took Gutter into the barracks and cleaned him up, adding his then-deceased wife's lipstick to Gutter's cheeks for a sense of life, and he survived.
Gutter is able to answer questions through ICT's natural language technology that interprets questions and finds the most relevant response from hours of recordings of possible answers. Playback technology enables him to seamlessly answer the question he was asked.
Conversing with Gutter is similar to using Siri on an Apple device. When the system doesn't recognize a question, or if you've not spoken clearly, a polite Gutter says, "Just repeat that." But instead of interacting with a computer, the experience is more human. Gutter's responses are authentic. They're just recorded with future dialogue in mind.
More Technology Stories
His account feels natural, like an in-person conversation. It isn't edited down and there are 15-20 second pauses where Gutter collects his thoughts. While his response to the question above is moving, it goes in a slightly different direction than anticipated, much like many in-person conversations do.
"If your question is not something that the system has seen before, but uses some of the same words, then you are likely to get a related response," ICT Director for Natural Language Research David Traum, said. "Very often these answers are very interesting in their own right, and move the conversation forward, suggesting new things to delve deeper into."
For the best results, it is important to ask questions that are as specific as possible.
The process of transforming Gutter's testimony from the interviews to its current state was lengthy. His testimony is the furthest along of a dozen survivors initially selected to participate in NDT from the Shoah Foundation's Visual History Archive. To narrow down the number of participants, factors like survivors' health, age and level of comfort sharing their stories were considered — aspects that would aid in the rigorous interview cycle.
Interviews were shot for five hours daily over the course of a week on ICT’s Light Stage, near USC's campus in South Los Angeles. Gutter answered thousands of questions from students, staff members and the public from the center of the stage's half-dome that's lined with a green-screen backdrop.
Answering hundreds of questions about what happened during the dark time of the Holocaust is exhausting, but, if given the chance, Hays thinks most participants would participate in the NDT interview process again.
"I think it's a very intense process, but [survivors] really see it as a part of their legacy that they're leaving behind," Hays said.
To ensure the recordings could be used with evolving technology in the future, the project was shot with around 100 cameras and aided with lights and microphones. The survivor's 3-D presence will be able to answer thousands of questions people will most likely ask.
It is tempting to refer to the finished product as a hologram or an avatar, but NDT is more of a time-offset interaction. Unlike Jon Hamm's role as a holographic version of Walter, who provides companionship and memory support to his widow in her last years of life with dementia in "Marjorie Prime," interactions with Gutter are with a previously recorded statement from a real person, not a computer, and there are no computer-generated elements in his responses.
While the data to create a hologram is there from the recorded interviews, for Traum, the best projection technique for NDT will be one that is possible to maintain and ensures survivors will appear naturally.
"We're not tied to one particular 3-D solution," he said. "The idea behind all the different camera recordings and different angles and different types of quality was so that when people figure out the best way to do it, the data's ready and they can apply that data either directly or through image processing techniques to get something reusable for whatever the 3-D of the future turns out to be."
After a pilot presentation of Gutter's testimony in Illinois, more than 95 percent of the visitors surveyed agreed that the technology enhanced their ability to connect with Gutter's story.
"It's not just some new cool gadget. It's something that has some value," Hays said. "All those people are really connecting and engaging with [the survivors]…. There's a sort of relationship that's being built between people when they interact, and I think that that's a very rewarding thing."
Gutter's great eye contact comes from the camera positioning during his interview. Because it feels lifelike, almost like a Skype call, kids often wave at him and wonder why he doesn't wave back. For the older crowd, Gutter's tales of the horrors he experienced often bring tears.
"This project is really preserving more than just the testimonies themselves," Traum said. "It's preserving a lot of the interactive experience that is just asking a survivor your own questions, and being able to take a little bit of control in terms of what you're learning rather than just passively watching."
The Illinois Holocaust museum exhibition marks the first time NDT has come out of the beta testing phase. In October, it will become a part of a pilot program in a New York museum. Before the end of 2017, there will be 16 testimonies total, including a testimony in Mandarin from a survivor of the 1937 Nanjing Massacre.
Eventually, the program hopes to place more testimonies in museums and classrooms to preserve survivors' legacies. Far after Gutter is gone, his 3-D image will be sitting in the armchair, waiting for the next question.
Top Image: Matus Stolov, a Holocaust survivor being filmed | Courtesy of USC Shoah Foundation
In honor of Hispanic Heritage Month, KCET and PBS SoCal will air special programming throughout the month of September and October.
Enter to win a pair of tickets to Gem of the Ocean.
A Q&A will immediately follow the screening with star Reneé Zellweger.
The latest salvo is California’s long-running water wars has the potential to emerge as one of the most important pieces of water regulation in recent years.
- 1 of 202
- next ›
From the typeface of “The Godfather” book cover to the Noguchi table, the influence of Japanese American artists and designers in postwar American art and design is unparalleled. Learn how the World War II incarceration affected their lives and creations.
"Artbound" looks at the dinnerware of Heath Ceramics and a design that has stood the test of time since the company began in the late 1940’s.
Inspired by Oaxacan traditions, Dia de Los Muertos was brought to L.A. in the '70s as a way to enrich and reclaim Chicano identity. It has since grown in proportions and is celebrated around the world.
Gospel music would not be what it is today if not for the impact left by Los Angeles in the late 60’s and early 70’s, a time defined by political movements across the country.
A behind-the-scenes look at the contemporary art world through the eyes of a legendary art dealer and curator, Jeffrey Deitch.
- 1 of 11
- next ›