Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Face Remapping

214 visualizaciones

Publicado el

  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Face Remapping

  1. 1. Artefact conceives, designs, and develops technology products and services for the 21st century artefactgroup.com Face Remapping: When Words Are Not Enough Daniel C. Robbins – Strategy Director INTRODUCTION A parent tears up in frustration after having spent an hour repeating a simple lesson for her autistic child. A grandma feels her distant grandkids drifting away because they don’t speak the same language. A rape victim is terrified that if people in her village find out who she is, she will be made an outcast. These three, very different tableaus, attest to the significance of human relations. We live, work, and play in relation to each other. While we often think and create on our own, as social creatures, the activities that form and maintain our relationships coalesce around face-to-face contact. Each of these tableaus rests on the ability of people to express themselves not only with speech but also with the subtle non- verbal cues of facial expressions, gaze direction, and posture. Without these cues, our communications become flat, lose their impact, and ultimately fail in their power to connect. Immediacy suffers. When we are at our best, we can infer interest, attention, concern, and general mood from each other’s faces. But what happens when an autistic person can’t easily interpret and respond to facial expressions? What happens when language differences get in the way of emotional connections across gaps in age and distance in a far-flung family? What happens when a person in an extreme situation needs to hide her face and is thus denied the ability to convey the full power of her situation? What happens when real people lose out on the power of nuanced expression and face-to-face communication? THE LIMITS OF EXISTING COMMUNICATION TECHNOLOGIES Current digital communication technologies have allowed us to bridge time, space, and imagination. We can send and receive communications when and where we want. But where we have scaled our ability to communicate, we have also diluted our face-to-face connections. A parent can download a therapeutic app for their autistic child but that app can’t yet present realistic human faces. Real-time translation systems can decipher words from another language but they can’t yet provide a natural connection with the speaker. And current video technologies can shield witnesses but they can’t yet convey the subtleties and emotional impact of their testimony.
  2. 2. Artefact conceives, designs, and develops technology products and services for the 21st century artefactgroup.com PUSHING THE LIMITS: FACE REMAPPING Face Remapping is a new set of technologies that has the potential to bring the human connection back to modern communications. Imagine the ability to perfectly manipulate the facial expression and appearance of anyone whose face is conveyed electronically. From the subtle to the brazen: a frown can be made into a smile, lips can be shown to speak a different language, or a ten-year-old girl can animate visage of the latest pop star. Brand new developments in Face Remapping mean that anyone will be able create these manipulations. Face Remapping brings together progress in face tracking, face model generation, machine learning, and new sensors to create a potentially disruptive technology. A capability that was, until recently, only possible in high-end special effects studies and academic labs is now rapidly evolving to the point at which designers, technologists, and even regulators should take notice. The most exciting academic work includes the papers “Real-time Expression Transfer for Facial Reenactment,” and “What makes Tom Hanks look like Tom Hanks?” Market interest is shown by recent acquisitions: Apple snatched up FaceShift and PrimeSense and Intel’s purchased Omek. The big players are also investing heavily in related technologies, such as face recognition, translation, and even emotion sensing. If we step back, we can see how these targeted investments might be the first steps along the way towards using Face Remapping as a means of putting a more human face on digital agents such as Siri, Google Now, Cortana, and Alexa. Face Remapping can seamlessly modify video of one person’s expressions and lip motions to match that of another person. Illustration by Daniel C. Robbins
  3. 3. Artefact conceives, designs, and develops technology products and services for the 21st century artefactgroup.com THE “ICK” FACTOR Even without peeking at the latest episode of “Black Mirror” or chuckling over a Bruce Sterling polemic, it’s easy to fathom the nefarious side of Face Remapping. It’s no great leap to see that the video production studio in our pockets, otherwise known as a smartphone, can not only snap family photos but will soon be able to spoof a celebrity speech, co-worker gaff, or distort the telling of an unfolding public event. This gives anyone the ability to rewrite transmitted and recorded reality. What is now a harmless fascination with snapshot filters could become a dangerous fascination with undetectable manipulation of video of people’s faces. While a first response might be to call for regulation and technological counter-measures, as an alternative, 21st century design perspectives can elucidate positive ways to embrace Face Remapping. THE POWER OF POSITIVE EXAMPLES A society carves out boundaries through regulation but also through selective purchasing, grass-roots promotion, and unanticipated adaptations. We can decide to be more comfortable with banks using big data to detect suspect credit card transactions than we are with insurance companies charging us more based on lifestyle choices revealed by that same data. Now is the time to put on our intellectual running shoes so, that in parallel with the research labs, corporate investments, and technology refinements we, as a society, can define the positive boundaries for this new technology. 21st century design principles mandate that we dig deep to identify people who currently experience compromised communications. Face Remapping can meaningfully empower these people by giving them better access to natural, face- based interactions. CONCEPTS We will sketch out three design concepts that suggest how Face Remapping can open up richer forms of personalized human interaction for an autistic child, a remote family member, and even a victim in a sensitive court case. Each of these concepts was chosen to help those who are underserved by current technological solutions and where it’s acceptable to modify facial representations. EMOTION COACH FOR AUTISM Children with autism benefit when they are repetitively coached as to how to read and respond to facial expressions and basic interactions. Ideally, it’s a trained professional, caregiver, or parent who brings these learning experiences to life but this kind of human coaching can’t happen as frequently or for as long duration as needed. While there are a bevy of smartphone and tablet autism apps they typically rely
  4. 4. Artefact conceives, designs, and develops technology products and services for the 21st century artefactgroup.com on simplistic cartoon graphics. When video of actual humans is provided, the content sets are limited and fixed. Face Remapping could bring a human touch to coaching applications for autistic children. We propose an application where an autistic child learns how to better read emotions by practicing with demonstration video segments that are remapped onto a wide variety of faces, including people they know. The creator of the app need only would record video segments of a small number of actors. Face Remapping is then used transfer these expressions onto any face, including the faces of people the autistic child routinely encounters: a regular bus driver, other teachers, or even extended family. Augmenting human coaching with a tireless and personalized digital app via Face Remapping, reduces the load on caregivers and frees them up for more intensive and cognitive tasks. Consider what happens in the future when we infuse this class of application with the intelligent systems being developed for digital agents such as Siri and Alexa. Instead of drilling with fixed sequences, a future system would be more interactive and respond back to the expressions and utterances of the child with autism. The pace, style, and content of the presented segments could dynamically adapt to the child’s progress. TRANSLATION FOR SPREAD OUT FAMILIES In our migratory world, staying in touch with far flung family is vitally important. Whether these global movements are initiated by armed conflict, social strife, political turmoil, economic hardships, or even a desire for a better education, it is the children who then lose fluency in the language of their roots. While current systems, such as Skype Translator, can help scattered families stay in touch, the communications are stilted. Attention is divided: we listen to the synthesized voice, we read the sub- titles, all while trying to concentrate on the face of a remote family member. This takes family members out of the moment and the emotional connection is compromised. Face Remapping helps create a wide array of emotion coaching videos for children with autism. Illustration by Kenneth Tay (Artefact)
  5. 5. Artefact conceives, designs, and develops technology products and services for the 21st century artefactgroup.com Face Remapping could bring a human touch to increasingly common remote family communications by adding perfect lip-sync to the translated video from remote family members. The result is a conversation between family members that conveys face, pose, expressions, and eye gaze. Face Remapping would allow natural communications between family members, regardless of where they live or what language they speak. Imagine a future where we combine 360 cameras and other sensors with the family translation systems. Family conversations move beyond the formal. A grandchild takes her grandparent on a rich visual tour of her new life, weaving in her daily wanderings, new friends, and little discoveries, all the while maintaining a natural conversation deepened by subtle facial cues and expressions. PROTECTING IDENTITY Victims of violence often face multiple layers of injustice: the initial crime, reprisals, and even social stigma from their communities. Victims of ethnic and domestic violence, along with their children, can be ostracized and abandoned by their communities if they seek out justice. If the victim attempts to protect her identity via video masking or visual screens she will then have an even harder time conveying the weight of her plight and garnering empathy. Face Remapping adds perfect lip-synch to translated video conversations for families who live far apart and speak different languages. Illustration by Kenneth Tay (Artefact)
  6. 6. Artefact conceives, designs, and develops technology products and services for the 21st century artefactgroup.com Face Remapping could help bring a more human touch to highly charged, constrained, and consequential court testimony. The victim would give her testimony via video that could be real-time or prerecorded. Face Remapping would be used to transfer the nuanced expressions and lip motions of the victim onto a neutral face that matched the general age, race, and gender characteristics of the speaker. The presented face presented could either be that of an actor, composited from multiple representative face components, or entirely synthesized. A corpus of neutral faces, that were appropriate to different locales, could be built up over time to provide a safer way for victims to testify. Face Remapping would enable the victim to convey her full conviction and determination without fear of people in the court or community directly identifying her. In the future Face Remapping could be applied in positive ways to other sensitive situations. Imagine if smartphones, by default, anonymized faces of people in crowds before uploading to social media. Protestors and bystanders alike, would be less vulnerable to scraping of social media for purposes of identification and reprisal. We could have a future where identity via social media was always opt-in – enforced not merely by voluntary policy, but through a technical intervention at the time of media creation. CONCLUSION Face Remapping is a new technology and it is too early to say exactly how it will play out. From the dire uses of TNT to the innocuous rise of the selfie, technologists are not oracles of unintended futures. But with Face Remapping it’s easy to conjure up dystopian misuses. While we implore a vigilance against abuse, such as political and social spoofing, we also apply 21st century design perspectives to identify constructive uses that genuinely bring a more natural and human quality back to our vast array of digital communications. We invite an enthusiasm for empowering scenarios, such as coaching, translating, and protecting. These small design sketches serve to increase our understanding of the positive and negative implications of Face Remapping. Our design examples point the way toward a preferable future where exciting new technologies can be used in ways that make the world a better place. Face Remapping protects a victim’s identity by changing her appearance while still conveying the emotional power of her testimony. Illustration by Kenneth Tay (Artefact)

×