Yao Feng
Max Planck Insistitute and ETH Zurich
yao.feng@tuebingen.mpg.de
Bio
Yao Feng is a PhD student supervised by Professor Michael J. Black at the Max Planck Institute for Intelligent Systems and Professor Marc Pollefeys at ETH Zurich. She is also a research scientist at Meshcapade. Previously, Yao completed her Master’s at Shanghai Jiao Tong University and her Bachelor’s degree at Chongqing University of Posts and Telecommunications. She was selected as a WiGRAPH Rising Star. Her research interests span the intersection of computer vision, computer graphics, and machine learning, with a particular focus on advancing the development of digital humans.
Areas of Research
- Computer Graphics and Vision
Learning Digital Humans from Vision and Language
The study of realistic digital humans has become increasingly important in computer vision, computer graphics, and machine learning due to their role in the metaverse and various applications like virtual presence, digital fashion, and healthcare. However, creating 3D humans presents challenges due to data scarcity. To address this, my work explores how machines can learn about digital humans by processing abundant and scalable visual and linguistic data. I developed a framework that generates detailed 3D faces from single images, achieving state-of-the-art accuracy without the need for paired 3D supervision. This model effectively disentangles identity and expression, enhancing facial animation. Additionally, I introduced a novel hybrid 3D representation to capture the body, clothing, face, and hair from videos, enabling easy transfer of features like hairstyles and outfits between avatars. Further, I leveraged text-visual models to create realistic 3D faces from text descriptions and utilized Large Language Models (LLMs) to interpret and generate 3D human poses from images and text. My approach significantly enhances the scalability and controllability of digital human creation, making it accessible and customizable across various fields. Looking forward, I see digital humans as beneficial to multiple research fields and plan to explore further areas such as robotics and biomechanics to enhance digital human research.