Computer Graphics Laboratory ETH Zurich

ETH
CGL » Research » Digital Humans

Digital Humans

Introduction

Mastering and understanding human faces generally encompasses a variety of different research challenges. We investigate novel approaches to acquire and represent human faces, and to develop algorithms that enable stunning visual effects and even medical applications.

Topics

Geometry Acquisition & Reconstruction

We design multiview face capture systems and develop algorithms that are able to recover the geometry and appearance of human faces with incredible accuracy and realism. Our acquisition technology and pioneering research in this field has been used in over 30 Hollywood movies so far and has led to an Academy Sci-Tech Oscar.


Performance Capture and Facial Animation

By leveraging our face capture technology, we further develop algorithms that can track nonlinear facial deformations during expressive facial performances. In addition to tracking the visible skin, our solutions also enable efficient and accurate tracking of other components of the face like the eyes, jaw, and teeth, for use in highly demanding production pipelines. Our solutions significantly reduce the manual burden of VFX artists and assists them in areas like facial animation, rigging, tracking, retargeting, secondary dynamics etc.


Neural Rendering

A primary drawback of traditional capture systems is their inability to model complex geometry and appearance of non-skin areas like the eyes, hair, clothing etc. In our pursuit of a photorealistic digital human, we leverage recent advances in neural networks and bridge gaps between traditional and neural pipelines in VFX with neural rendering. Our work in this area has significant benefits for data generation, the creation of digital doubles, face recognition and much more.


Deep Face Models

Furthermore, we build holistic models of facial geometry and appearance with neural networks that enables us to succinctly represent the manifold of human faces and allows for easy, semantically meaningful navigation of this subspace of shapes. Our work in this area has applications in sculpting, geometry synthesis and animation for background characters, face reconstruction and tracking, retargeting and much more.


Generative Facial Appearance Model

We work towards machine learning algorithms capable of producing high resolution generative facial appearance models. These models produce geometry and various textures representing different components such as albedo, normals and more. The resulting methods are applicable in numerous areas including medical visualization, film and video game production, communication applications, and augmented- and virtual reality experiences, to name a few.


Digital Humans for Medical Applications

Finally, we leverage our work on human faces and digital humans for various medical applications to improve people's lives. Examples include dental treatment simulation and outcome visualization, as well as digital modelling and treatment planning for craniofacial malformations such as cleft lip and palate. For details please visit our page on medical research.

Publications





© CGL, ETH Zurich