Face2Face: Real-time Face Capture and Reenactment of RGB Videos

From Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt and Matthias Nießner:

We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination... (full paper)

 

Comments (0)

This post does not have any comments. Be the first to leave a comment below.


Post A Comment

You must be logged in before you can post a comment. Login now.

Featured Product

TUV Rheinland Robot Integrator Program

TUV Rheinland Robot Integrator Program

The industry's first comprehensive Robot Integrator Program saves robot integrators significant time and cost investments by allowing them to mark each cell compliant with ANSI/RIA R15.06 with the TUV Rheinland Mark. As opposed to a traditional certification or an on-site field labeling, TÜV Rheinland's Robot Integrator Program certifies the knowledge and skill-set of robot integrators in addition to testing robotic cells and processes against ANSI/RIA R15.06. This reduces the need for frequent onsite or off site testing and allows manufacturers to apply a single TÜV Rheinland label to multiple cells. The Robot Integrator Program individually assesses a robot integrator's understanding of the ANSI/RIA R15.06 standard along with the ability to consistently produce compliant robot cells. Following the requirements and procedures of the new program will enable robot integrators to produce individually compliant robotic cells under one serialized TÜV Rheinland Mark, which meets the national electric code and allows acceptance by Authorities Having Jurisdiction (AHJ) and end users.