Model based face reconstruction for animation

Forensic Sciences Research, 2018

This article presents a new numerical method for facial reconstruction. The problem is the following: given a dry skull, reconstruct a virtual face that would help in the identification of the subject. The approach combines classical features as the use of a skulls/faces database and more original aspects: (1) an original shape matching method is used to link the unknown skull to the database templates; (2) the final face is seen as an elastic 3D mask that is deformed and adapted onto the unknown skull. In this method, the skull is considered as a whole surface and not restricted to some anatomical landmarks, allowing a dense description of the skull/face relationship. Also, the approach is fully automated. Various results are presented to show its efficiency.

A major unsolved problem in computer graphics is the construction and animation of realistic human facial models. Traditionally, facial models have been built painstakingly by manual digitization and animated by adhoc parametrically controlled facial mesh deformations or kinematic approximation of muscle actions. Fortunately, animators are now able to digitize facial geometries through the use of scanning range sensors and animate them through the dynamic simulation of facial tissues and muscles. However, these techniques require considerable user input to construct facial models of individuals suitable for animation. Realistic facial animation is achieved through geometric and image manipulations. Geometric deformations usually account for the shape and deformations unique to the physiology and expressions of a person. Image manipulations model the reflectance properties of the facial skin and hair to achieve smallscale detail that is difficult to model by geometric manipulation alone.

A major unsolved problem in computer graphics is the construction and animation of realistic human facial models. Traditionally, facial models have been built painstakingly by manual digitization and animated by adhoc parametrically controlled facial mesh deformations or kinematic approximation of muscle actions. Fortunately, animators are now able to digitize facial geometries through the use of scanning range sensors and animate them through the dynamic simulation of facial tissues and muscles. However, these techniques require considerable user input to construct facial models of individuals suitable for animation. Realistic facial animation is achieved through geometric and image manipulations. Geometric deformations usually account for the shape and deformations unique to the physiology and expressions of a person. Image manipulations model the reflectance properties of the facial skin and hair to achieve smallscale detail that is difficult to model by geometric manipulation alone.

Real-Time Imaging, 2001

Three-dimensional human head modeling is useful in video-conferencing or other virtual reality applications. However, manual construction of 3D models using CAD tools is often expensive and time-consuming. Here we present a robust and e cient method for the construction of a 3D human head model from perspective images viewing from di erent angles. In our system, A generic head model is rst used, then three images of the head are r equired to adjust the deformable contours on the generic mod e l t o m a k e i t more closer to the target head. Our contributions are a s follows. Our system uses perspective images that are more r ealistic than orthographic projection approximation used in earlier works. Also for shaping and positioning face organs, we present a method for estimating the camera f o cal length and the 3D coordinates of facial landmarks when the camera transformation is known. We also provide an alternative for the 3D coordinates estimation using epipolar geometry when the extrinsic parameters are absent. Our experiments demonstrated that our approach produces good and realistic results.

The Journal of …, 2001

Generating realistic 3D human face models and facial animations has been a persistent challenge in computer graphics. We have developed a system that constructs textured 3D face models from videos with minimal user interaction. Our system takes images and video sequences of a face with an ordinary video camera. After five manual clicks on two images to tell the system where the eye corners, nose top and mouth corners are, the system automatically generates a realistic looking 3D human head model and the constructed model can be animated immediately. A user, with a PC and an ordinary camera, can use our system to generate his/her face model in a few minutes.

2001

This paper proposes a camera-based real-time system for building a three dimensional (3D) human head model. The proposed system is first trained in a semiautomatic way to locate the user's facial area and is then used to build a 3D model based on the front and profile views of the user's face. This is achieved by directing the user to position his or her face and profile in a highlighted area, which is used to train a neural network to distinguish the background from the face. With a blink from the user, the system is then capable of accurately locating a set of characteristic feature points on the front and profile views of the face, which are used for the adaptation of a generic 3D face model. This adaptation procedure is initialized with a rigid transformation of the model aiming to minimize the distances of the 3D model feature nodes from the calculated 3D coordinates of the 2D feature points. Then, a nonrigid transformation ensures that the feature nodes are displaced optimally close to their exact calculated positions, dragging their neighbors in a way that deforms the facial model in a natural looking manner. A male hair model is created using a 3D ellipsoid, which is truncated and merged with the adapted face model. A cylindrical texture map is finally built from the two image views covering the whole area of the head by exploiting the inherent face symmetry. The final result is a complete, textured model of a specific person's head. c 2001 Elsevier Science (USA)

2015

Facial animation done by performance capture techniques is a subject of growing importance while the quality of graphics and animation in entertainment industry is constantly improving. Most realistic results in facial animation can be obtained using realistic acquisition of human face by various three-dimensional scanning devices. Such registration, however, is prone to errors related to specifity of scanned area, therefore some preprocessing is needed so the obtained model could be used. The aim of this paper is to present typical, face-specific issues as well as solutions related to preprocessing of mesh constructed by scanning techniques. Mesh traversing is applied to reduce number of noisy data related to light dispersion and reflection. Non-manifold edges and vertices are corrected on basis of specifity of studied area, strips of noisy triangles typical for hair are removed and holes typical for chin-neck part of model are filled. Resulting mesh represents single, continuous surface without non-manifold edges or vertices and with hair-related noise removed. Although model after preprocessing is ready for animation, future study to minimize data not related to facial expressions might be needed.

Programming and Computer Software, 2004

In this paper, a survey is given of the approaches, methods, and algorithms used for creating personalized three-dimensional models of human's head from photographs or video. All stages of the model construction are considered in detail. These stages include image marking, camera registering, geometrical model adaptation, texture formation, and modeling of additional elements, such as eyes and hair. Some technologies of the animation of the models obtained, including that based on the MPEG-4 standard, are analyzed, and examples of the applications that use these technologies are given. In conclusion, some prospects of the developments in this field of the computer graphics are discussed.

2010 23rd SIBGRAPI Conference on Graphics, Patterns and Images, 2010

In this paper we introduce a Facial Animation system using real three-dimensional models of people, acquired by a 3D scanner. We consider a dataset composed by models displaying different facial expressions and a linear interpolation technique is used to produce a smooth transition between them. One-to-one correspondences between the meshes of each facial expression are required in order to apply the interpolation process. Instead of focusing in the computation of dense correspondence, some points are selected and a triangulation is defined, being refined by consecutive subdivisions, that compute the matchings of intermediate points. We are able to animate any model of the dataset, given its texture information for the neutral face and the geometry information for all the expressions along with the neutral face. This is made by computing matrices with the variations of every vertex when changing from the neutral face to the other expressions. The knowledge of the matrices obtained in this process makes it possible to animate other models given only the texture and geometry information of the neutral face. Furthermore, the system uses 3D reconstructed models, being capable of generating a three-dimensional facial animation from a single 2D image of a person. Also, as an extension of the system, we use artificial models that contain expressions of visemes, that are not part of the expressions of the dataset, and their displacements are applied to the real models. This allows these models to be given as input to a speech synthesis application in which the face is able to speak phrases typed by the user. Finally, we generate an average face and increase the displacements between a subject from the dataset and the average face, creating, automatically, a caricature of the subject.

Symmetry

Facial Animation is a serious and ongoing challenge for the Computer Graphic industry. Because diverse and complex emotions need to be expressed by different facial deformation and animation, copying facial deformations from existing character to another is widely needed in both industry and academia, to reduce time-consuming and repetitive manual work of modeling to create the 3D shape sequences for every new character. But transfer of realistic facial animations between two 3D models is limited and inconvenient for general use. Modern deformation transfer methods require correspondences mapping, in most cases, which are tedious to get. In this paper, we present a fast and automatic approach to transfer the deformations of the facial mesh models by obtaining the 3D point-wise correspondences in the automatic manner. The key idea is that we could estimate the correspondences with different facial meshes using the robust facial landmark detection method by projecting the 3D model to ...

2005

The creation of personalised 3D characters has evolved to provide a high degree of realism in both appearance and animation. Further to the creation of generic characters the capabilities exist to create a personalised character from images of an individual. This provides the possibility of immersing an individual into a virtual world. Feature detection, particularly on the face, can be used to greatly enhance the realism of the model. To address this innovative contour based templates are used to extract an individual from four orthogonal views providing localisation of the face. Then adaptive facial feature extraction from multiple views is used to enhance the realism of the model.

2011

In the past decade, 3D statistical face model (3D Morphable Model) has received much attention by both the commercial and public sectors. It can be used for face modeling for photo-realistic personalized 3D avatars and for the application 2D face recognition technique in biometrics. This thesis describes how to achieve an automatic 3D face reconstruction system that could be helpful for building photo-realistic personalized 3D avatars and for 2D face recognition with pose variability. The first systems we propose Combined Active Shape Model for 2D frontal facial landmark location and its application in 2D frontal face recognition in degraded condition. The second proposal is 3D Active Shape Model (3D-ASM) algorithm which is presented to automatically locate facial landmarks from different views. The third contribution is to use biometric data (2D images and 3D scan ground truth) for quantitatively evaluating the 3D face reconstruction. Finally, we address the issue of automatic 2D f...

Signal Processing: Image Communication, 2006

In this paper, we present an image-based method for the tracking and rendering of faces. We use the algorithm in an immersive video conferencing system where multiple participants are placed in a common virtual room. This requires viewpoint modification of dynamic objects. Since hair and uncovered areas are difficult to model by pure 3-D geometry-based warping, we add image-based rendering techniques to the system. By interpolating novel views from a 3-D image volume, natural looking results can be achieved. The image-based component is embedded into a geometry-based approach in order to limit the number of images that have to be stored initially for interpolation. Also temporally changing facial features are warped using the approximate geometry information. Both geometry and image cube data are jointly exploited in facial expression analysis and synthesis.

Iraqi Journal for Electrical and Electronic Engineering, 2021

Animating human face presents interesting challenges because of its familiarity as the face is the part utilized to recognize individuals. This paper reviewed the approaches used in facial modeling and animation and described their strengths and weaknesses. Realistic face animation of computer graphic models of human faces can be hard to achieve as a result of the many details that should be approximated in producing realistic facial expressions. Many methods have been researched to create more and more accurate animations that can efficiently represent human faces. We described the techniques that have been utilized to produce realistic facial animation. In this survey, we roughly categorized the facial modeling and animation approach into the following classes: blendshape or shape interpolation, parameterizations, facial action coding system-based approaches, moving pictures experts group-4 facial animation, physics-based muscle modeling, performance driven facial animation, visua...

As the 3D human face reconstruction is becoming very popular in recent times, it attracts many researchers. Construction of 3D human face using only two orthogonal images and twelve landmark features are the main context of the proposed approach. For3D object modeling, the Open Graphics Library (OpenGL) is used as the platform through which modeling, modification and rendering is performed on the morphable model. The proposed approach trails through semi-automatic identification of the facial landmark features, calculation of the 3D coordinates of human face, morphable model construction in OpenGL, reshaping of the morphable model and rendering of the morphable model. The facial landmark identification is semiautomatic method, as the module requires the manual interaction for marking the facial landmarks on the image. The reshaping of morphable model is required as the morphable model does not fit to the actual face in most of the cases. The morphable model is reshaped by calculating root mean square (RMS) error of face coordinates. The rendering process does not require a wide screen image because the approach performs rendering using input front face image and side face image as textures. Applications of this research help to overcome the disputes of fields like crime detection, 3D game characterization, the ornaments exhibition and in the area of medical technology like plastic surgery.