Computer Facial Animation

Contact: Steph Davy, [email protected], +44 (0)793 942 9983/+ 44 Currently, there is no single answer to the many challenges facing forensic facial reconstruction. The process of completing a three-dimensional clay reconstruction can take several days to complete. With the advent of user-friendly computer software and methods, the time taken to produce a facial reconstruction process could potentially be reduced to mere hours. As computer technology progresses and develops, computer generated facial reconstruction techniques will improve. These developments could save both time and money, as well as increasing the reliability of the technique.

"Gracias a los últimos avances de la tecnología, se ha podido abrir el camino a la investigación sobre interfaces sociales y el uso de los llamados agentes virtuales inteligentes. Estos agentes se hacen visibles en la interfaz como actores animados que representan seres humanos.
Estos agentes tienen la finalidad de interactuar con un usuario, para lograr realizar tareas de una forma natural, para esto, el agente debe ser creíble. No es suficiente que un personaje virtual luzca real en apariencia para lograr una credibilidad por parte del usuario.
El personaje debe además comportarse y comunicarse como lo haría el ser vivo que simula.
Los agentes virtuales se comunican de forma similar a como lo hacen los seres humanos, a través de expresiones verbales y no verbales. El interés de este trabajo es el de lograr que el personaje pueda expresarse de forma no verbal de una manera que sea coherente al contexto. La dificultad inherente a esta tarea se da por el hecho de que el ser humano puede
experimentar una gran variedad de emociones, las cuales son representadas por un amplio rango de expresiones, que también están determinadas por la personalidad del sujeto y del contexto. Las expresiones no verbales llevan el mayor peso de la comunicación de emociones y sirven de apoyo en la transmisión de los sentimientos e intenciones. El uso de agentes se ha extendido a una diversidad de áreas, como la terapia, el entretenimiento, la educación y tutoría.
Nuestro trabajo consiste en seleccionar la expresión facial de un agente virtual, que mejor represente una cierta combinación de atributos generados por un modelo comportamental (el estado emocional, la actitud y la intención). Siendo esto una primera aportación a la futura creación de un modelo kinésico que manipule todas las expresiones no verbales que el agente podrá transmitir."

The development of digital three-dimensional characters, inside the field of animation, and the constant demand for compelling technology solutions, allied to a very singular esthetic, has contributed to the success and the affirmation of three-dimensional animation in the entertainment industry. However, all the work that seeks or exploits the digital/3D language becomes 'victim' of limitations due to rendering image sequences, to rising financial and human costs, as well as the influence and difficulty involved in achieving the objectives and deadlines.
Real-time has increasingly assumed a predominant role in the interactive animation industry. With the evolution of technologies, there has been more surching for an appropriate methodology to boost real-time and low budget 3D animations, even with open source software for cost reduction, to enable the discarding and dependence of offline render in 3D animation. Real-time character’s development enables the rise of a new approach, the interactivity in the art of animation, which can introduce a wide range of new applications and consequently contribute to increase interest and curiosity by the audience. However, insertion, implementation and (ab)use of technologies in the field of animation, raises contemporary issues about the role of the animator.
This dissertation seeks to analyze those aspects, supporting to a 3D animation project in real-time, called 'PALCO'.

Facial animation is an important aspect to make  character more life in virtual world. Early stage to make facial  animation  is  facial  rigging. Facial  rigging  is  the  process  of  creating  the  animation  controls  for  a facial  model  and  the  animator’s  interface  to those  controls. Facial  feature  points  which  associated  with
facial  skeleton  makes  process  of  rigging  easy  to  do. Weighting  feature  points  on  the  face  makes  facial
skeleton  used  as  the  basis  of  the  manufacture  of
facial  control  to  manual  or  automatic  animation  with motion data.

Abstrak Animasi wajah memegang peranan penting dalam menghadirkan keberadaan karakter virtual 3 dimensi. Proses pembentukan animasi ekspresi wajah memerlukan waktu yang lama bahkan bagi animator yang handal dikarenakan kompleksitas dari wajah manusia itu sendiri. Salah satu teknik pembentukan animasi wajah yang dianggap efektif dari sisi kecepatan produksi adalah teknik penggunaan ulang animasi wajah (retargetting). Teknik ini mengusung penggunaan data facial motion capture yang kemudian di transformasikan melalui transformasi ruang berbasis radial basis function. Karakter virtual 3 dimensi yang dihasilkan memapu melakukan visualisasi ekspresi wajah yang sesuai dengan ekspresi wajah manusia dengan memiliki nilai rata-rata simpangan 0.0034. Kata kunci: animasi, animasi ekspresi wajah, karakter virtual 3 dimensi, facial motion capture, retargeting, radial basis function, game. Abstract Facial animation is an important role in bringing the existence of the 3D virtual character. Process of making facial animation take a long time even for professional animator due the complexity of the human face itself. One of the techniques to make facial animation considered effective in terms of speed of production is the using repeatedly facial animation (retargetting). This technique carries the facial motion capture data usage which is then transformed through the transformation of the space-based radial basis function. Generated 3D virtual character can visualizing facial expression corresponding to the expression of a human face by having an average deviation value of 0.0034.

La generación de vi-sílabas como técnica de sincronización de labios en personajes virtuales permitirá aumentar la credibilidad de la animación facial. En esta investigación, se presenta la creación de vi-sílabas de frases escritas en el idioma español aplicando fundamentos lingüísticos para seleccionar uno ó dos visemas y asociarlo a una sílaba (vi-sílaba).

Este trabajo presenta una propuesta para la creación de vi-sílabas de frases escritas en el idioma español con el fin de aumentar la credibilidad de la sincronización labial en personajes
virtuales. Lo que se busca es asociar un solo visema a una silaba (vi-silaba) aplicando reglas lingüísticas a través de un proceso de presilabeo, transcripción fonológica, punto de articulación y de silabeo.

An automatic model is presented for animating gaze shifts of virtual characters towards target locations in a virtual environment. Two connected components are described: an eye-head controller and a blinking controller. The gaze control model is based on results from neuroscience, and dictates the contributions of the eyes and head to a gaze shift according to an individual's head movement propensity; that is, their tendency to recruit their head when making gaze motions under different conditions. The blink controller simulates gaze-evoked blinking, a specific category of behaviours that accompany gaze shifts. The probability of occurrence of such blinks, and their amplitude, is related to the gaze shift. These factors may allow for more believable gaze behaviour, providing variety in animation. The model is of particular significance to serious game environments, where the quality of a character's gaze behaviour may affect engagement, immersion and learning outcomes.

An intelligent virtual agent is an intelligent agent with a digital representation and endowed with conversational capabilities; this interactive character exhibits human-like behavior and communicates with humans or virtual interlocutors using verbal and nonverbal modalities such as gesture and speech. Building credible characters requires a multidisciplinary approach, the behaviors that must be reproduced are very complex and must be decoded by their interlocutors. In this paper we introduce a kinesics model in order to select the nonverbal expression of intelligent virtual agents able to express their emotions and purposes using gestures and face expressions. Our model select the nonverbal expressions based on causal analysis, and data mining algorithms applied to perceptual studies in order to identify significant attributes in emotional face expressions.

Rapid developments in three-dimensional (3D) digitised image capture, computer visualisation modelling and animation have begun make inroads into many areas of the forensic sciences, including the rather conservative specialisation of forensic pathology. This has been the result of an interdisciplinary collaboration between forensic pathologists, biological anthropologists and computer scientists. For example ), we have modelled the track of the blade in a sharp force trauma or stabbing incident and been able to exclude certain body postures as having occurred at the time of the injury (March et al. 2003). Forensic facial reconstruction serves to illustrate one aspect of the results of these collaborations, but surveying and reconstruction and modelling and animation of accident (Figure 2) or crime scenes (Figure 3) are other fields in which 3D computerised methods are gradually being adopted (Noond et al. 2002). Again, visualisation can be used as an investigative tool-in comparing scenarios based on conflicting witness statements for example.

Facial soft tissue thicknesses are mainly used in facial reconstruction for the purpose of forensic identification. Several studies of facial soft tissue thickness (FSTT) in subadults have been published from different ancestral backgrounds such as Caucasoid, African-American, Hispanics and Japanese subjects. However, there is no existing available FSTT data for Turkish subadults. This study reports FSTT data in Turkish subadults of both sexes, aged between 3 and 18. FSTTs were measured at
31 anatomical points on the faces of 450 individuals. A table of mean FSTTs was developed for boys and girls divided into three age groups (3–8; 9–13; 14–18). Differences between boy and girl measurements in three age groups were compared using Student’s t-test. Significant differences were observed at some landmarks in each age group. In order to observe differences among age groups, an ANOVA (one-way analyses of variance) followed by a LSD (least significant difference) Post Hoc test was applied to the variances. The boys showed a general increase in tissue thickness with an increase at all the midline and cheek points. The girls showed increased tissue thickness with age at all points except supraglabella, upper lip, frontal eminence, lateral nasal, sub-orbital and supraglenoid.

The making of virtual human character to be realistic and credible in real time automated dialog animation system is necessary. This kind of animation carries importance elements for many applications such as
games
, virtual agents and movie animations. It is also considered important for applications which require interaction between human and computer. However, for this purpose, it is compulsory that the machine should have sufficient intelligence for recognizing and synthesizing human voices. As one of the most vital interaction method between human and machine, speech has recently received significant attention, especially in avatar research innovation. One of the challenges is to create precise lip movements of the avatar and synchronize it with a recorded audio. This paper specifically introduces the innovative concept of multimodal dialog systems of the virtual character and focuses the output part of such systems. More specifically, its focus is on behavior planning and developing the data control languages (DCL).

Animasi wajah merupakan aspek penting di lingkungan virtual 3D yang menghadirkan keberadaan karakter, baik manusia maupun yang menyerupainya. Animasi wajah biasa digunakan pada game, perangkat lunak interaktif, dan film. Problematika muncul saat proses penggandaan animasi dari satu model ke model yang lainnya dikarenakan kompleksitas proses pembentukan animasi ekspresi wajah. Pendekatan featurepoint cluster memungkinkan terjadinya proses penggandaan animasi ekspresi wajah yang berdasarkan cluster area gerak wajah. Metode yang digunakan dengan membagi wajah dalam beberapa area cluster gerak dan melakukan transfer animasi dari data facial motion capture menggunakan metode retargeting. Proses penggandaan animasi ini memberikan hasil berupa pergerakan animasi yang sesuai tanpa perlu adanya bantuan percakapan atau dialog.

Este trabajo presenta una propuesta para la creación de vi-sílabas de frases escritas en el idioma español con el fin de aumentar la credibilidad de la sincronización labial en personajes
virtuales. Lo que se busca es asociar un solo visema a una silaba (vi-silaba) a través de un proceso de transcripción fonológica y de silabeo aplicando reglas lingüísticas.

Various approaches for face reconstruction on a skeletonized skull of an unknown individual have been presented since the 19th century. Recently, tremendous advances in information technology lead to significant further developments and refinements of the reconstruction methods. In combination with modern imaging technology, 3D simulation methods have been applied that create facial expressions. An international conference series on reconstruction of soft facial parts (RSFP) has been initiated to inform police officers and international scientists about this complex scientific branch and to simultaneously encourage future international cooperative networks of the involved disciplines. This book includes contributions of the multi-faceted status quo of scientific developments in reconstruction of soft facial parts.

In the domain of image processing, face recognition is one of the most well-known research field. When humans have very similar biometric properties, such as identical twins, the face recognition system is considered as a challengeable problem. In this paper, the AdaBoost method is utilized to detect the facial area of input image. After that the facial area is divided into some local regions. Finally, new efficient facial-based identical twins feature extractor based on the geometric moment is applied into local regions of face image. The used feature extractor is Pseudo-Zernike Moment (PZM) which is employed inside the local regions of facial area of identical twins images. To evaluate the proposed method, two datasets, Twins Days Festival and Iranian Twin Society, are collected where the datasets includes scaled and rotated facial images of identical twins in different illuminations. The experimental results demonstrates the ability of proposed method to recognize a pair of identical twins in different situations such as rotation, scaling and changing illumination

ABSTRAK Pendekatan tradisional animasi ekspresi wajah sangat tergantung pada animator dalam pembuatan gerakan kunci dan rangkaian gerakan ekspresi wajah. Problematika yang sering dijumpai adalah penggunaan kerangka dan gerakan wajah yang sama untuk model yang berbeda membutuhkan waktu yang lama dikarenakan kompleksitas ekspresi wajah manusia. Pendekatan simulasi kulit wajah dan otot pada praktiknya masih memerlukan intervensi animator untuk pengaturan kulit wajah terhadap tulang/tengkorak kepala dan konfigurasi sambungan otot gerak di wajah. Hal ini menyebabkan produksi animasi wajah untuk satu wajah tidak dapat digunakan ulang secara langsung untuk wajah lainnya karena kekhususannya tersebut. Oleh karena itu, proses pengamatan perubahan bentuk ekspresi wajah dengan adanya area bobot pada model wajah 3D menggunakan pendekatan klaster di titik fitur gerak mempunyai peran penting untuk mengidentifikasi proses penyesuaian bentuk wajah yang berlainan dan variasi pengaruh gerakan pada wajah karakter kartun. Kata kunci: deformasi wajah, klaster titik, karakter kartun ABSTRACT Cartoon Character Face Deformation Based on Motion Feature-Point Cluster. The traditional approach animated facial expression is highly dependent on animator to create key of movement and continuity the motion of facial expressions. The problems frequently encountered is the use of the skeleton and the same facial movements for different models takes a long time because of the complexity on human facial expressions. Simulation approach to facial skin and muscles in practice still requires intervention animators to control the facial skin to bone/skull and connection configuration in facial muscle movement. This leads to the production of facial animation for one face can't be reused directly to the other face model because of their specialization. Therefore, the observation of deformation facial expressions with weights area on a 3D face model using motion feature-point cluster approach have an important role to identify the adjustment process on different facial shapes, and variations of movement on cartoon character face.

This work presents the application of linguistic rules for the creation of vi-syllables of phrases written in the Spanish language in order to increase the believability of the labial synchronization in virtual characters. Especially, we describe the implementation of the phonological transcription and Syllabication as part of Phonemicization's process.

Rigging is the process of setting up a group of controls to operate a 3D model, analogous to the strings of a puppet. It plays a fundamental role in the animation process as it eases the manipulation and editing of expressions, but rigging can be very laborious and cumbersome for an artist. This difficulty arises from the lack of a standard definition of what is a rig and the multitude approaches on how to setup a face. This survey presents a critical review on the fundamentals of rigging, with an outlook of the different techniques, their uses and problems. It describes the main problems that appear when preparing a character for animation. This paper also gives an overview of the role and relationship between the rigger and the animator. Continues with an exhaustive analysis of the published literature and previous work, centered on the facial rigging pipeline. Finally, the survey discusses future directions of facial rigging.

An intelligent virtual agent is an intelligent agent with a digital representation and endowed with conversational capabilities; this interactive character exhibits human-like behavior and communicates with humans or virtual interlocutors using verbal and nonverbal modalities such as gesture and speech. Building credible characters requires a multidisciplinary approach, the behaviors that must be reproduced are very complex and must be decoded by their interlocutors. In this paper we introduce a kinesics model in order to select the nonverbal expression of intelligent virtual agents able to express their emotions and purposes using gestures and face expressions. Our model select the nonverbal expressions based on causal analysis, and data mining algorithms applied to perceptual studies in order to identify significant attributes in emotional face expressions.

Speech is one of the most important interaction methods between the humans. Therefore, most of avatar
researches focus on this area with significant attention. Creating animated speech requires a facial model capable
of representing the myriad shapes the human face expressions during speech. Moreover, a method to produce the
correct shape at the correct time is also in order. One of the main challenges is to create precise lip movements of
the avatar and synchronize it with a recorded audio. This paper proposes a new lip synchronization algorithm for
realistic applications, which can be employed to generate synchronized facial movements among the audio
generated from natural speech or through a text-to-speech engine. This method requires an animator to construct
animations using a canonical set of visemes for all pair wise combination of a reduced phoneme set. These
animations are then stitched together smoothly to construct the final animation.

Current facial reenactment techniques are able to generate results with a high level of photo-realism and temporal consistency. Although the technical possibilities are rapidly progressing, recent techniques focus on achieving fast, visually plausible results. Further perceptual effects caused by altering the original facial expressivity of the recorded individual are disregarded. By investigating the influence of altered facial movements on the perception of expressions we aim to generate not only physically possible but truly believable reenactments. In this paper we perform two experiments using a modified state-of-the-art technique to reenact a video portrait of a person with different expressions gathered from a validated database of motion captured facial expressions to better understand the impact of reenactment.

We propose a fuzzy rule-based system to map representations of the emotional state of an animated agent onto muscle contraction values for the appropriate facial expressions. Our implementation pays special attention to the way in which continuous changes in the intensity of emotions can be displayed smoothly on the graphical face. The rule system we have defined implements the patterns described by psychologists and researchers dealing with facial expressions of humans, including rules for displaying blends of expressions.

An automatic model is presented for animating gaze shifts of virtual characters towards target locations in a virtual environment. Two connected components are described: an eye-head controller and a blinking controller. The gaze control model is based on results from neuroscience, and dictates the contributions of the eyes and head to a gaze shift according to an individual's head movement propensity; that is, their tendency to recruit their head when making gaze motions under different conditions. The blink controller simulates gaze-evoked blinking, a specific category of behaviours that accompany gaze shifts. The probability of occurrence of such blinks, and their amplitude, is related to the gaze shift. The model forms the basis for a number of experiments investigating the impact of blinking, eye-head ratio and direction of head movements on user perception. In addition to other application domains, the findings are of significance to serious games environments, where the perceived quality of a character's gaze may affect engagement, immersion and learning outcomes.

Facial animation is an important aspect to make character more life in virtual world. Early stage to make facial animation is facial rigging. Facial rigging is the process of creating the animation controls for a facial model and the animator's interface to those controls. Facial feature points which associated with facial skeleton makes process of rigging easy to do. Weighting feature points on the face makes facial skeleton used as the basis of the manufacture of facial control to manual or automatic animation with motion data.