Omniverse Audio2Face is an AI-powered application developed by NVIDIA that simplifies the animation of a 3D character to match any voice-over track. It can be used in real-time applications or within traditional character creation workflows. With Audio2Face, users can create expressive facial animation from just an audio source using generative AI.
The application works by using a pre-trained Deep Neural Network to analyze the audio input and drive the 3D vertices of the character mesh in real-time, creating the facial animation. Users also have the option to edit various post-processing parameters to customize the performance of their character. The results generated by Audio2Face are mostly raw outputs with little to no post-processing.
One of the key features of Audio2Face is its ability to process audio in any language, making it accessible to users worldwide. It also offers the option to animate characters live using a microphone, allowing for real-time facial animation. Additionally, Audio2Face allows users to easily retarget the facial animation to any 3D human or human-esque face, whether realistic or stylized, with just a few clicks.
Another notable feature of Audio2Face is its ability to run multiple instances of the application with as many characters in a scene as desired. This enables users to animate multiple characters from the same or different audio tracks, creating dynamic and synchronized performances. Users can also control the level of facial expression on each face and batch output multiple animation files from multiple audio sources.
Audio2Face also provides users with the ability to choose and animate their character’s emotions. The AI network automatically manipulates the face, eyes, mouth, tongue, and head motion to match the selected emotional range and intensity. Alternatively, it can infer emotion directly from the audio clip.
In addition to these features, the latest update to Omniverse Audio2Face includes blendshape conversion and blendweight export options, as well as support for export-import with Blendshapes for Blender and Epic Games Unreal Engine.
Overall, Omniverse Audio2Face is a powerful tool that simplifies the process of animating facial expressions for 3D characters. Its generative AI capabilities, real-time animation, and support for multiple languages make it a versatile tool for game developers, filmmakers, digital assistants, and anyone looking to bring their characters to life.
