Let’s face it, voice actors are typically easier to find and won’t cost you as much, but sometimes, the audio track may need a face. Meet NVIDIA’s Omniverse Audio2Face. Put simply, it can generate an animation of a 3D character to match any voice-over track, whether it be for a video game, movie, real-time digital assistants, or just as an experiment. You can make use of the app for interactive real-time applications or as a traditional facial animation authoring tool. Read more for a video demonstration and additional information.
How does it work? The tool comes pre-loaded with “Digital Mark”, which is essentially a 3D character model that can be animated with an audio track. Getting started is as easy as selecting your audio and then uploading it. This audio track is then fed into a pre-trained Deep Neural Network and the output drives the 3D vertices of your character mesh to create the facial animation in real-time.
Audio2Face lets you retarget to any 3D human or human-esque face, whether realistic or stylized. Watch this test as we retarget from Digital Mark to a Rhino! It’s easy to run multiple instances of Audio2Face with as many characters in a scene as you like – all animated from the same, or different audio tracks,” said NVIDIA.
