Google introduces new artificial intelligence that creates audio for videos

DeepMind, Google’s arm focused on artificial intelligence (AI), introduced a new tool that can create audio for videos and text descriptions. This feature interprets the user’s command and combines it with the context of the images to create the desired sound.

The still unnamed solution is a “video to audio” tool (“V2A” in English). The platform leverages the textual and visual interpretation capabilities of DeepMind models to produce audio specific to a variety of contexts, including music and dialogue.

According to Google, the tool is capable of offering various possibilities for recordings, such as music, movies, speech scenes and ambient sound.

In its statement, the company explained how the technology works. There is still a combination of video and text encoders that are responsible for producing the compressed audio, along with a distribution model that creates the final audio once decoded.

As with image generators, DeepMind’s AI does not require the user to be meticulous in text description; but the more detail the better. The company is talking about this the tool can create an “unlimited” amount of sound effects for videos.

The result still looks like an AI-generated production. The audio is of high quality and DeepMind is working to improve lip-syncing with the dialogue.

Although the tool is in the early stages of development and not publicly available, the work represents a significant advance in the preparation of AI and more robust tools.

Additionally, Google’s solution is slightly more robust than solutions available on competing platforms, such as ElevenLabs’ audio generator.

DeepMind’s video-to-audio tool will now undergo an in-depth security evaluation and trial. There are no predictions regarding the exact launch of the new product.