Nvidia’s new generative AI creates music, voice, and voice with every text

Nvidia has introduced a new generative artificial intelligence (AI) developed by the company for audio processing. Platform name Essential Generative Audio Transformer Opus 1, or Fugatto in the abbreviation created by the brand.

Fugatto’s area of ​​expertise modify and create sounds, music or voices in general It is based on two input formats: text commands or other audio files.

The idea, according to Nvidia, is for AI to be used by creative industry professionals in areas such as advertising, game design and music. In this case, it is seen as ideal.”Create a prototype or edit a song idea by trying different styles” or adding effects to a ready-made file.

Outside of Fugatto There are no plans for a commercial release Nvidia is also working on artificial intelligence features to speed up the training of humanoid robots and make game NPCs “extremely realistic.”

What can Fugatto do?

Fugatto’s secret lies in a feature called MouldableARTIt combines multimodal actions (i.e. multiple formats) in content creation that were previously used by AI tools only during training.

According to the video published by Nvidia, it is possible to ask artificial intelligence to beat a certain rhythm using certain instruments. He can do it too Separate vocals from a music fileIt separates even if the recording quality is poor.

If your text prompt is well written, even Want a complete soundscapea soundtrack accompanied by sounds from certain elements. Finally, it is also possible create realistic sounds from scratchindicates emphasis and speaking tone.

Platform according to Nvidia uses 2.5 billion parameters and capable of producing completely new soundscreating combinations not found in other databases or AI.

It was trained with a base of content stored on Nvidia systems running from multiple H100 GPUs. The company does not disclose what sources are used, but possible copyright infringement and unauthorized harvesting from other platforms has already sparked controversy in the industry.