Meta explained ImageBind, an open-source artificial intelligence tool that predicts links between data similar to our perception and imagination. Unlike other image generators, ImageBind goes even further by binding text, images, audio, 3D measurements, temperature and motion data.
This new resource opens up possibilities in line with Meta’s ambitions. such as virtual reality, mixed reality, and metaverse.. Imagine a future virtual reality device capable of instant rendering. Full 3D scenes with sound, motion and more.
A practical example of what can be created with ImageBind is a virtual reality environment where the user can explore a busy city. Using image/video data, audio, depth information, and thermal data, ImageBind can combine this information to reproduce weather and environmental conditions, characteristic city sounds, and even the sense of proximity of moving cars.
Game developers can also benefit from streamlining the creation process. Additionally, creators will have the ability to produce realistic soundscapes and immersive videos with motion based solely on text, images or audio.
How does ImageBind work?
The main idea of the project is to combine different types of data into a single multidimensional embedding space, allowing content production based on this interconnection. The six modalities are: text, image/video, and audio, with information captured by sensors that record depth (3D), temperature (infrared radiation), and inertial measurement units (IMUs) responsible for calculating motion and position.
There’s a look on the Meta side for this technology to expand beyond the six existing “sensations” and points to an expansion of possibilities.
“While we have explored six modalities in our current research, we believe that by introducing new modalities that connect as many senses as possible – such as touch, speech, smell, and functional brain MRI (fMRI) signals – we will be able to create more comprehensive, human-centered AI models,” he says. company.
Meta makes the code open so that interested developers can explore the tool.
Source: Tec Mundo
I am a passionate and hardworking journalist with an eye for detail. I specialize in the field of news reporting, and have been writing for Gadget Onus, a renowned online news site, since 2019. As the author of their Hot News section, I’m proud to be at the forefront of today’s headlines and current affairs.
