Apple launches open-source artificial intelligence for natural language image editing

Efforts Manzana in area artificial intelligence They are moving forward at a steady pace, despite the fact that many people think otherwise. Researchers from the Cupertino firm have teamed up with the University of California, Santa Barbara (UCSB) to create MGIEa new open source model that allows edit images using natural language.

It’s true that today the web is full of tools that allow you to create images using generative artificial intelligence. However, the efforts of Apple and the experts of the aforementioned university were focused on “guided editing”, taking advantage of multimodal language models large scale (MLLM, for English followers).

According to the developers of this tool, MGIE interprets the image and the order that the user enters and starts editing, even without further context of the material or the request itself. For example, one of the cases presented is a photograph of pizza. Using only the description “make it healthier,” Apple’s AI altered the image to include tomatoes and greens.

“MGIE consists of an MLLM and a diffusion model. MLLM learns to receive brief, expressive instructions and provides explicit visual guidance. The diffusion model is jointly updated and performs image editing with the latent imagination of the intended target through end-to-end skill training. In this way, MGIE benefits from its inherent visual processing and eliminates ambiguous human commands to achieve intelligent editing. […], it’s hard to know what “healthy” means without more context. “Our MGIE can accurately associate “plant ingredients” with pizza and carry out appropriate editing according to human expectations.”

Researchers from Apple and the University of California, Santa Barbara.

Together with MGIE, researchers from Apple and UCSB want to demonstrate that large-scale multimodal language models can help simplify image editing using artificial intelligence. Especially, providing the necessary instructions to obtain the desired results.

Apple’s new AI can edit images in natural language

Those responsible for the project note that human instructions are often too brief for current AI editing techniques to properly understand and process. Thus, they claim that using MLLM for this task “improves control and flexibility” when editing images without the need for region masks or overly complex descriptions.

The examples they present make it easier to understand what the story is all about. In addition to what we already mentioned about pizza, Apple’s A.I. can edit Photoshop style. In one image, a man can be seen in the foreground and a woman behind him, in the distance, sitting in a chair. With the “remove woman in background” command, MGIE alters the photo so that only the subject in the foreground is visible. But it is not limited to the destruction of women; This also shifts the focus to the man’s facial expression.

Experts from UCSB and Apple also managed to make artificial intelligence work. local publications. For example, changing what is displayed on a computer screen in a photo without affecting the rest of the image. And also what he can indicate global optimizationsfor example, increasing brightness or adjusting the clarity of the material, as well as other options.

Since this is a research project, it is not yet known whether Apple plans to include this artificial intelligence in its publicly available software. However, as we indicated at the beginning, it is clear that Cupertino residents are paying more and more attention to this type of technology. Let’s not forget that Apple recently introduced MLX, a tool that allows you to create machine learning models.

If you want to try MGIE, you can do so directly from the trial version in Hugging Face Spaces. Although this is a project Open sourceyou can download the information, code, and pre-trained models from this GitHub repository and try them out for yourself.