The group is considering Apple has introduced ReALM (Reference Resolution as Language Modeling), a new AI model that aims to dramatically improve voice assistants’ understanding and response to commands.
The company says this large language model can decipher “ambiguous references” in on-screen objects, as well as provide understanding of spoken and background context. ReALM can lead to more efficient and natural interactions with devices.
Ambiguous references refer to users’ pronouns and similar elements of speech during a conversation.
For voice assistants, this historic opportunity has been a major challenge. Apple’s ReALM system attempts to turn a complex process into a language modeling problem.
ReALM reconstructs the visual layout of a screen using textual representations, including analyzing screen objects and their locations to create textual form, reflecting screen traditions and variations. Apple researchers have found that this strategy and careful design of language models significantly outperform traditional methods, including those used in ChatGPT.
Apple’s new AI system could allow you to interact more effectively with digital assistants based on what’s currently displayed on the screen, meaning you won’t have to say anything to clarify. This could make digital assistants more effective in a variety of settings. For example, helping drivers navigate car media systems while driving or assisting users with disabilities by providing easier and more accurate means of indirect interaction.
Source: Iphones RU
I am a professional journalist and content creator with extensive experience writing for news websites. I currently work as an author at Gadget Onus, where I specialize in covering hot news topics. My written pieces have been published on some of the biggest media outlets around the world, including The Guardian and BBC News.
