Microsoft Copilot can now talk to you and ‘see’ your PC screen

Microsoft’s Copilot virtual assistant will be able to talk to you and “see” what’s on your computer screen. The news is part of a wide suite of new features announced for the platform this Tuesday (01).

There are three new features in the application: Copilot Daily, loud news summary and weather forecastlike a radio host; Copilot Voice, Function that allows you to chat naturally with the chatbot; and Copilot Vision, Gives AI the ability to see what appears on the screen.

The most notable addition among the additions was Copilot Vision, due to its similarity to Recall, a controversial Windows 11 feature that takes screenshots every 5 seconds and uses artificial intelligence to interpret them. But the tool looks much simpler than the system’s native solution.

Unlike Recall, Copilot Vision can only interpret the content currently visible on the screen.. It responds instantly to the user’s request based on the view, but doesn’t save anything for later, not even for model training.

Fortunately for privacy-conscious users Copilot View is completely optional. In the first stage, it will be ready to be tested at Copilot Laboratories.

Copilot Voice is like an adapted version of ChatGPT’s conversation tool. By speaking out loud, the user can speak naturally to the virtual assistant, ask questions, and even interrupt him.

In this case, four different voice options will be available to engage the chatbot. The tool will initially be available in English only in Australia, New Zealand, the United Kingdom and the United States. However, more regions need to be included in the future.

Copilot’s more detailed answers

Another new feature announced for Copilot is “Think Deeper” (free translation: “Think some more”). When enabled, the feature does the following: Chatbot takes more time to analyze the request and think about the responsethus, more robust and possibly more accurate interactions are possible.

Microsoft emphasizes that “Think Deeper” is still in development and is therefore exclusive for testing at Copilot Labs. The new features announced for Microsoft Copilot will be available this Tuesday (1) in the web version of the assistant and in the Android and iOS applications.