WASHINGTON: Microsoft wants to make its AI even more of a personal assistant for users, and is rolling out new abilities on its latest Copilot version to read out a summary of the latest news, maintain a conversation and even see what you see on a screen.
Announced in October, one major new feature is called Copilot Vision, and it allows the AI to answer questions about what you are currently seeing on a website.
At launch, this only works on certain pages, Microsoft says. The data will not be stored permanently and will not be used to train the AI models.
The company was recently criticised for opening users up to potential security problems with a feature for PCs that takes constant screenshots so that users can quickly find things again.
The Copilot improvements are Microsoft's next step in an AI race with Google, Apple, Facebook group Meta and several start-ups. The Windows company secured early access to technology from ChatGPT developer OpenAI with a billion-dollar pact and can build on this.
At the same time, Apple and Google are building new AI features directly into their two smartphone platforms. While Copilot also runs on mobile phones in an app, Microsoft can make deeper software integrations on its own Windows platform.
Prominent AI developer Mustafa Suleyman, who joined Microsoft this year from start-up Inflection AI, outlined the vision of an "AI companion for everyone" at the launch of the new features, which could automatically take notes at a doctor's appointment, for example, and also help with planning a child's birthday party. However, the path to this vision, in which software can also advise people on complex life decisions, will take years, he said. – dpa
