This Week in AI: Desktop AI Gets Serious

Image: ChatGPT; Prompt: Dan Ackerman

Online products like OpenAI's ChatGPT, Google's Gemini, and Microsoft's Copilot are fun to work with, but many tech companies are betting that on-device artificial intelligence is where the future is. The latest example comes from NVIDIA, which announced it teamed up with French startup Mistral AI to release a new language model for business PCs.

The two companies said they collaborated to create the 12B Mistral NeMo model, which is being released for free on the web. Its context window is 128k, meaning the system can consider much more text and other information when generating a response. That's similar to Microsoft's Phi-3 mini, which itself is much higher than the 8k context of other on-device models like Google's Gemma and Meta's Llama.

NVIDIA and Mistral's joint efforts are another example of how AI technologies are evolving to meet people's needs. Internet-based AIs are capable of producing music, translating multilingual conversations and coding new apps. But companies like Apple have increasingly turned people's attention to on-device AI as a faster and potentially more privacy-protective alternative.

On-device AIs that run on local hardware are still less capable in some ways than their internet-based cousins, in part because many companies have only just begun integrating neural processing units, or NPUs, into modern desktop and laptop computers. Many processors still don't offer the same processing power or capabilities as more expensive chips made by the likes of NVIDIA and used by companies including OpenAI, Microsoft, Apple, Google and Facebook, but they are getting better and nearly all future chips will include discrete NPUs, which we've already seen in 2024 PCs like the Microsoft Surface Laptop. 

OpenAI announces GPT-4o mini

OpenAI's announcements lately have been about internet-based AIs rather than on-device, but even then there are some important developments. In May, OpenAI announced GPT-4o, which promised much faster responses and real-time conversation, among other features. Now, OpenAI has announced another version, called GPT-4o mini, which it says is more cost-efficient, potentially opening up broader uses for AI.

OpenAI says GPT-4o mini has a context window of 128k, and has knowledge up to October, 2023.

"We’re committed to continuing this trajectory of driving down costs while enhancing model capabilities," the company said in its announcement blog post. 

Microsoft Designer app officially launches

Microsoft's AI-powered designer app launched for free on the web and through Windows, Apple iOS and Android this week, allowing users to create custom images, greeting cards, stickers, invitations and more. The app also includes image editing tools that can create custom picture frames, swap out the background of an image or change its style. 

Though Microsoft's designer app is free, the company includes 15 daily "boosts" with each account, allowing users to perform AI actions quicker. Microsoft increases that number to 100 daily "boosts" for customers who pay for a Copilot Pro subscription at $20 per month. That subscription also adds additional AI technologies to Microsoft's Office apps, including Word and PowerPoint.

Google launches Vids tests

Google's testing a video-creation app called Vids that's designed to help you create presentation videos out of files from Google Drive. The way it works is you describe what presentation you want, then it will create a suggested outline based on whatever materials you select to include. Afterward, you pick a style, and it automatically inserts stock images, videos, music, and even a suggested voiceover. 

"Unleash your creativity with our royalty-free stock content library and land your message with Vids recording studio," Google said in a post announcing Vids availability.

Vids is currently part of the Workspace Labs, which is serving as a testing ground for AI products as part of Google's paid subscription service, typically through your business or school. If you want to try it out you might have to wait a while. Google says, "We’re currently testing this new application with a select group of trusted testers," and that "We’ll iterate and refine these experiences through Workspace Labs before making them available to select Gemini for Google Workspace plans."