Nvidia can no longer keep up: Microsoft's AI strategy devours GPUs

As the US magazine The Information reports, Microsoft is now internally restricting access to the server systems used for the new developments based on artificial intelligence in order to keep the capacities free for other types of use. It is said that anyone who works internally on AI technologies at Microsoft must now be put on a waiting list.

Waiting list due to server capacity shortage

Apparently there is a bottleneck in the server hardware behind this, since Microsoft cannot expand the capacities quickly enough. According to other reports, the most important supplier is Nvidia , because the manufacturer supplies graphics cards in bulk, which are installed in Microsoft servers to process the data for the AI ​​systems.

More than 30,000 Nvidia GPUs are currently said to be in use to enable the currently available AI applications from Microsoft. The Redmond company relies on enterprise-grade models such as the Nvidia A100 and H100, which are installed in large numbers. Nevertheless, the capacities should not yet be sufficient to enable perfect availability of the services in the medium term.

According to the report, because Microsoft also wants to present new AI functions for Office and its other productivity applications today, the group is trying to prioritize the use of the limited hardware available. One obviously wants to ensure that the announcements about new AI services, which have received a lot of attention, do not suffer from failures or short availability.

Behind all this is the group's attempt to gain a competitive advantage over the competition with its AI-focused strategy. The Bing bot was launched as part of a multi-billion dollar cooperation with the start-up company OpenAI, which develops the underlying technology in the form of GPT-4, caused a "red alert" at Google. So Google is desperately trying to keep up so as not to lose touch.

Microsoft did not say specifically when the new AI copilot features would debut and what the pricing would be, only saying that “in the months ahead, we’re bringing Copilot to all our productivity apps—Word, Excel, PowerPoint, Outlook, Teams, Viva, Power Platform, and more.”

The company added that it’s testing Copilot “with a small group of customers to get feedback and improve our models as we scale,” but did not disclose the name of the customers testing the software. A Microsoft spokesperson added in an email that the company “is testing Copilot with 20 customers, including eight in the Fortune 500.”

Jaime Teevan, a chief scientist and technical fellow at Microsoft, said Copilot passed several privacy checks and has “mitigations in place” in the case the software “gets things wrong or has biases or is misused.”

According to the report, because Microsoft also wants to present new AI functions for Office and its other productivity applications today, the group is trying to prioritize the use of the limited hardware available. One obviously wants to ensure that the announcements about new AI services, which have received a lot of attention, do not suffer from failures or short availability.

Behind all this is the group’s attempt to gain a competitive advantage over the competition with its AI-focused strategy. The Bing bot was launched as part of a multi-billion dollar cooperation with the start-up company OpenAI, which develops the underlying technology in the form of GPT-4, causing a “red alert” at Google. So Google is desperately trying to keep up so as not to lose touch.