Shulou(Shulou.com)11/24 Report--

According to reports, people familiar with the matter revealed that although Microsoft is about to announce a complete set of Office 365 tools using GPT-4 artificial intelligence (AI) software, the company is now facing a shortage of internal AI server hardware.

This forced Microsoft to ration hardware resources and limit the use of other internal AI tool development teams to ensure that the new Bing chatbot based on GPT-4 and the upcoming new Office tools get enough resources. This hardware shortage also has an impact on Microsoft customers: At least one customer said it took a long time to use OpenAI software already available through Microsoft Azure cloud services.

Microsoft says it is ramping up the purchase of more hardware to increase AI computing power, but if it isn't fast enough, it risks limiting the appeal of its Azure OpenAI service to new customers. These customers will use this service to add AI functionality to their apps. Microsoft was already ahead of Google in commercializing this new AI tool, but if it can't solve the hardware shortage problem, this advantage will be difficult to play out.

Google beat Microsoft to the punch this week by releasing its own AI-assisted writing tools, including software that automatically composes text in Google Docs and Sheets based on brief prompts.

The server shortage comes at a time when OpenAI and Microsoft are trying to rapidly expand sales, and customers can use their own data to fine-tune the AI software to develop customized AI tools such as image generation, document summarization, search engines and chatbots.

Microsoft has invested billions of dollars in OpenAI and has the right to sell software to startups through its Azure OpenAI service. OpenAI also licenses the software directly to customers-and as part of the partnership with Microsoft, all OpenAI machine learning models must run on Azure servers. Both services are priced the same, charging less than a penny per request.

The services launched by both companies in the past three months have attracted widespread interest from enterprise users, who both want to integrate the underlying technology of OpenAI's ChatGPT chatbot into their own products. But new customers of Microsoft Azure OpenAI services face long wait times.

At the heart of the hardware shortage is the fact that new AI big models such as GPT-4 need to run on GPUs, server chips that can process large amounts of data at once. To handle the massive computing demands of this model, Microsoft has consolidated tens of thousands of GPUs into clusters that are distributed across data centers.

Microsoft currently has to reserve a lot of its existing GPU resources for the new Bing GPT-4 chatbot and the upcoming GPT-4 Office tool until user demand is clear.

Microsoft just launched Bing's smart Chat features in February and is still working to measure how many people are willing to use the service on an average day, people familiar with the matter said. This makes it difficult for engineers to predict the computing resources required for the feature, according to people familiar with the matter.

The upcoming Office GPT-4 tool could also lead to a significant surge in demand from Microsoft customers for OpenAI chatbot technology. These new features include AI document summaries, personalized writing suggestions, and editing suggestions.

Other Microsoft teams working on AI had to give way to Bing and Office, including teams developing various machine learning models, including Microsoft's Turing natural language generation model, which understands text and previously provided the underlying technology for search tools in email and Office apps.

People familiar with the matter said that when these teams want to use GPUs to develop new AI tools or test existing AI software, they must submit a special application to a company vice president for approval before using the hardware.

People familiar with the matter said some applications took days or even weeks to be approved. Microsoft has been rationing internal GPU resources since late 2022, but the wait has been getting longer since January.

A Microsoft spokesperson added in the statement that the company is adding more AI resources to services such as Azure and following a "process of prioritizing customer needs and adjusting to them." They added that Microsoft isn't worried about AI resources.

At least one Microsoft customer revealed that they also experienced problems with delayed access. Edo Segal, founder and CEO of marketing software startup TouchCast, said: "It's almost impossible to use the program immediately." Edo's company is developing interactive user manuals based on GPT for several car companies, and the technology is licensed through Microsoft Azure.

Existing AzureAI customers are barely affected. Spokesmen for self-driving car developer Cruise and AI search startup Perplexity both said they had no difficulty using Azure's GPU resources.

Last November, Microsoft and GPU maker NVIDIA announced that Microsoft would add tens of thousands of processors to expand AI processing power, but it is unclear how far the purchase will go.

The two companies also announced Monday that Azure customers will soon be able to pre-empt Nvidia's new H100 GPU, which has yet to be widely released. A small number of Microsoft internal teams have begun using H100, but most departments have not yet obtained this permission, people familiar with the matter said.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.