Shulou(Shulou.com)11/24 Report--

In sharp contrast to the ambition of the domestic cloud AI chip companies to surpass Nvidia in the PPT press conference three or four years ago, these companies have hit a brick wall everywhere in reality, and many of them can't even find the north.

"just after the New year, when we sent the chip to the customer for testing, we found that our software could not meet the needs of the customer's application scenario at all, because some functions were not taken into account when we were designed, and we were dumbfounded on the spot."

This is still the pain that companies with AI chips can feel. Companies whose products have not yet come out are struggling to raise capital in the cold winter of 2022.

"I can only use bloodshed to describe the horror of grabbing financing." Lucy, a practitioner who has seen AI chip companies scramble for financing, said, "in order to get financing, some start-ups write overhaul reports that smear their competitors and send them separately to investors to block their financing."

Of course, in order to get financing, AI chips told a new story about GPU.

"people who don't understand will not vote, and people who really understand will only vote for AI chip companies that can really land." This is almost the consensus of the industry.

But when the cloud AI chip companies landed, they really realized that only when they got the "ticket" to design the chip, did they really realize how difficult the software was.

What is more clear to cloud AI chip entrepreneurs is that, not to mention surpassing Nvidia, selling an AI accelerator card is the most realistic goal.

This leaves a question worth pondering: if AI chip participants and investors just want to make quick money, what will the end of the carnival leave to the domestic semiconductor industry?

In the cold winter of capital, in order to grab financing, the cost of developing a single chip is as high as hundreds of millions of yuan, and the annual salary expenditure of employees is as high as hundreds of millions of yuan of large chip start-ups, a large amount of financing has become the key to survival. "with more financing, you have a better chance of killing competitors, and you will eventually survive." People in the digital investment community and the chip industry all told Leifeng.

Financing ability has become an important ability of a large chip start-up company at the helm, and the founders of large chip companies will be even more tested in the capital winter of 2022.

"the capital on the market is limited, in order to be able to obtain financing, in addition to fighting for strength, some companies have also played a means." Lucy was outraged. "in order to hinder competitor financing, a chip company dug up information from former employees of a competitor company and asked a third party to send an one-on-one survey report to investors. Investors received the report and questioned its credibility." ask the organization that wrote the report to confirm that the organization stumbled and replied that it was written by an intern, which is a bit clumsy. "

It's easy to understand that the GPGPU startup, which is trying to grab financing, will stop at nothing. After all, as one of the few companies that haven't officially released their products, they have to get financing in order to get tickets to the competition, especially in 2022, when investors are becoming more cautious and rational.

"fortunately, the founder cheated us at that time. If the founder had been particularly rational in analyzing the risks of investing in big chips, we would not have dared to invest in big chips. now that we are on board, we will row together." An aftertaste of AI chip investors became more cautious in 2022.

This is very different from what happened a few years ago.

After the AI boom before 2016, the founders of AI chips could get hundreds of millions of dollars of financing without even a complete business plan. Later, the founders of AI chips were able to raise money by surpassing Nvidia on PPT. But now, investors have to look not only at the products, but also at the MLPerf Benchmark of the chip (a set of general benchmarks for measuring and improving the performance of machine learning software and hardware), and more practical investors look directly at the invoices of orders for chip landing. " Zhang Wei (a pseudonym), a practitioner of AI chips, sees it very clearly.

"it's too difficult to come out with AI chip startups, even though we Angel Wheel invested in a very potential AI chip company, and their products are also in the process of landing, but the high valuation does not have enough support, which is a lot of pressure for me, and we are considering quitting." Wang Jun (a pseudonym), an investor who invested in a number of early projects, expressed his true thoughts.

With regard to the valuation of AI Chip, Blake, an investor who gave up investing in AI Chip start-up, said, "there is not a good anchor for the valuation of AI Chip. It can be increased or reduced by 50%, which is not a good opportunity for investors."

Why is capital still pouring into AI cloud chips? "one is track logic, where Nvidia invests when it sees good business and high market capitalization. The other is that investors in the primary market are huddled together. And many investors are not professional, especially cross-sector investors and institutions." Blake thinks.

In fact, not only the companies without products are scrambling for financing, but the AI chip companies with products but tight funds are also involved in the "grab financing".

"Morning Bird", which got up early in the morning and caught up with an evening episode, is interesting to say. Company A, which has products but has to grab financing, has faced a financing crisis before, and two upheavals have cast a shadow over the company's prospects. Another AI chip startup B, which got up early, prepared its products early, but was trapped in the software, and its first-mover advantage was exhausted.

Two AI chip companies got up early in the morning and caught up with the evening collection, one from the dimension of company management, the other from the dimension of products, showing the great challenge of large chip entrepreneurship.

After two upheavals, Company A was founded early and released its first product in 2018, giving it a first-mover advantage, but former employees of the company told Leifeng that more than a year after the company released its first product, financing had been in round B, and the financing schedule could not keep up with the demand for research and development, so it began to reduce business, arrears of wages, layoffs, and even CEO was "laid off".

According to a former employee of Company A, "in that big change, the company only retained the chip team, the company's management and the business reshuffle." however, financing is only one of the factors in the company's unrest, and it is actually a combination of various factors. "

The industry says that the company's CEO and its co-founders don't agree on the technology path, and the co-founders and investors work together to knock CEO out.

The play of the founder being kicked out is not new, but unfortunately, the change of key people in the company seems to be a "robbery" for the company.

Company A rallied after the first upheaval, released new products and ushered in a new man at the helm, and suffered another upheaval when everything was back on track.

"the new CEO of Company An is a bit arrogant, claiming that a company's project is bound to be won, and that it will be able to land a project of 200 million in 2022." A number of people in the AI chip industry have mentioned to Leifeng Network (official account: Leifeng Network).

But the experienced CEO was investigated before he actually generated $200 million in revenue.

"for a moment, the popular CEO seems to have become a hot potato. Not to mention NT $200 million, I wonder if Company A can achieve tens of millions of dollars in revenue this year." This has become the topic of concern of the guild.

Each generation of products are different, there is no high-quality landing project Company A due to human factors failed to take the lead, Company B was limited to the perception of the founding team.

"the software is so bad that customers can't use it, so it's hard to land." Several former employees of Company B and their counterparts in the industry have made such comments.

The software didn't work because there was something wrong with the hardware design.

"Company B's chips have reached the third generation, but each generation of chip architecture is changing, and even the chief architect is different. The hardware micro-architecture designed is also very different. The hardware architecture is not continuous. No matter how hard the software engineers try, the software is also difficult to reuse, and each generation of product software seems to start from scratch." Ma Chao (a pseudonym), who knows Company B, said: "on the other hand, Company B does not have the technology to really control the entire software stack."

A former employee of Company B revealed that the company does not have a top bull in the industry, and although the software team has a good background, it is almost impossible to make a good product because of the background that they do not like each other. Of course, the company has hired a master in a certain technology field in the industry, but in the face of such a large software package of AI chips, it is estimated that it is also very difficult to control, and left after more than a month or two months.

For this situation, several people in the industry have some consensus that the founder of this company is indeed a chip expert with deep accumulation, but after all, he is not an expert in chip architecture, and it is normal to have limitations. However, there is no software in the founding team of this company, and it is difficult for people recruited from the outside to be "accepted" no matter how capable they are, and they do not have a voice in the core management team, and it is difficult to work together to make good products.

The lack of efficient and easy-to-use software is also the key reason why investors in Company B failed to pass the grayscale test and enter the large-scale procurement process, although they bought a small number of chip tests. Of course, Company B also got a government project, and the order amount is not small, but it is doubtful whether it can actually produce real profits, so it is not a high-quality, replicable landing project.

In fact, at present, most of the founding teams of AI cloud chip companies in China have a profound chip industry background, and there are obvious limitations in the understanding and importance of software.

A head of software of a domestic AI chip company said bluntly: "neither CEO nor CTO can understand my work. I think there is a gap of more than ten years between the founders of some domestic AI companies and the leading international companies."

So, what is the difficulty of AI chip software?

Troubled by AI software stack AI chip company head Chen Jun (a pseudonym) pointed out that, on the one hand, AI chip software is from scratch, with natural complexity, different from CPU, each AI chip computing architecture and instructions are different, from the compiler to the library and then to the framework adaptation, there are not open source reusable things like CPU.

On the other hand, the software ecology of AI is actually the Nvidia ecology, but the software ecology of Nvidia, especially the core parts related to CUDA, are closed and closed. It is easy to imagine how difficult it is to make your own software compatible with Nvidia's ecology. To build a new AI ecology on your own in a short period of time is a pipe dream.

Finally, the variability of cloud-based AI reasoning applications. At present, the algorithm and model of AI are still developing and iterating rapidly, so it is not easy to optimize the ResNet 50 model of image classification, the new language model BERT model becomes popular again, the natural language processing model begins to become bigger and bigger, and various deformations of BERT flourish, which also increases the difficulty of developing AI chip software.

In particular, the difficulty of compiler-related development, and the model generalization ability of automatic performance optimization through compilers for different models without manual optimization, the lack of this ability has almost become the core reason why most AI chips stay in "sending test" and can not get orders.

These are the technical challenges faced by all Nvidia's challengers, as well as the challenges of talent.

Ma Chao and most practitioners of AI chips share the same point of view: "to create a complete and easy-to-use AI software stack, there must be a software Daniel who is not only familiar with drivers, firmware, and other basic software, but also can see from the top down, and has a comprehensive understanding of the whole AI software ecology and has sufficient experience and ability."

"before AI chips became popular, working as a compiler in China was a very unpopular profession." Chen Jun said: "the compiler is an important part of the AI chip software stack, domestic chip-related software personnel are very scarce."

With a ticket in hand, defeated in the customer "abnormal" model, the problem of AI chip start-up software stack is not unsolved, Kunlun core, which has been landed in Baidu, and Sim computing, which has jumped to the ground in bytes, is one of the few domestic AI chip companies that have been tested and passed by the commercial market, and have explored a commercialization road that can be replicated.

Wang Lei (a pseudonym), who is familiar with Sim computing, said, "the software is all related to the scene. If you want to do the software well, you can only get close to the customer and go deep into the business." not only talk to the customer who is in charge of the system, but also communicate deeply with the people of operation and maintenance, business scenarios and algorithms, otherwise it is very difficult to do the software well. "

"if you want to do a good job of software, there are no shortcuts. Domestic AI chip companies are all on the same starting line, and there are investors from big Internet companies. Shim can run a little faster, or the people who are responsible for landing the market are grinding out of customer offices and factories every day, but even so, there is no guarantee of future success." Wang Lei thinks.

But AI chip companies can not easily get the opportunity to communicate with customers in depth. Generally speaking, companies with demand (such as BAT and mobile operators) will invite public bids, and AI chip companies will seek cooperation. After preliminary screening, AI chip companies that meet the needs can send test products and run customer-given AI models on the site.

"A lot of companies can't even get through that level of compilation." Wang Lei said, "even if it can be compiled, many companies claim that their computing power is twice that of Nvidia's products of the same level, but it is actually less than its performance."

Zhang Wei said, "it is mainly because the compiler is not good enough, usually it is manually optimized for a specific model according to the chip storage characteristics, and does not have the generalization ability." So when you encounter a customer's special 'abnormal' model, you will encounter difficulties, and even if you compile manually, the performance will be limited. For example, in general, images with a size of 96 to 96 can be compiled smoothly, and the throughput performance is good, but customers will adjust the size of the input images according to their business needs, such as 1280 to 720, then the performance will be greatly affected, or even fail the compilation. What's more, the model structure will undergo changes such as basic operators and logic, so the generalization ability of the compiler is difficult to support running directly. "

This is another difficulty of the AI chip compiler. Because the customer's AI model is closely related to its business and involves trade secrets, it will not give the model directly to the chip company, and it is difficult for the AI chip company to do targeted optimization in advance.

But even if progress is faster, Kunlun core needs more time to refine its software stack. What Zhang Wei knows is that Kunlun core's AI chip has more than double the performance advantage of Nvidia's products in search scenarios, but has little or no advantage in other scenarios.

"have heard Kunlun core customer feedback left Kunlun core people to help debug, the chip is still very difficult to use." "the software is still not easy to use," Chen said. "all AI chip companies still need time to polish, which takes a process."

This is a task that takes a lot of time and effort for all cloud AI chip companies. During the landing process, hundreds of features may need to be developed for customers, which are requirements that can not be fully defined at the beginning of the design of hardware and software stack, and even do not realize what application scenarios and requirements users will have.

This year's papers will be handed in, and the landing race for cloud AI chip companies to be phased out in 2024 has begun. Ma Chao believes that the second half of this year is the time for AI chip companies to hand in papers to investors and the market. If it is not possible to land on the ground this year, some companies may begin to shrink at the end of this year and early next year.

Chen Jun believes that the pattern of cloud reasoning AI chips will be clearer next year.

Wang Lei believes that even AI chip companies, which raise billions of people, will last until 2024 at most according to the size of a thousand people and a salary of one million per capita, when they will be able to see people swimming naked.

If you want not to be eliminated in the AI market competition, the product is as important as the choice. AI cloud chip companies are giving priority to purchasing billion-dollar head Internet companies and government projects.

"the demand for government projects seems to be great, but the actual demand is much smaller than expected," said Zhang Wei. "

"the government's projects seem to cost a lot of money, and the chip companies have to bear high costs, but in fact the profits are not high. More importantly, the government's AI projects do not have continuity and replicability." "in the Cambrian period, government projects were signed every year in recent years, and now the market capitalization is less than 30 billion, which is enough to illustrate the attitude of capital," Zhang said. "

"how can those companies whose valuations are about to catch up with the Cambrian period continue to develop in the future?" Many people in the AI chip circle are skeptical.

Therefore, the current test of a cloud AI chip company, whether using DSA (domain-specific architecture) or GPGPU architecture, can be landed in the Internet company is the embodiment of hard power.

Internet companies have stringent performance and stability requirements for AI chips, and being able to land in the scenarios of Internet companies not only proves the availability of their products, but also shows the replicability of their AI chips.

But we should also see that the growth of BAT is slowing down, the iterative speed of AI algorithm is also slowing down, although the future of AI chip is bright, but the road is still tortuous, especially with Nvidia, a leader who is difficult to surpass.

It took Nvidia more than a decade to build a CUDA-based AI ecosystem, with a large number of partners to optimize software and adapt to the latest algorithms, top hardware teams in the industry to iterate over products, and customers who have long been accustomed to Nvidia's software platform. "how can we catch up with other people's success for more than a decade in a few years? I dare not do that." This is the voice of CEO, an AI chip startup.

"the gap and difficulty do exist, but many people just want to make quick money, first expand the company, do not properly polish the products, rush to commercialization, and then go public as soon as possible, what can be left to China's semiconductor industry?" This is an unanswered question left by practitioners.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.