Shulou(Shulou.com)11/24 Report--

Over the past few years, I believe we all have the feeling that our work and life are becoming more and more different because of AI.

Using mobile phones to take more and more amazing photos, assisted driving systems to make driving safer and easier, real-time recording and translation with intelligent recorders during meetings, and even more and more realistic man-machine battles in games.

All these are the proof that AI makes our life better and more convenient.

Recently, with the popularity of ChatGPT, generative AI has set off a new wave of artificial intelligence all over the world. It shows us new ways in which artificial intelligence can transform the world.

According to THERE'S AN AI FOR THAT, a large aggregation site, there are more than 3000 generative AI applications and features available. Artificial intelligence is ushering in a new "explosive moment".

Such an "outbreak" is of great significance not only to the development of AI itself, but also to our lives.

Behind the rapid development of generative AI, hybrid AI will become the future of AI. With the rapid development of generative AI, many large model applications and products emerge. These many generative AI models with billions of parameters put forward high requirements for computing infrastructure. In other words, AI needs to find a new development model suitable for generative AI.

Hybrid AI is the most important mode of scale expansion of generative AI, and it is also the future of AI.

The so-called hybrid AI means that the terminal-side AI and the cloud AI work together to distribute the workload of AI computing in an appropriate scenario and time, so as to provide a better experience and utilize resources efficiently.

Specifically, in some scenarios, computing will be primarily terminal-centric, diverting tasks to the cloud if necessary. In the cloud-centric scenario, the terminal will share some AI workloads from the cloud according to its own capabilities.

This kind of structure and mode can be said to be the inevitable road in the process of AI development.

Just as traditional computing is evolving from mainframe and thin client to the current combination of cloud and edge terminals, AI computing is bound to become super-large-scale and complex, so AI processing must be distributed in both the cloud and terminals in order to maximize the potential of AI.

The reason for this is that hybrid AI has sufficient advantages in cost, energy consumption, performance, privacy, security, personalization and other aspects, which can provide support for the large-scale expansion and popularity of generative AI around the world.

For example, first of all, in terms of cost. It is estimated that the cost of each web search query based on generative AI is 10 times that of traditional search, and this is only one of many applications of generative AI. If these operations, processing are put in the cloud, no matter for the size of search engine enterprises, it is a very uneconomical cost burden.

Under the hybrid AI architecture, transferring some processing from the cloud to the edge side terminals can reduce the pressure on the cloud infrastructure and reduce expenses. Hybrid AI can take advantage of billions of edge-side terminals that have been deployed and have AI capabilities for computing, which will greatly reduce the burden on cloud infrastructure. Otherwise, the computing of these billions of terminals will be in the cloud, and the cost can be imagined.

Secondly, in terms of energy consumption, edge terminals can run the generative AI model with very low energy consumption, especially when combining processing and data transmission, which can help cloud service providers significantly reduce the energy consumption of the data center.

In addition, in the hybrid AI architecture, the terminal-side AI processing also has higher reliability. For example, when people use some large model products, they often encounter slow response or even generation failure during peak hours. In the hybrid AI architecture, due to the transfer of a considerable part of the computing load to the terminal side, the cloud demand of spanning AI queries is more likely to avoid peak congestion, thus effectively reducing queuing, high delay, and even denial of service.

In addition, the terminal-side AI essentially helps to protect the privacy of users, because sensitive data and information can be retained on the terminal, which is very important for both enterprise and individual users.

After privacy security is guaranteed, the immediate benefit is that personalized experience becomes possible. Because service providers can provide corresponding services based on users' personalized data to the maximum extent without sacrificing users' privacy. For example, AI assistants on our mobile phones can have a more in-depth understanding of our habits, interests, and needs, so as to draw more accurate user portraits to provide personalized services. And these user portraits are retained in the terminal.

In a word, mixed AI will become the inevitable direction of AI development, and this architecture model will promote the large-scale expansion and popularization of generative AI, and then profoundly change all aspects of our life and production.

How does hybrid AI develop? Let's take a look at Qualcomm's leadership. From the above introduction, it is not difficult to find that terminal-side AI capabilities are the key to enabling hybrid AI and enabling generative AI to scale globally. But in fact, before the emergence of generative AI, the processing power of AI has been transferred to the edge side terminals, mobile phones, notebooks, XR head display, cars and many other edge side terminals have also shown excellent AI processing capabilities, and there are practical applications, such as dark light shooting on mobile phones, face unlocking and so on.

Speaking of which, there is a role that has to be mentioned, he has been committed to promoting the development of terminal-side AI, that is, Qualcomm.

Qualcomm has been involved in AI for more than 15 years and has provided industry-leading AI hardware and software solutions for edge terminals such as billions of mobile phones, cars, XR headsets and glasses, PC and the Internet of things, with considerable leadership in terminal-side AI.

In terms of software, Qualcomm first has strong leadership in terminal-side generative AI. Their research on generative AI can be traced back to spanning counternetwork (GAN) and variational self-encoder (VAE). Using VAE technology, Qualcomm has created better video and voice codecs to control the model size to less than 100 million parameters.

A few months ago, Qualcomm also implemented the world's first Stable Diffusion terminal-side demonstration on an Android phone. Stable Diffusion is a super-large neural network basic model with more than 1 billion parameters, which can generate pictures based on input text prompts. This terminal-side demonstration is conducted in flight mode. Through Qualcomm's full-stack AI optimization, this model can run completely on the terminal side, completing 20-step reasoning in 15 seconds and generating images full of details.

Next, Qualcomm is planning to support models with tens of billions of parameters on the terminal side in the future, which will become a major differentiation advantage of products based on Qualcomm technology.

In addition, running a generative AI model with more than 1 billion parameters in the cloud may require hundreds of watts of power, while Qualcomm needs only a few milliwatts to run on the terminal side. The ability to support terminals to complete more processing work at a given power consumption is also their unique advantage in the field of generative AI.

Qualcomm has carried out full-stack AI optimization for Stable Diffusion. In June last year, Qualcomm launched a leading software stack product specifically for edge-side AI, Qualcomm AI software stack, which can support model optimization at the software level.

For example, in terms of algorithm and model development efficiency, the previous models trained on cloud servers generally use 32-bit floating-point operations (FP32), which means that a lot of processing work is needed to complete model reasoning. For Stable Diffusion, Qualcomm uses 8-bit integer operation (INT8). At the end of last year, the second generation Snapdragon 8 mobile platform further supported 4-bit integer computing (INT4) capability, which will greatly improve the efficiency of AI computing.

In addition, through Qualcomm's AI engine Direct, Qualcomm is able to leverage hardware capabilities in the most efficient way, combined with the second generation of Snapdragon 8 industry-leading Hexagon processors, which will bring generative AI capabilities far ahead of this use case on the terminal.

Through Qualcomm's AI software stack, Qualcomm has been able to run Stable Diffusion on laptops equipped with Snapdragon computing platform and achieve excellent performance. With the industry-leading Qualcomm AI engine, notebooks based on Snapdragon's computing platform were the first to achieve excellent MLPerf benchmark results on MLCommons V3.0, a clear lead in the industry.

Qualcomm AI software stack provides Qualcomm partners, users and developers with an integrated platform that integrates all AI frameworks, developers' libraries and operating systems, allowing them to create an excellent experience on terminals equipped with Snapdragon platform. At the same time, they can use all the tools provided by Qualcomm AI Studio.

In short, the core advantage of Qualcomm's AI software stack is that once the model is developed, it can be used in different places. It is combined with hybrid AI deployment to form a killer combination, which will help generative AI expand on a large scale on different terminals and achieve the popularity of generative AI.

In addition, in terms of hardware, Qualcomm's hardware provides industry-leading energy efficiency, nearly twice as much as competitors in the mobile sector. Among them, Qualcomm AI engine is composed of a number of software and hardware components, which can achieve terminal-side AI acceleration on Snapdragon and Qualcomm platforms. In terms of hardware, Qualcomm AI engine uses heterogeneous computing architecture, including Hexagon processor, Qualcomm Adreno GPU and Qualcomm Kryo CPU, all designed to run AI applications quickly and efficiently on the terminal side. Through heterogeneous computing, developers and OEM vendors can optimize the AI user experience on smartphones and other edge-side terminals.

Among them, the Hexagon processor is the most important part of Qualcomm AI engine. In the second generation Snapdragon 8 mobile platform, the latest Hexagon processor uses a dedicated power supply system, which can adapt power according to the workload. Special hardware improves the performance of packet convolution, activation function acceleration and Hexagon tensor accelerator. Support for microslice reasoning and INT4 hardware acceleration can provide higher performance while reducing energy consumption and memory footprint. Transformer acceleration greatly increases the reasoning speed of the long attention mechanism that is fully used in generative AI, resulting in an astonishing 4.35x improvement in AI performance in specific use cases using MobileBERT.

In short, Qualcomm AI engine is the core of Qualcomm's terminal-side AI advantage, it plays an important role in Snapdragon platform and many other products, it is the crystallization of Qualcomm's full-stack AI optimization for many years, and can provide industry-leading terminal-side AI performance with very low power consumption, which I believe everyone is familiar with.

In addition to the Qualcomm AI engine, another thing that can not be ignored is that the scale of the edge-side terminals deployed by Qualcomm is very large, the number of listed user terminals equipped with Snapdragon and Qualcomm platforms has reached billions, and hundreds of millions of new terminals are still entering the market every year. These terminals cover a wide range of products, including mobile phones, cars, XR, PC and the Internet of things and so on.

All in all, Qualcomm relies on industry-leading hardware to support higher performance at given power consumption; the industry-leading Qualcomm AI software stack; and industry-leading tools such as Qualcomm AI Model Enhancement Kit (AIMET) constitute its unique advantage in enabling hybrid AI scale expansion on a global scale.

What kind of innovative AI application experience can we get in the future? As we mentioned earlier, generative AI is in the ascendant, the era of hybrid AI architecture is coming, and Qualcomm is becoming a leader in promoting large-scale expansion of generative AI with forward-looking early research and product development investment, as well as extensive and in-depth layout on the edge. So, what kind of innovative application experience can hybrid AI bring to us in the future? Here we might as well think about it boldly. I believe it will make you look forward to the development of hybrid AI.

The large-scale expansion of hybrid AI-driven generative AI will undoubtedly drive a new round of content generation, search and productivity-related applications, specifically through a wide range of edge-side terminals as carriers, including smartphones, laptops and PC, cars, XR and a wide range of Internet of things.

First of all, people have the most daily contact and use the most frequently used smartphones. In fact, the most important thing for us to use mobile phones is to search for information and get services. With the development and popularity of generative AI, there will be a great change in the way we search for information on mobile phones. Smartphones will become our real digital assistants. For example, when you finish a week's work and want to relax on weekends, just ask on your phone and it will give you a complete set of strategies for weekend play, diet and rest according to your interests and habits. Especially when generative AI can support other input modes such as text, voice, image, video, etc., the communication between you and your phone will be more concise and accurate. Even when you travel, you send it a photo of the scenic spot, it can tell you the background information and game strategies of the scenic spot, and recommend you the restaurant that best suits your taste.

In this process, the smartphone can grasp your user profile and all other sensitive information, which will be kept on the terminal side, and some basic AI reasoning processes will also take place on the terminal side. At the same time, the cloud and the terminal AI will cooperate with each other to implement the above use cases. All this will be done based on the mobile terminal of Qualcomm's Snapdragon mobile platform.

In addition to smartphones, there are Snapdragon laptops with Snapdragon computing platform, which is mainly to help us in terms of productivity.

In the future, Snapdragon, which introduces generative AI capability, will be able to quickly help you develop a to-do list based on the voice transcription of video conferencing, and automatically generate a complete presentation for direct use by users, enabling productivity to grow exponentially.

For example, Microsoft Office 365, which is used by many people on computers, will speed up our daily workflow greatly when the generative AI capability is integrated. A lot of content only needs AI to help us complete, tasks that used to take hours or days can be completed in only a few minutes.

At the same time, the Snapdragon computing platform has a dedicated hardware unit that can natively support the local use of generated AI, which is completely different from other products on the market.

In addition, there are innovative applications in the automotive field. In the future, the generative AI-powered cockpit will provide a highly personalized experience that will help you plan your travel route. Specifically, according to the starting point and destination information, combined with the rich sensor data of the car, make different route plans, find the best route, and recommend dining options on the way to the restaurant. Or list today's work items on the way to work.

For example, the car will be able to identify each passenger and provide customized experiences and content such as music and podcasts, and the media entertainment experience in the cockpit will be transformed. As in-car AR applications become more common, digital assistants can provide customized displays according to the preferences of drivers.

In addition, car maintenance and services will also become more autonomous and seamless. By analyzing data such as sensor input, maintenance history and driving behavior, the digital assistant can predict when maintenance will be needed.

These per-second scenarios will depend on Qualcomm's Snapdragon cockpit platform. Take the latest generation of Snapdragon cockpit platform as an example, which can provide industry-leading in-vehicle user experience, as well as safety, comfort and reliability, and become a new benchmark for digital cockpit solutions in the era of Internet-connected cars.

Driven by hybrid AI architecture, generative AI can also bring huge application prospects for XR, of which the most promising one should be the popularity of 3D content creation. With the blessing of the next generation of AI rendering tools, you can generate 3D objects and scenes and eventually create a complete virtual world by entering various types of prompts such as text, voice, image or video. For example, when you need to design a decoration plan for your home, you only need to say the specific decoration plan in XR, and you can render the final decoration result, and you can place yourself in it and choose the best one according to different rendering results.

Such a scene may sound a little far away to you, but for Qualcomm, it is on the way to realization step by step. For example, Stable Diffusion and other text-generated image models mentioned earlier will soon empower content creators to generate realistic textures on 3D objects. According to Qualcomm's estimates, these functions will be implemented on smartphones and extended to XR terminals within a year.

In the next few years, the first 3D models of text generation and image generation will be possible to achieve edge-side deployment to generate high-quality 3D object point clouds. In a few years, these models will be upgraded to the level where they can generate high-quality 3D texture objects from scratch. In about a decade, the model will go a step further, supporting high-fidelity full 3D spaces and scenes generated by text or images.

Finally, in a wide range of areas of the Internet of things, generative AI can also help to build professional-oriented GPT type models, as well as IoT assistants to help users complete different tasks. For example, when you come to a new city, generative AI can provide you with travel destination recommendations. In addition, it is also suitable for other vertical areas, such as health care, retail, hotel management and so on.

In retail, for example, in the future, mall guides will be able to help customers customize menus with recipes based on weekly specials, budget constraints and family preferences by standing next to kiosks or smart shopping carts. Store managers can predict aperiodic promotion opportunities based on upcoming events and prepare accordingly.

They can also use simple prompts to have AI help rearrange shelves to make room for profitable products, or use data from nearby chain stores to minimize product shortages.

Generally speaking, as hybrid AI becomes mainstream, generative AI can continue to evolve and bring more and more transformative and innovative application experiences. Through the edge-side terminals of Qualcomm and Snapdragon platforms, we can see firsthand how hybrid AI subverts the way we live, work and play.

Conclusion the trend of hybrid AI is unstoppable. From now on, the collaboration between cloud and terminal-side AI will be closer to create the next-generation user experience through powerful, efficient and highly optimized AI capabilities. Qualcomm's strong leadership in terminal-side AI will bring them unique advantages in the transition to a hybrid architecture. They have more than 15 years of forward-looking research and product investment in the field of AI, excellent terminal-side AI technology and full-stack optimization system, and extensive and large number of global edge-side terminal layout and scale. What they need to do is to more closely support developers, OEM vendors and other ecosystem innovators to quickly and efficiently build new generative AI applications and solutions, so that technology and ecology have a complete landing.

In the future, when we enjoy the all-round enhanced life experience brought by hybrid AI, don't forget the driving role of Qualcomm behind it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.