Shulou(Shulou.com)11/24 Report--

Many people are cheering the arrival of the AI era, but in fact, the concept of AI was born as early as the 1950s, and has been developing and evolving at a high speed, which is nothing new.

However, for decades, AI has basically stayed in professional areas or specific industries, far away from ordinary users, it is difficult for ordinary people to really feel the power of AI.

ChatGPT's greatest contribution is precisely to bring AI--, rather, generative AI (AIGC)-into the lives of ordinary people.

With ChatGPT and similar applications, anyone who needs only an ordinary computer or mobile phone can feel the convenience that AI brings to our work, life, entertainment and leisure-you can get the answers to the questions you want with a question and answer, and complete a beautiful PPT in a few minutes.

On the other hand, although almost everyone is talking about AI at this time, from large multinational companies to small start-ups, it seems that they all work around AI overnight, but as Buffett famously said: "only when the tide goes out do you know who is swimming naked." Only through the initial uproar can we see who is serious about doing AI and who really has the strength to do a good job of AI.

Recently, Intel held an annual technological innovation conference. AI is naturally the key word in the key words. "AI Everywhere" is not only reflected in the whole conference, but also in the full range of Intel products and solutions. We have discussed this topic many times before from different angles.

Of course, Intel, which belongs to the top ranks of both hardware and software on the planet, is naturally one of the giants most qualified to talk about AI.

As just said, AI is everywhere, and there are completely different rich scenes from products to technology to applications. What ordinary users can feel most directly is AIGC, including Wen Sheng Wen, Wen Sheng Chart, Picture Picture, Wen Sheng Video, Picture Life Video and so on.

In order to achieve enough practical AIGC, from the hardware with powerful computing power to the large model with rich parameters, from accurate and reasonable algorithm to efficient and convenient application, it is indispensable.

We know that in the past, AIGC is more on cloud-side servers, although the performance, model, algorithm are not a problem, but on the one hand, it requires a lot of capital investment, and the other is the lack of delay, privacy and so on.

As a result, more and more AIGC is sinking to the terminal side, so that ordinary PC computers and smartphones can run AIGC, or even execute offline.

Mr. Gao Yu, General Manager of Intel China Technology Department, said in an interview that the research on terminal-side operation of AIGC has achieved fruitful results, such as the latest 13-generation Kouri computer, which can run large models with 7 billion to 18 billion parameters smoothly after nationalization, especially 7 billion to 13 billion parameters.

Of course, these are still in the initial stage, the current optimization is mainly aimed at CPU processors, the next step will give full play to the performance potential of GPU kernel display, while the next generation of core Ultra code-named Meteor Lake will not only have stronger CPU and GPU computing power, but also integrate NPU unit for the first time, a dedicated AI accelerator, peak computing power exceeding 11TOPS, the combination of the three can achieve better results.

For the implementation of end-to-side running AIGC applications in PC, Gao Yu gives an example. BigDL-LLM, an open source framework being built by Intel, is specially designed for low-bit quantization of Intel hardware, and supports a variety of low-bit data accuracy, such as INT3, INT4, INT5, INT8, etc., with better performance and less memory consumption.

Based on this framework, using i9-12900K processor, only four cores are opened to run the ChatGLM2 6 billion parameter model, the generation effect is quite fast, while opening all 8 P cores and 8 E cores, the effect is even faster, and the output performance is about 47 milliseconds per Token, which is not inferior to many cloud-side computing.

The two cases are compared because sometimes you need to put all your computing power into the operation of the AI model, and sometimes you may have to do other tasks as well.

It can be seen that in either case, the Intel PC side has been able to complete the corresponding AI work well, providing satisfactory computing power and efficiency.

In addition, the Intel Core processor can also achieve good running speed on the LLaMA2 13 billion parameter large language model and the StarCoder 15.5 billion parameter code large model.

When switched to the Arc GPU graphics card, the Intel hardware running end-to-side AI is as fast, or even faster, whether it is ChatGLM2 6 billion parameters, or LLaMA2 13 billion parameters, or StarCoder 15.5 billion parameters, and can even be shortened to less than 20 milliseconds in the ChatGLM2 model.

Of course, the large model mentioned above may still be a long way from ordinary people, and the key to the widespread popularity of any technology is to subvert the user's personal work, life, and entertainment experience, and AI is no exception.

In Gao Yu's view, based on the above large model, the typical application of AI on the end side is still quite rich, and there will be more and more, sometimes the effect will be better than running on the cloud side.

For example, the super personal assistant, through low bit quantization, can get better results on the PC side.

For example, document processing, including central idea refinement, syntax error correction and so on, the PC side can not only run well, but also help to protect personal privacy and data security.

For example, the Stable Diffusion of today's fire and the Ventures and video applications of derivative models, the computing power of the PC side is also sufficient.

Using a notebook independent graphics card such as Arc A730M, you can complete high-quality textual graphics, graphic graphics, image style conversion, etc., in a few seconds, thus greatly saving workload and putting more energy on creativity.

This is enough to prove that an ordinary notebook running a large model on the end side, using ordinary unique display or even set display, can still get a fast response speed and a good experience, of course, thanks to the special optimization of Intel.

Of course, in the final analysis, the popularity of AIGC applications in PC is inseparable from enough ecological software that is easy to use.

Such software, on the one hand, can come from a variety of commercial software, they can integrate small and medium-sized large language models, provide a variety of AIGC content, and some creative software can even integrate Stable Diffusion.

On the other hand, you can come from various PC OEM brand manufacturers to integrate specially developed and optimized AIGC software in their computers, pre-installed and provided to users, so that AIGC is really available.

Of course, the end-to-side operation of AIGC is not omnipotent, first, the computing power is not as powerful as the cloud, and the other is limited memory.

At present, the mainstream memory capacity is still 16GB, even if 32GB is popular next year and the year after next, the number of model parameters is limited (less than 13 billion), which requires low-bit processing, such as converting FP16 to INT4. Fortunately, the question answer quality in the large language model will only decline slightly in single digits, while in the Diffusion model, the parameters are not very large, so you can continue to run FP16 accuracy.

In fact, although AI research has achieved considerable results, which will inevitably affect every industry and everyone in the future, AI is still in its early stage. Extensive AI workloads involve different model sizes, model types, the complexity of the overall infrastructure, and the adaptability of cloud-side, end-to-side, hybrid and other different environments, which need to be explored and optimized continuously.

I believe that as powerful enterprises like Intel not only make breakthroughs in AI applications, especially bring more and more AIGC applications to the side, so that more and more people feel the charm of AI, it is bound to go deeper into our work and life more widely and carefully, become an indispensable part of people's daily life, and even unwittingly enjoy the convenience brought by AI.

This is the origin of technology for the benefit of mankind.

(source: Pixabay)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.