Shulou(Shulou.com)11/24 Report--

The hottest topic in the technology industry over the past year has been generative AI. The popularity of ChatGPT has set off a new wave of artificial intelligence around the world, and the phrase "AI changes the world" seems to have a real impact for the first time.

In this new era, in addition to ChatGPT, various generative AI models are also springing up like bamboo shoots after a spring rain. The "Map Research report on large models of artificial Intelligence in China" released by the China Institute of Science and Technology Information and other institutions shows that as of May this year, the number of large models that have been publicly disclosed in China has reached 79.

There are not only large models of natural language, but also covering image, voice, video and other multimodal fields. In addition to various enterprises, there are different types of subjects, such as domestic universities and scientific research institutions, who are actively participating in the research and development of large models.

All of a sudden, it is like a spring breeze blowing overnight, like thousands of trees and thousands of pears in full bloom.

Under this background, on June 28, at the "V-Tech experience Innovation Technology Summit" organized by the volcano engine and co-organized by Nvidia, the volcano engine made an eye-catching move: the launch of the large model service platform "Volcano Ark", which provides enterprises with a full range of platform services (MaaS, namely Model-as-a-Service), such as model fine tuning, evaluation, reasoning and so on.

While most companies are focusing on the construction of the big model itself, the volcanic engine does not go the usual way and chooses to build the platform ecology. why are they doing this? And where will this "ark" lead volcanic engines in the new era of generative AI? All these are worthy of attention.

Why does the volcano engine want to build a large model service platform? There is no need to say how hot the generative AI model is. For ordinary users, we can use the general large model to quickly obtain all kinds of knowledge and materials, let it assist us in office work, generate copywriting, illustrations, etc., or just chat with the big model, which is also very interesting. But as a technology that can change the world, it is clear that its mission is much more than that. And its greater value is actually on the B side. When the AI model can be truly integrated into the whole process of scientific research, production, manufacturing and management of thousands of industries, it is undoubtedly decisive for the improvement of social productivity.

For example, hospitals can use the AI model to generate more accurate and comprehensive electronic medical records for patients, and allocate and manage medical resources more efficiently.

Enterprises can make more accurate production plans through the powerful data analysis ability of large models and various office systems, as well as real-time detection of production process, real-time analysis of equipment operation status, workers' production capacity, raw material consumption, etc., to control production costs.

With the help of the AI model in the school education system, teachers will be able to grasp the changes of students' learning situation more quickly and accurately, and at the same time assist in the design of teaching courses.

These beautiful scenes can be said to be the development direction that the AI model must follow, otherwise it will not be able to show its power of change.

Based on this, the volcanic engine also has its own clear judgment for the future development of the large model industry, which is the basis for them to find their own position.

First of all, the future large model market should be a multi-model ecology in which a hundred flowers blossom. Perhaps in this ecosystem, there will be a few very large-scale models at the same time, and more importantly, there will be multiple medium-scale large-scale models and more vertical models for more industries. As a result, a multi-cloud and multi-model ecology is formed. in turn, open market competition and model diversity will further promote the development of the whole technology.

Secondly, the volcano engine believes that in the future enterprises, especially the industry leaders, their own application of large models will be a "1 + N" application model.

That is, through self-research or in-depth cooperation with tripartite model service providers, enterprises form their own main model; in addition to this main model, in different scenarios, enterprises will also apply N external models at the same time.

Under such a development trend, some important and objective problems will arise.

As the large model market becomes more and more yuan, the competition is becoming more and more fierce. For large model providers, the cost of training and reasoning will be higher and higher, and how to control the cost is a problem. And how to make their own large model products accurately find the corresponding customers, is not easy.

In turn, for model users, they also need to find large model products that meet their business needs efficiently and quickly. Moreover, many enterprises have diverse businesses and will not only use a large model, but how to access these services easily and quickly is also very important.

At this time, we need a platform, a platform that can serve both suppliers and users of large models, so that suppliers can easily and cheaply train and sell their own large models, so that users can call the ability of large models at any time, that is, use-and-take, so that the channel between supply and demand can be opened up.

Such a platform is the role that volcanic engines want to play in such a blueprint.

To take an example that may not be appropriate, the rich and varied large model products of the future are like various App of the mobile Internet era, and what the volcano engine has to do is AppStore. The experience of the mobile Internet era has told us that only by doing platform and ecology can we build the most unbreakable moat.

What does a volcanic ark look like? After understanding the logic behind the volcanic engine building the volcanic ark, let's take a look at what kind of platform the volcanic ark is.

The volcanic Ark platform is composed of several core parts, and the action route design accords with the working habits of the large model application, and highlights the volcanic engine's understanding of how to make good use of the large model.

The Volcano Ark first has a model square, where users can see many excellent model providers, as well as their different versions / sizes of models.

At present, the Volcano Ark has integrated the large models of many AI technology companies and research institutes, such as Baichuan Intelligence, going out to ask, MOSS of Fudan University, IDEA Research Institute, Lanzhou Science and Technology, MiniMax, Zhisu AI, and so on, and has started the invitation test.

Users can interact with these models directly to get an intuitive experience. Inference API can be called directly on the volcano engine to access everyone's production environment. This is an agile short link suitable for rapid analysis and AB experiments, continuously shortening the distance from new ideas to a try for algorithm engineers and business teams.

In addition, the development of domestic large models is still in the initial stage, can not meet the needs of enterprises through an API, need to do fine tuning and continuous training combined with industry scenarios and data, Volcano Ark also provides such a platform.

On the Volcano Ark, enterprises can also carry out model evaluation, which can be divided into manual evaluation and automatic evaluation. Manual evaluation can refine the subjective performance of the model, while automatic evaluation can help you keep up with the pace of model iteration. In the process of continuous comparison / evaluation / experiment, enterprises can accumulate evaluation data, give a variety of business scenarios and business entrances, and select different and most appropriate models.

After selecting a large model, you can also fine-tune the model. Because the business of many enterprises is highly vertical, they need to use their own data and domain non-public data for continuous training before they can be applied to the normal workflow. On the Volcano Ark, enterprise customers can freely use the fine tuning function of the model by manually setting advanced parameters, verification sets, test sets, and so on.

When the user initiates the fine tuning task of the model, the Volcano Ark will also be evaluated automatically, and the fine tuning effect and operation indicators will be tracked in real time on the platform and shown to the user in a concise and intuitive way.

In addition, because the Volcano Ark is the application of large models and the complete integration of machine learning platform. Therefore, the above workflow is also suitable for the large model of customer's own training, so that the training / evaluation / comparison / reasoning / iteration can be tightly linked together.

This is not enough for a good large model service platform.

For the Volcano Ark, it is also committed to solving three problems.

The first is security and trust. In the era of large model, data security will usher in a new proposition. According to a survey by Cyberhaven, a cyber security company, at least 4 per cent of employees previously entered corporate sensitive data into ChatGPT, accounting for 11 per cent of the input. At the beginning of 2023, Samsung discovered that its semiconductor equipment-related confidential data had been leaked less than 20 days after using ChatGPT, and there were three similar accidents in succession.

The Volcano Ark does make safety and security a top priority. According to Wu Di, head of intelligent algorithm for volcano engine, at present, they have launched a security mutual trust calculation scheme for large models based on security sandbox, using computing isolation, storage isolation, network isolation, traffic audit and other ways. the confidentiality, integrity and availability of the model are guaranteed, which is suitable for customers with low delay requirements for training and reasoning.

At the same time, Volcano Ark also uses a hardware-based trusted computing environment, combined with CPU's TEE technology, and Nvidia H800 trusted computing technology, which will bring hardware-level trust environment reinforcement in the follow-up. In addition, federated learning technology, which has been developed over the years, will also play an important role in the field of large models, which gains mutual trust through the split of data assets.

The second problem is the ratio of performance to price, that is, to reduce the cost of using the large model.

At the press conference, the volcano engine showed two growth curves of the large model industry, the first curve is the calculation power consumption carried by the model training, and the second curve is the calculation power consumption of the model application and tuning load. They predict that by the fall of 2024, the cost of reasoning-based large model applications will exceed 60% of the cost of pre-training, and at some point in 2025, it will exceed the cost of pre-training.

We know that the cost of training large models is already very expensive, and these two curves show that the cost of model reasoning will exceed the cost of training in the future, and it will be more expensive. Therefore, reducing the reasoning cost will be an important factor in the landing of the large model, and it is also the significance of the existence of the volcanic ark.

The volcano engine provides high reliability for the reasoning of large models, as well as enterprise-level load balancing and fault tolerance. As the platform iterates, the supply of resources to the large model will be more flexible / dynamic and cheaper.

At the same time, the Volcano Ark will further reduce the unit cost of reasoning by means of flow peak misalignment and integration of training and reasoning, which is also an important advantage brought by using cloud / upper cloud in the era of large models, because the scale on the cloud is the largest, and the scale determines the unit cost, so using cloud is very important for the cost control of large model reasoning.

Only when the cost of training and reasoning is low enough, can people focus more attention on the business level, and the big model can really be widely used.

Finally, there is the problem of the ecosystem. If the large language model is compared to CPU, then the Volcano Ark, as a service platform, has to produce the entire motherboard in addition to CPU, that is, the whole supporting work should be done well.

Specifically, the first is the plug-in, and later, the Volcano Ark will provide a large number of plug-ins, and each plug-in will be equipped with a data set, like instructions or drivers, to tell the pedestal model how to interact with it correctly.

Then there is the microservice network of the domain model. The volcano engine believes that the general pedestal model will better accomplish multimodal tasks in the future. However, in a certain period of time, and in some vertical applications that attach great importance to the cost of reasoning, domain models and large language models will work closely together to complete some complex work. Later on the Volcano Ark, a micro-service network of domain model will be built, including image segmentation, speech recognition and many other professional models, which can be called by the CPU at any time.

In short, at this stage, enterprises choose a large model or plan to develop a large model, there are many things to consider, including performance-to-price ratio, what reasoning cost, delay and so on, as well as their own business needs, security, convenience and so on. As a large model service platform that has just set sail, the integrity of Volcano Ark is amazing enough, covering all aspects such as cost, safety, simplicity of use, and the functionality of training and pushing, and so on. At present, the types of integrated large models are also rich enough.

If all kinds of large models are compared to goods and stores, then iFLYTEK Ark is like a large shopping supermarket. now this supermarket has an atmospheric environment, complete supporting facilities, low store rents, and a good location, so is there any reason for merchants not to move in? how can consumers not like shopping?

The engine of the Volcano Ark Volcano Ark where the giant ship can sail smoothly, largely depends on the engine, and this engine is the volcanic engine. As we mentioned earlier, the Volcano Ark is actually an organic whole of "large model technology" and "volcano engine machine learning platform", so it can provide model suppliers with abundant computing support and continuous performance optimization. there is also excellent performance-to-price ratio, these advantages are essentially from the volcanic engine.

We know that the volcano engine is the computing service provider of Douyin. Tik Tok's business scenario has very stringent requirements on traffic, latency, stability, and so on, and the volcanic engine has been fully trained and verified. Douyin also perfectly completed the live broadcast of the World Cup last year, and it is the volcanic engine that provides arithmetic support.

Just in April, Volcano engine also merged with byte beat domestic business. Based on internal and external unified cloud native infrastructure, idle computing resources of Douyin and other businesses can be quickly scheduled to volcano engine customers, offline business resources dispatch 100,000 core CPU at minute level, online business resources can also be tidal reused, and dozens of EB of enterprise storage. These are the leading domestic companies. It constitutes a wealth of training and reasoning resources for large model suppliers.

The future large-scale computing center will be dominated by the hybrid computing structure of "CPU+GPU+DPU". Not long ago, Volcano engine self-developed DPU (Data Processing Unit) has also been successfully applied and deployed in tens of thousands of DPU servers. Among the new computing instances based on self-developed DPU, the performance of NVIDIA GPU computing instances is 3 times higher than that of the previous generation, and the performance of new CPU instances and small-size instances is up to 6 times higher. And their intelligent recommendation-high-speed training engine can also greatly optimize the training reasoning efficiency of the model.

In terms of algorithm training, Volcano engine also has a new machine learning platform, which can support 10,000-card-level large model training, microsecond delay network, and flexible computing, which can save 70% of the computing cost. moreover, it has been polished for a long time by Douyin's massive user business, so it's very reliable.

In terms of commercial landing, the volcano engine is also supported by a complete ToB service system and team. in fact, before the launch of the Volcano Ark, most of the companies with larger models mentioned above are already on the volcano engine cloud, such as MiniMax, Zhisu AI, Baichuan Intelligence, Lanzhou Technology, going out to ask, and so on.

For example, MiniMax has achieved large model iteration speed from month to week on the volcano engine and exponential growth in user interaction. "it may be the first company in China to implement parallel training of thousands of cards on the public cloud." MiniMax has also developed a super-large-scale reasoning platform, which steadily supports hundreds of millions of large model reasoning calls every day. At the same time, their text, voice and visual models have all landed on the Volcano Ark, promoting cooperation between the two sides to further deepen.

For example, Jingtai Technology, which specializes in AI medicine, has also achieved extremely flexible computing requirements on volcanic engines. Based on the original cloud base, both sides have created a hybrid high-performance computing platform that is high-speed and flexible, and can accommodate huge computing power. With the efforts of both sides, Jingtai Technology has achieved a high resource utilization rate of more than 95% of the cluster packing rate, and a high-performance example of AI Pharmaceutical screened with ultra-high throughput. And intelligent scheduling of heterogeneous computing power accurately issued by 500000 tasks (CPU&GPU). The digital twin technology of volcano engine has also accelerated the automated experiments of drug research and development, such as full-process AI generation strategy, protein twin visualization and vision-assisted algorithms, enabling drug development to improve efficiency.

It can be said that the volcanic engine, as well as the technical capabilities, service capabilities, and proven actual combat capabilities precipitated behind it, is the strength that the Volcano Ark can dare to be a service platform and brave the wind and waves in the generative AI tide.

Conclusion the large model is the most exciting technological innovation at present. if the large model is compared to an ecosystem, it is obvious that only more diverse species can ensure the health and sustainability of the whole ecosystem.

At the press conference, Tan Ju, president of the volcano engine, said. At the beginning of the voyage of the Volcano Ark, there may be many areas that need to be improved, but its exploration in the large model service platform, as well as the adhesion of upstream and downstream development enterprises, service enterprises, application enterprises, entrepreneurs and developers in the whole large model industry can not be ignored. It is believed that with the efforts of the volcanic engine, a good ecology of "multi-cloud and multi-model" will certainly be built, and the large model technology will certainly be able to better serve all trades and industries and jointly promote the progress of society as a whole.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.