Shulou(Shulou.com)11/24 Report--

After a year of catching up, the domestic big model is gradually achieving the lead and transcendence of ChatGPT.

In the past year, the number of large models released in China has reached 158, and the number of large models with more than 1 billion parameters exceeds 80, which is comparable to that of the United States, making it another peak of artificial intelligence in the world.

At the same time of the sudden rapid development in the capacity of the base, the market has gradually reached a consensus: the big model itself does not produce value, its value must be realized by enabling thousands of industries.

Recently, the National Research Institute of Economics of the Development Research Center of the State Council has carried out an evaluation of the application capability of the large model industry to compare the industry performance of the domestic large model with the international first-class large model. and on this basis, it puts forward policy suggestions for the high-level development of China's large model industry.

It is understood that this evaluation selected Spark Model version 3.0, ChatGPT, GPT-4 and other large domestic models as the evaluation objects, and the evaluation industry chose knowledge-intensive producer services (legal services and industrial design), personalized and viable services (education and retail) and some manufacturing industries (automotive engineering, computer). Based on the qualification examination of clinical practitioner, the qualification examination of traditional Chinese medicine practitioner, the national unified legal qualification examination, the professional qualification of professional and technical personnel of motor vehicle inspection and maintenance, and the professional and technical qualification of national computer technology and software, the evaluation questions are constructed to evaluate the performance of the large model in the dimensions of industry knowledge, skill mastery level, production and management scene understanding and so on.

After comparison and evaluation, iFLYTEK newly released iFLYTEK spark 3.0 comprehensive capacity has reached the international first-class level, the performance in all seven evaluation industries are significantly higher than ChatGPT, and in some industries better than GPT-4, leading in China.

(figure: comparison of comprehensive accuracy of different industries)

From the specific evaluation results, the comprehensive accuracy of Spark Model 3.0 in law, education, retail, automotive engineering, computer and industrial design is 69.3%, 71.4%, 82.2%, 61.2%, 78.4%, 76.9% and 66.4%, respectively, with an average accuracy of 72.3%. It performs better than GPT3.5 in all evaluation industries, and has its own advantages and disadvantages with GPT4.0. And the gap between the relatively backward items is also less than 10%.

The National Research Institute of Economics concluded in the report: "the knowledge reserve and language understanding ability of the spark model version 3.0 already have the ability to independently complete some industry tasks and assist human beings to complete complex tasks."

In, law, education and other industries, the performance of the spark model is particularly outstanding. According to the report, the Chinese language knowledge and language comprehension ability of Spark 3.0 in the legal field has exceeded the performance of GPT4 by 5.3% and 4.1% respectively, and the gap between the performance of basic skills in education and that of GPT4 is less than 1%.

From the perspective of application ability, the spark model has a high level of industry knowledge, and has the initial ability to deal with complex problems in the industry. The spark model performs prominently in the Q & An of basic knowledge and domain knowledge in various industries, and its accuracy is higher than that of GPT3.5 in all evaluation industries.

Among them, the assessment fields such as clinical diagnosis, legal case judgment and retail enterprise strategy formulation are more complex topics, which require the model to extract key information from a given scene while having industry knowledge and make judgments. The spark model performs well in this kind of problem, with the correct rate of 65.2%, 63.0% and 66.7% respectively, which is better than that of GPT3.5. The correct rate in clinical diagnosis and legal case judgment is close to that of GPT4.0, and only slightly weaker than GPT4.0 in retail strategy formulation.

The lead of the spark model in the scene is not achieved overnight. In fact, as early as 2017, Fei Zhi Medical assistants have already passed the national qualification examination for medical practitioners, ranking more than 96.3% of human candidates, and have provided help to doctors in grass-roots hospitals and grade hospitals. It is understood that the ability of iFLYTEK has been successfully applied on a large scale in more than 400 counties and districts across the country, providing doctors with a total of 690 million auxiliary diagnoses and correcting more than 1 million first inappropriate diagnoses made by grass-roots doctors.

However, because of the particularity of the scene, "it needs to be treated very strictly", the spark model has not been made public. Until recently polished mature, the overall beyond the GPT4 before the official release. It is understood that iFLYTEK's large model is the first large model to be evaluated by the standards and specifications developed by the Institute of Information and Communication and the National Health Commission.

The National Research Institute of Economics pointed out that the industry application will be the only way for the future development of the large model, and with the continuous improvement of the large model base technology, exploring the landing mode of enabling different industry scenarios will become an important direction for the rapid development of large model enterprises in China, and the industry application value will also become the core index to judge the large model.

Liu Yuanchun, president of Shanghai University of Finance and Economics, pointed out in an interview with the media that for general artificial intelligence, the long-term value of the large model will be realized through industry applications, and the application scenario is the key. Deng Zhidong, a professor and director of the Visual Intelligence Research Center of the Institute of artificial Intelligence at Tsinghua University, also said that the value of the large model lies in application, and only by enabling the development of intelligent economy and intelligent society in a variety of practical application scenarios can industrial value be found.

In the middle of this year, Goldman Sachs Research Institute pointed out in a report that generative AI has great economic potential and is expected to increase global labor productivity by more than 1% a year after it is widely used in the next decade. By using generative AI, enterprises can improve production efficiency, reduce costs, and even create new business models.

However, it is not easy to achieve this large-scale transformation. For most enterprises, exploring application innovation based on large model for vertical scene, vertical industry and vertical field will be the key direction in the future.

The National Research Institute of Economics concluded that with reference to the development path of the mobile Internet, only with the emergence of thousands of AI native applications to solve the real needs of production and life, can the large model really change from "model room" to "commercial housing", go deep into every corner of the social economy, help the industrial upgrading of various industries, promote the rapid recovery of China's economy, and profoundly change people's way of life.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.