Shulou(Shulou.com)11/24 Report--

CTOnews.com, August 10, recently, the team of Shenyang, a professor and doctoral supervisor of the School of Journalism and Communication of Tsinghua University, released a report on the Comprehensive performance Evaluation of the Big language Model (hereinafter referred to as the "report"). The report shows that the total score rate of GPT-4 is the first, and Baidu Wenxin ranks first in China among the 20 indicators of the three dimensions, surpassing ChatGPT, among which Chinese semantic understanding ranks first. Part of the Chinese language ability surpasses GPT-4.

It is understood that this evaluation of the report selected seven major language models, namely, GPT-4, ChatGPT 3.5, Wen Xin Yiyan, Tongyi Qianwen, iFLYTEK Spark, Claude, and Tiangong, to comprehensively examine 20 indicators, including context understanding, Chinese semantic understanding, misleading information identification, logical reasoning, content security, and privacy protection, around the three dimensions of generation quality, use and performance, security and compliance. Generally speaking, the semantic understanding ability of literary heart is outstanding, especially the ability to understand Chinese better and understand Chinese culture better. At the same time, it has strong timeliness and subtle grasp of content security, which stems from the technological innovation of knowledge enhancement, retrieval enhancement and dialogue enhancement.

In terms of generation quality, based on the comprehensive evaluation of semantic understanding, output expression and adaptation generalization, the score rate of Wenxin word is 76.98%, second only to GPT-4 and ahead of other large language models, including ChatGPT. Among them, in some aspects of Chinese semantic understanding, Wen Xin Yiyan ranks first with a score rate of 92%.

CTOnews.com noted that in terms of security compliance, based on the comprehensive evaluation of content security, bias and fairness, and privacy protection, Wenxin scored 78.18%, ranking first with GPT-4.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.