Shulou(Shulou.com)11/24 Report--

CTOnews.com, October 20, Stanford University recently released the basic AI model "transparency index", in which the highest indicator is Meta's Lama 2, but the related "transparency" is only 54%, so researchers believe that almost all AI models on the market are "lack of transparency."

It is reported that the study, led by Rishi Bommasani, director of the HAI basic Model Research Center (CRFM), surveyed the 10 most popular basic models overseas:

Llama 2 of Meta.

BloomZ of BigScience,

GPT-4 of OpenAI,

Stable Diffusion of Stability AI,

Claude of Anthropic PBC,

Google's PaLM 2.

Command of Cohere,

Jurassic-2 of AI21 Labs,

Inflection of Inflection AI,

Amazon's Titan.

Rishi Bommasani believes that "lack of transparency" has always been a problem for the AI industry. In terms of specific model "transparency indicators", CTOnews.com found that The related evaluation contents mainly focus on "copyright of model training data set", "computing resources used in training model", "credibility of model generated content", "model's own ability", "risk of model being induced to generate harmful content", "user privacy of using model", etc., a total of 100 items.

The final survey showed that Meta's Lama 2 topped the list with 54 per cent transparency, while OpenAI's GPT-4 transparency was 48 per cent and Google's PaLM 2 ranked fifth with 40 per cent.

▲ source Stanford University in the specific indicators, the top ten model scores are "the best" is the "model basic information" (Model Basics), this assessment mainly includes "whether the model training accurately introduces the model, scale, structure", the average transparency is 63%. The worst performer is Impact, which mainly assesses whether the underlying model "fetches user information for evaluation", with an average transparency of only 11%.

Percy Liang, director of CRFM, said that the "transparency" of business basic models is very important for promoting AI legislation, and related industries and academia.

Rishi Bommasani says lower model transparency makes it harder for companies to know whether they can safely rely on relevant models and makes it difficult for researchers to rely on them for research.

Rishi Bommasani finally believes that the above ten basic models are "failed" in terms of transparency. Although Meta has the highest score in Llama 2, it does not meet the needs of the outside world. "the transparency of the model must be at least 82% before it can be recognized by the outside world."

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.