Shulou(Shulou.com)11/24 Report--

Beijing time on May 17 morning news, according to reports, Google released last week the latest large language model used training data, almost five times the previous generation model in 2022. As a result, the model performed better in programming, math, and creative writing.

Google is launching PaLM2 at its I / O developer conference. Internal documentation shows that the model was trained on 3.6 trillion tokens. Tokens refer to word strings, which are an important basis for training large language models because they teach the model how to predict the next word likely to appear in a string.

The previous version of PaLM was released in 2022, when 780 billion tokens were in use.

While Google is keen to demonstrate the power of its AI technology and its effects embedded in search, email, word processing and spreadsheets, the company is reluctant to release the scale and other details of its training data. Microsoft-backed OpenAI also keeps the details of its latest GPT-4 language model secret.

The companies said they did not disclose the information because of commercial competition concerns. Google and OpenAI are both trying to attract users who want to replace traditional search engines with chatbots to get answers directly.

But as the AI arms race heats up, researchers are also calling for greater transparency.

Since the release of PaLM2, Google has said that the new model is smaller than previous large language models, which means that the company's technical efficiency is improved, but it can complete more complex tasks. Internal documentation shows that PaLM2 is trained on 340 billion parameters-a metric that tells you how complex the model is. The original PaLM was trained on 540 billion parameters.

Google hasn't commented yet.

Google said in a blog post about PaLM2 that the model uses a new technique called "computer-optimized scaling." This makes big languages "more efficient and better overall, including faster inference, fewer parameter calls, and lower service costs." "

In announcing PaLM2, Google confirmed previous media reports that the model was trained for 100 languages and could perform a wider range of tasks. It has been used in 25 features and products, including Bard, the company's experimental chatbot. There are four types of models in the model, from small to large: Gecko, Otter, Bison, and Unicorn.

Based on publicly disclosed information, PaLM2 is more robust than any existing model. Facebook's LLaMA large language model, announced in February, uses 1.4 trillion tokens. The last time OpenAI disclosed the training scale of GPT-3, it said it was based on 300 billion tokens. When OpenAI released GPT-4 in March, it said it showed "human-comparable performance" on many professional tests.

LaMDA is a conversational large language model launched by Google two years ago, and it was also promoted with Bard in February this year. The model was trained on 1.5 trillion tokens.

As new AI applications rapidly enter the mainstream, the controversy surrounding the underlying technology is also growing.

El Mahdi El Mhamdi, a senior research scientist at Google, resigned in February, citing a lack of transparency in AI technology. OpenAI CEO Sam Altman, who appeared at a congressional hearing on privacy and technology on Tuesday, agreed that a new regime is needed to address potential problems with AI.

"For a completely new technology, we need a completely new framework. Altman said,"Of course, companies like us should take a lot of responsibility for the tools we put out. "

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.