Shulou(Shulou.com)11/24 Report--

In a new paper submitted to the preprinted document Library (ArXiv), three Google researchers pointed out that transformer, the underlying technology in the field of artificial intelligence, is not good at generalization.

Transformer is the foundation of the big language model behind artificial intelligence tools such as ChatGPT. In a new paper submitted to ArXiv on November 1, three authors, Steve Yadlowsky, Lyric Doshi and Nilesh Tripuraneni, wrote: "when tasks or functions are needed beyond the scope of pre-training data, transformer has a variety of failure modes and finds that even simple task extension reduces their inductive ability."

According to this paper, the deep neural network transformer is good at performing tasks related to training data, but not very good at dealing with tasks beyond this scope.

For those who want to achieve general artificial intelligence (AGI), this problem can not be ignored. General artificial intelligence is a hypothetical artificial intelligence used by technicians to describe anything that human beings can do. For now, artificial intelligence is very good at performing specific tasks, but it is not as capable of transferring skills across domains as humans do.

Pedro Domingos, emeritus professor of computer science and engineering at the University of Washington, said the new study meant that "at this point, we should not be too crazy about the coming artificial intelligence."

AGI is touted as the ultimate goal in the field of artificial intelligence. In theory, it represents that human beings create something as smart as or smarter than themselves. Many investors and technicians are devoting a lot of time and energy to this.

On Monday, OpenAI chief executive Sam Altman took the stage with Microsoft chief executive Satya Satya Nadella to reiterate his vision of "working together to build AGI."

Achieving this goal means enabling artificial intelligence to accomplish many inductive tasks that can be accomplished by the human brain, including adapting to unfamiliar scenes, creating analogies, processing new information, and abstract thinking.

But, as the researchers point out, if this technology is difficult to achieve even a "simple task extension", then it is clear that we are still a long way from the goal.

Arvind Narayanan, a computer science professor at Princeton University, wrote on social media platform X: "this paper has nothing to do with the big language model, but it seems to be the last straw to break the collective belief bubble, making many people accept the limitations of the big language model."it's time to wake up."

Jin Fan, a senior artificial intelligence scientist at Nvidia, questioned why the findings of the paper surprised people because "transformer is not a panacea".

Mr Domingos said the study highlighted "a lot of people who are very confused" about the potential of a technology touted as the road to AGI.

He added: "this is a paper that has just been published. It's interesting who will be surprised and who won't be surprised."

Although Domingos acknowledges that transformer is an advanced technology, he believes that many people think that this deep neural network is much more powerful than it really is.

"the problem is that neural networks are very opaque, and these big language models are trained on unimaginable amounts of data, which makes a lot of people very confused about what they can and can't do," he said. "they always think they can work miracles."

More advanced artificial intelligence may do a better job of generalization. Google researchers used the GPT-2 scale model instead of the more mainstream GPT-4 scale model in their research.

Sharon Zhou, chief executive of artificial intelligence start-up Lamini AI, says she doesn't think transformer is hard to generalize as a problem.

"that's why I started a company that trains models instead of just asking them questions so they can learn new things," she said. "they are still very useful and can still be guided and adjusted."

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.