Shulou(Shulou.com)11/24 Report--

The explosion of ChatGPT has ignited people's enthusiasm for general artificial intelligence. When the world is looking forward to GPT-4, Marcus made seven predictions, which poured cold water on everyone. Performance improvement but unable to understand the world, will large language models be the future of AGI?

Towards the end of 2022, OpenAI released a chat robot called ChatGPT, which quickly became popular on the Internet.

Less than two weeks after the launch, more than 1 million people have signed up for an online trial. As long as users enter text, they can immediately get paragraphs of incredible articles, stories and poems.

Its writing is so good that some people use it to write a date prologue on Tinder ("do you mind if I sit here?" The way you do hip punch makes my legs a little weak. ")

Not only that, but to the shock of educators around the world, students began to use ChatGPT to write term papers. Others are using it to try to reshape search engines. All of a sudden, the whole world is talking about the magic of ChatGPT.

Still, Marcus says, we can't filter chatbots too thick.

Although ChatGPT seems to know everything, it is also error-prone. In an interview, Marcus said that ChatGPT, like before, is "still unreliable, still doesn't understand the real world, still doesn't understand the psychological world, and is still full of mistakes." "

In other words, ChatGPT often makes things up; much of what it says is not true at all.

For example, induced by users, ChatGPT can say that Youtiao is very suitable for surgery because "they are small in size, can achieve greater accuracy and control during the operation, reduce the risk of complications and improve the overall outcome of the operation".

Chatbots talk nonsense so seriously that the famous website Stack Overflow has temporarily banned computer-generated answers.

And mistakes are endless. Although ChatGPT often adjusts according to user feedback, a few weeks after the product launch, many netizens are still speechless about its answer:

Similar mistakes occur so frequently that even Sam Altman, the CEO of OpenAI, has to admit the reality:

ChatGPT still has many limitations, but enough to create the illusion of greatness.

It is still too early to rely on ChatGPT to accomplish important tasks. We still have a lot to do to improve robustness and authenticity.

In short, ChatGPT sounds like science fiction like a computer in Star Trek, but at the moment, people can't fully trust it.

Of course, ChatGPT is a gift from AI enthusiasts in 2022. What about 2023?

In 2023, what Silicon Valley and the whole world are looking forward to is GPT-4.

People who have actually tried GPT-4 in the world of GPT-4:AI are impressed by this product. According to some rumors, GPT-4 will be released in the spring of 2023. At that time, it will overshadow ChatGPT; to be sure, more people will talk about it.

In many ways, expectations for GPT-4 are very high:

Nick Davidov, founder of DVC, a venture capital firm, said that the emergence of GPT-4 would bring "an economic impact similar to the COVID-19 epidemic". The rapid spread and use of GPT-4 can "rapidly increase the productivity of hundreds of millions of knowledge workers".

Technically speaking, GPT-4 will have more parameters, more processors and memory, and will be trained for more data.

GPT-1 was trained in 4.6GB data, but in GPT-3, the amount of data soared directly to 750GB. Thus it can be seen that the amount of training of GPT-4 will be even more amazing, and it will even learn most of the entire Internet.

OpenAI knows that more training means better output. After each iteration, GPT behaves more and more like a human. For GPT-4, it may evolve into a performance monster.

But will it solve the problems encountered before? Marcus still has a question mark about it.

While GPT-4 certainly looks smarter than its predecessor, there are still problems with its internal structure.

Marcus said he suspected that people would have a sense of deja vu when they were on GPT-4: first there was a fire all over the Internet, and then a few days later, people found that many problems still existed.

According to current information, GPT-4 is basically the same as GPT-3 in architecture. If so, it can be expected that some basic problems remain unsolved: chatbots still lack an internal model to understand how the world works.

As a result, GPT-4 cannot understand things at an abstract level. It may be better at helping students write papers, but it still doesn't really understand the world, and the answers still reveal the characteristics of the machine between the lines.

Seven predictions about GPT-4 so while the AI world was delighted with the arrival of GPT-4, Marcus made seven less positive predictions.

1. GPT-4 will still make all kinds of stupid mistakes like its predecessors. Sometimes it does a good job of accomplishing a given task, sometimes it goes on strike, but you can't predict in advance what's going to happen.

2. GPT-4 's reasoning on physics, psychology and mathematics is still unreliable. It may be able to solve some of the previously unsuccessful projects, but it is still at a loss in the face of longer and more complex scenarios.

For example, when asked about medical questions, it either refuses to answer or occasionally utters nonsense that sounds reasonable but dangerous. Although it has engulfed a lot of content on the Internet, it is not credible and complete enough to provide reliable medical advice.

3. Fluency hallucinations (fluent hallucinations) will still be common and easily induced. In other words, large language models are still a tool that can be easily used to produce information that sounds reasonable but completely wrong.

4. GPT-4 's natural language output is still unable to provide services to downstream programs in a reliable way. Developers who use it to build a virtual assistant will find that they cannot reliably map the user's language to the user's intention.

5. GPT-4 itself will not be a general artificial intelligence that can solve any task. Without outside help, it can neither beat Meta's Cicero; in Diplomacy nor drive a car reliably, let alone drive Optimus Prime in Transformers, or be as versatile as Rosie in the Jason family.

6. The "docking" between what humans want and what machines do is still a key and unsolved problem. GPT-4 will still be unable to control its output, some suggestions are surprisingly bad, and examples of masking biases will be discovered within days or months.

7. When AGI (General artificial Intelligence) is implemented, large language models like GPT-4 may be part of the final solution, but only part of it. Simple "extension", that is, building a larger model until it absorbs the entire Internet, will prove to be useful to some extent. But trustworthy universal artificial intelligence, which is consistent with human values, must come from more structured systems. It will have more built-in knowledge and include clear reasoning and planning tools. And these are what the current GPT system lacks.

Marcus believes that within a decade, perhaps less, the focus of artificial intelligence will shift from the expansion of large language models to combining with a wider range of technologies.

Cool stuff is always fun, but that doesn't mean it can lead us to credible general artificial intelligence.

In response, Marcus predicts that what we need in the future is a new architecture with explicit knowledge and world models at the core.

Reference:

Https://garymarcus.substack.com/p/what-to-expect-when-youre-expecting

This article comes from the official account of Wechat: Xin Zhiyuan (ID:AI_era), edited by Xin Peng and so sleepy.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.