Shulou(Shulou.com)11/24 Report--

[guide to Xin Zhiyuan] recently, many netizens reacted that GPT-4 was not easy to use and regretted recharging. Later, netizens revealed that OpenAI had redesigned the architecture with MOE, which affected the performance, but there was no clear answer from the authorities.

Recently, many OpenAI users have reacted that GPT-4 has become stupid!

It is generally believed that GPT-4 has been generating content faster since May, but the quality of the content has declined significantly.

Some users in the OpenAI forum even posted the content they generated, saying that compared with the content generated by the same prompt two months ago, the ability of GPT-4 can be called GPT-3.6 at most.

One after another expressed their heartache for the 20 dollars a month.

However, after reading so many complaints from GPT-4 heavy users, the editor suddenly began to sigh:

GPT-4 is really way ahead.

Leading the way, other big models are working overtime in a few days to release an updated version, hoping to catch up with OpenAI as soon as possible.

His own home is good, quietly "reduce costs and increase efficiency" and "reverse upgrade", and act as a "pacesetter against inward rolls" in the large model circle.

Possible causes of performance degradation for GPT-4 performance degradation, users seem to have reached a consensus and have begun to discuss the causes of GPT-4 "recession".

According to foreign media reports, some experts in the field of language models speculate that OpenAI is creating a number of small GPT-4 models that behave similar to large models but with lower operating costs.

According to an expert named Sharon Zhou, OpenAI uses a framework called the "Hybrid expert Model" (Mixture of Experts,MOE) to build GPT-4.

Each smaller expert model is trained in different tasks and subject areas.

There may be a mini-GPT-4 model for biology, or other mini-models for physics, chemistry and other fields.

When a GPT-4 user asks a question, the new system judges and sends the prompt to one or more expert models.

"this idea has been around for some time, and this is a natural direction of development. "Zhou said.

Zhou likens the situation to the "ship of Theseus," in which parts are gradually replaced and the whole ship slowly becomes a new ship.

"OpenAI is turning GPT-4 into a fleet of small boats," she said.

From my point of view, this is a new model, but many people may not care about this change. "

After leaking a lot of details about the parameters of GPT-4 online this week, many bosses also gave their own comments on the MoE section mentioned in it.

Oren Etzioni, founding CEO of Allen artificial Intelligence Research Institute, wrote in an interview with foreign media: "I think these guesses are roughly accurate, although there is no way to confirm it." "

He explained that there are two main reasons for using MOE: better generation of responses and cheaper and faster responses.

Etzioni added: "the right mix of expert models will guarantee both advantages, but as with anything, there is usually a trade-off between cost and quality. "

If this assumption is true, OpenAI may indeed sacrifice some quality in terms of reducing costs.

Another proof is that Greg Brockman, the founder of OpenAI, mentioned MOE as a technology path in his own research.

Through the hybrid expert model (MoE) method, only a small portion of the network is used at a time to calculate the output of any one input. This allows you to add more parameters without increasing the computational cost. "Brockman and his colleague Lilian Weng wrote in the paper.

The decline in performance may be related to the elimination of harmful content. In addition to the boss's analysis from the technical level, netizens have also speculated from various angles.

Some netizens think that OpenAI's relatively conservative moral policy may have reduced the quality of the model response.

In the GPT-related sub-sections of the official OpenAI forum, many people mentioned that ChatGPT had changed the inclusiveness of Prompt before OpenAI adopted new content restrictions.

The side supports the view that OpenAI may control the output capacity and scope of the model for security reasons.

In the discussion of YCombinator, some netizens pointed out that the performance of the model has been getting worse and worse since OpenAI began to declare its concern for security.

Take GPT-4 's task of letting GPT-4 draw unicorns using TikZ (TikZ is probably the most complex and powerful tool for creating graphical elements in LaTex) in the initial release of the report, for example, it is difficult to reproduce the results on the report in subsequent releases of GPT-4, and the performance is getting worse and worse.

On the other hand, netizens believe that the reason for the decline in GPT-4 performance may be that human beings have instilled a set of contradictory beliefs in GPT-4 for security reasons.

This practice of eliminating "bad thoughts" for the sake of safety will obviously prevent GPT-4 from having a deeper understanding and understanding of things.

In fact, OpenAI sets various filters to prevent malicious acts, such as violence, racism or hate speech.

However, these filters often interfere with normal use, such as novel writing, brainstorming and other forms of creation.

Although it is difficult to measure the impact of these changes, Sam Altman, founder of OpenAI, acknowledges that policy restrictions interfere with the user experience.

In addition, in a conversation between Humanloop co-founder Raza and Sam Altman on May 29, Altman mentioned that OpenAI is currently severely limited by the number of GPU.

When ChatGPT breaks through the 100 million user mark and the number of monthly active users reaches 1.8 billion, OpenAI's servers are often overloaded.

The shortage of computing power has not only delayed many of their short-term plans, but also led to a lot of complaints and disputes from customers about the reliability of GPT responses.

Maybe the reason lies in this year's netizens? In the response of the official technical spokesman Logan.GPT, it is indicated that the quality of GPT-4 has not declined in the API mode.

Just yesterday, Peter Welinder, vice president of product at OpenAI, commented on the rumor.

"No, we didn't make GPT-4 stupid, on the contrary: we made each new version smarter than the previous one. "

As for the questions raised by netizens, his understanding is, "when you use it more, you will begin to notice problems that you didn't notice before." "

The implication is that GPT-4 is not as strong as it used to be, and now GPT-4 is its real strength.

There's a little bit of Versailles.

Sure enough, netizens didn't buy it very much, and they kept replying to it under tweets.

Isn't it self-contradictory that you say you haven't changed, and then say that every version is better?

The hypothesis you said is not valid, and the answer of the same prompt in July is worse than that in June!

However, some netizens do have different explanations for the performance degradation of GPT-4:

From the psychological point of view of perceptual degradation, after the surprise brought by the initial exposure of GPT-4 disappears and gradually goes deep into our daily life, people will get used to its ability.

As soon as the freshness wears off, users' tolerance for it decreases, and they pay more attention to the defects in technology and use.

In addition, netizens who posted on Reddit that the performance of GPT-4 was declining, showed in a recent update that the question that day may have been influenced by the mentality, because the GPT-4 work has been going well since then.

Reference:

Https://the-decoder.com/openai-comments-on-reports-of-gpt-4-performance-loss/

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.