Shulou(Shulou.com)11/24 Report--

For the future of the scientific community, join the open source LLM camp!

Free ChatGPT is very cool to use, but the biggest disadvantage of this closed-source language model is that it is not open-source, and it is impossible for outsiders to understand the training data behind it and whether it will disclose users' privacy, which has led to a series of alpaca models such as LLaMA jointly opened up by industry and academia.

In a recent article on Nature worldview, Arthur Spirling, a professor of politics and data science at New York University, called on people to use open source models more frequently. The experimental results can be reproduced and are in line with academic ethics.

The point is, in case OpenAI gets upset one day, shuts down the language model interface, or raises prices by closed monopoly, users can only say helplessly, "after all, academia has been defeated by capital."

Author Arthur Spirling will join Princeton University to teach political science in July this year. His main research interests are political methodology and legislative behavior, such as textual data (text-as-data), natural language processing, Bayesian statistics, machine learning, project response theory and the application of generalized linear models in political science.

Researchers should avoid the temptation of commercial models and jointly develop transparent large-scale language models to ensure repeatability.

Embracing open source and rejecting monopoly seems to be the launch of a new large language model (LLM) every day, and its creators and academic stakeholders make impassioned remarks about the ability of the new model to communicate smoothly with humans, such as helping users change code, write letters of recommendation, write abstracts of articles, and so on.

As a political and data scientist who is using and teaching how to use these models, I think scholars should be vigilant, because the most popular language models are still private and closed, that is, they are operated by companies. they will not disclose specific information about the basic model, but will only independently check or verify the ability of the model, so researchers and the public do not know which files are used in the training of the model.

The rush to incorporate language models into one's own research process may be problematic and may threaten the hard-won progress in "research ethics" and "reproducibility of results".

Not only can we not rely on commercial models, researchers should also work together to develop large open source language models that are transparent and independent of the interests of a specific company.

Although commercial models are very convenient and can be used out of the box, it is a historical trend to invest in open source language models. We should not only find ways to promote development, but also let the models be applied to future research.

I optimistically estimate that the future of language modeling tools must be open source, similar to the development history of open source statistical software. At the beginning, commercial statistical software is very popular, but now almost all communities are using open source platforms such as R or Python.

For example, the development team of BLOOM, an open source language model released last July, is a New York-based artificial intelligence company that has worked with more than a thousand volunteers and researchers to build it, partly funded by the French government; other teams are also working to open source large language models.

I think open source projects like this are great, but we need more cooperation and pooling international resources and expertise.

Teams with large open source language models are usually not as well funded as large companies, and development teams need to keep up with the latest developments in the field: the AI domain is growing so fast that most language models become obsolete weeks or months after launch.

So the more scholars involved in open source, the better the effect of the open source model will eventually be.

Using open source LLM is critical to "repeatable research", because closed-source commercial language model owners can change their products or their training data at any time, possibly changing the results of the model generation.

For example, one research team may publish a paper to test whether the language recommended by commercial language models can help clinicians communicate with patients more effectively; if another team tries to replicate the study, who knows whether the basic training data of the model are the same as at that time? Even whether the model is still in operation is unknown.

GPT-3, an auxiliary tool commonly used by researchers, has been replaced by GPT-4, and all research based on GPT-3 interfaces may not be repeated in the future. For companies, it is not a high priority to keep the old model running.

In contrast, with open source LLM, researchers can view the internal architecture and weights of the model, understand how the model works, customize the code and point out errors, including the adjustable parameters of the model and data for training the model, and community participation and oversight can help keep this model robust for a long time.

The use of commercial language models in scientific research also has a negative impact on research ethics, as the texts used to train these models are unknown and may include direct information between users on social media platforms or content written by children.

Although the person who produced the public text may have agreed to the platform's terms of service, this may not be the informed consent standard that researchers would like to see.

In my opinion, scientists should stay away from using these models in their work as far as possible. We should turn to an open language model and promote it to others.

In addition, I think scholars, especially those with a large number of social media followers, should not push others to use commercial models, and if prices soar or companies fail, researchers may regret spreading the technology to colleagues.

At present, researchers can turn to open language models made by private organizations, such as LLaMA, which is open source with Meta, the parent company of Facebook, which is initially released in the form of user application and review, but the full version of the model is later leaked online; you can also use Meta's open language model OPT-175 B.

In the long run, the downside is that the release of these models is too dependent on the kindness of the company, which is an unstable situation.

In addition, there should be a code of academic conduct for cooperation with language models, as well as corresponding regulatory measures, but these will take time, and based on my experience as a political scientist, I expect these rules to be very imperfect at first. and slow to work.

At the same time, large-scale collaborative projects are in urgent need of support to train open source language models for research, such as the European Institute for Particle Physics (CERN), the International Particle Physics Organization, and governments should increase funding through grants.

The area is growing at a lightning pace and now needs to begin to coordinate domestic and international support.

The scientific community needs to be able to assess the risks of the resulting model and to release it to the public carefully, but it is clear that the open environment is correct.

Reference:

Https://www.nature.com/articles/d41586-023-01295-4

This article comes from the official account of Wechat: Xin Zhiyuan (ID:AI_era)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.