Shulou(Shulou.com)11/24 Report--

More important than Detective GPT, AI generates the "tolerance" of content in different industries.

The hottest topic in the tech world in the last two months has undoubtedly been OpenAI's conversational AI app ChatGPT, which allows it not only to write you a poem, answer any of your questions, but even to help write less complex code. The industry even believes that ChatGPT even has the potential to replace Google.

ChatGPT was also cheered by another group of people, "social animals" and students, who found that tasks such as monthly summaries, school papers and assignments could be done by ChatGPT, who even wrote well, not much worse than real people.

ChatGPT seems to make life easier for students, but it gives teachers more "headaches" because it is difficult for teachers to confirm whether the words are written by students or by OpenAI products. To this end, the New York Department of Education even banned the use of ChatGPT in public schools.

In the midst of this confusion, a Princeton student, Edward Tian, launched an app that kills ChatGPT-"GPTZero". With this "demon mirror", whether the content is written by a man or a machine, the truth will be revealed immediately.

"GPTZero" quickly became popular on the Internet, attracting the attention of Silicon Valley venture capitalists, including A16z. But Tian, the app's creator, believes that the most important thing is to "make AI more transparent."

01. "ChatGPT Killer" after ChatGPT became an artifact for students to be lazy, educational and scientific research institutions had to resist this new nightmare.

The New York Department of Education announced a ban on students using ChatGPT in public schools, and ICML, one of the world's leading machine learning conferences, also banned the publication of papers containing content generated by ChatGPT and other similar systems to avoid "unintended consequences."

Out of concern about the moral issues related to the use of ChatGPT in academia, Chinese brother Edward Tian spent a winter vacation in a local coffee shop to develop GPTZero, hoping to restore rigor in academia.

Tian, 22, is still a senior at Princeton University, majoring in computer science, specializing in natural language processing and a minor in cognitive science and journalism.

GPTZero developer Edward Tian | Network he was also a researcher at the BBC and Bellingcat, an open source intelligence site, and an analyst at Miburo Solutions, a counter-terrorism startup acquired by Microsoft. There, he monitors false information and robot detection. "all these experiences are the driving force behind his development of GPTZero," Tian said.

On January 2, 2023, Tian posted GPTZero on the Internet, and only a few dozen people were expected to try it. I had no idea that it would cause a world-class uproar.

Within hours of uploading the software to the Internet, more than 2000 people had tested a public version of GPTZero on Steamlit.

On January 5, the third day of release, Tian made updates and improvements to GPTZero and significantly reduced the false alarm rate; by this time, the new program has more than 10,000 users! Tian can't help but be shocked by its "explosive growth" and "viral spread".

According to NPR, more than 30,000 people tried GPTZero within a week, even causing the app to crash due to unexpectedly high network traffic. Streamlit, the free platform hosting GPTZero, has since stepped in to support Tian with more memory and resources to handle network traffic.

Edward Tian shows how the app distinguishes texts written by humans from those written by artificial intelligence by showing its analysis of a New Yorker article and a post from the ChatGPT generator on LinkedIn.

The principle of GPT Zero is to detect the "Perplexity" and "Burstiness" of the text, and rate them respectively, and determine whether the text is written by artificial intelligence or human according to the statistical characteristics. Overall, if the scores for both parameters are low, then the text is likely to be written by AI.

The "perplexity" here refers to the complexity and randomness of the language from the works written by human beings.

This indicator is mainly used to measure the randomness of the text in a sentence and whether the construction of a sentence will confuse GPTZero.

Every time a user enters a test into GPTZero, it calculates "the total confusion of the text", "the average confusion of all sentences" and "the confusion of each sentence".

The lower these values, the more likely it is that the text is "familiar" to GPTZero, so it is likely to be generated by AI; conversely, if the higher these values, the more they indicate that GPTZero is "surprised" by the construction of sentences or the way words are used in the text, then it is more likely to be human.

This is because artificial intelligence has been trained by a database, and the resulting text will show more uniform and constant confusion over a period of time, and the choice of words will be more predictable, while human-written texts will not be like this. Real people's choice of words and sentences are generally more random and easier to write more unexpected words than machines.

Use GPTZero to detect whether text is generated by ChatGPT | Twitter and "sudden" refers to a change in sentence structure used by humans.

The main purpose of this parameter is to compare the changing degree of sentence complexity and measure their consistency.

This is because human beings tend to write highly complex texts, while texts produced by artificial intelligence are low-complexity; in addition, because human thinking structures are not linear, their sentence structures follow similar patterns.

This means that when human beings use sentence structure, they will vacillate between long and complex sentences and short and simple sentences, and there are more sentence pattern changes, such as the alternating coexistence of complex and simple sentences, followed by shorter sentences after a long and difficult sentence; on the other hand, machine-generated sentences tend to be more unified and rarely have a series of sentences with large differences in length.

In short, "simple" and "familiar" in the choice of words, and the use of "unified and neat" sentences are the hallmarks of works generated by artificial intelligence, while more complex and diverse things indicate that they are written by human beings. This is also the reason why the two indicators of "bewilderment" and "abruptness" can be used as criteria.

In addition to Edward Tian's own testing of GPTZero, many netizens have used it to test the content generated by ChatGPT and some GPT-3 derivatives many times. The final results show that GPTZero can capture the text generated by AI every time and correctly identify the text written by humans in more than a dozen cases.

With the sudden success of GPTZero, Tian has won the favor of well-known venture capitalists such as A16z, Menlo Ventures and Red Swan. However, in the face of private Twitter messages and phone bombardment, Tian appeared extremely calm, modestly saying that he would not refuse calls from investors, but he would not forget that he was still a graduating senior.

At the same time, he also said that his GPTZero has not yet been completed and still needs to be improved and further developed, and even plans to let people continue to use his program for free to support the work of new English teachers everywhere.

02. Much-needed "AI Transparency" as to whether GPTZero is a new program for AI writing, public opinion on Twitter is mixed. Most adults represented by teachers like to hear and hear, while students satirize the creator of GPTZero, Tian, as an "academic narcotics policeman".

Indeed, when GPTZero was launched, Tian received positive feedback from teachers about the app's testing of articles written by AI, and countless teachers from around the world expressed their gratitude to Tian-making it much easier for them to teach.

Of course, it is not difficult to understand that many students are not optimistic about Tian, a software that fights academic Jerry-building and gains without hard work.

In fact, not only Tian, but even OpenAI, the developer of ChatGPT, has demonstrated his commitment to preventing plagiarism in artificial intelligence.

In December 2022, Scott Aaronson, a researcher at OpenAI who specializes in artificial intelligence security, revealed that the company was working on "mitigation measures" that would use an "imperceptible secret signal" to "watermark" GPT-generated text to identify its source and crack down on cheating systems.

The technique will work by subtly adjusting the choice of specific words selected by ChatGPT, which readers won't notice, but for anyone looking for signs of machine-generated text, it's statistically predictable.

"We use ChatGPT as a preview technology for new research and hope to learn from real-world applications," a company spokesman said. We believe that this is a key part of developing and deploying a powerful and secure AI system. We will continue to learn from feedback and lessons learned, "he said.

GPTZero homepage | GPTZero in addition, OpenAI has joined forces with Harvard and other university institutions to create a detector: GPT-2 Output Detector.

The authors first released a "GPT-2 generated content" and WebText dataset to help AI understand the differences between machine language and human language.

Subsequently, the RoBERTa model is fine-tuned with this data set, and the AI detector is obtained. Among them, all human languages are recognized as True,AI-generated content, and all contents are identified as Fake.

It is worth mentioning that RoBERTa is an improved version of BERT. The original BERT used a dataset the size of 13GB, but RoBERTa used a 160GB dataset containing 63 million pieces of English news.

In spite of this, many people think that "AI text detector" is doomed to a failed "arms race", and its actual effect is not ideal, let alone the development of AI language models such as ChatGPT.

However, although Tian established GPTZero, he is not opposed to the use of artificial intelligence tools such as ChatGPT, and he believes that the purpose of GPTZero applications is not to prevent the use of these new technologies, but to provide a way to use them responsibly and to provide the necessary protection.

At the same time, perhaps more important than confronting or banning a technology is how to set norms and standards for its use. For example, in the advertising, film, television and entertainment industries, the tolerance for AI-generated content may be appropriately increased, while in the academic, educational and scientific research fields, there is a great emphasis on accuracy and originality, and there is no doubt that the tolerance for AI-generated content is lower.

How to determine the "transparency" used by AI tools may be more effective and meaningful than studying how to "anti-AI".

This article is from the official account of Wechat: geek Park (ID:geekpark), by Meiyi

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.