Shulou(Shulou.com)11/24 Report--

Thanks to CTOnews.com netizen Coje_He for the clue delivery! CTOnews.com April 23 message, ChatGPT chatbot can generate a variety of text, including code, based on user input. However, four researchers at the University of Quebec in Canada found that the code generated by ChatGPT often has serious security problems, and it does not actively remind users of these questions, and only admits their mistakes when they ask.

The researchers presented their findings in a paper that CTOnews.com looked at and found that they had asked ChatGPT to generate 21 programs and scripts in C, C++, Python and Java. These programs and scripts are designed to show specific security vulnerabilities, such as memory corruption, denial of service, deserialization, and encryption implementations. The results showed that only 5 of the 21 programs generated by ChatGPT on the first attempt were safe. After further prompting to correct its error steps, the large language model managed to generate seven more secure applications, but this is only "security" related to the specific vulnerability being evaluated. This is not to say that there are no other exploitable vulnerabilities in the final code.

The researchers point out that part of the problem with ChatGPT is that it does not consider hostile code execution models. It repeatedly tells users that security problems can be avoided by "not entering invalid data", but this is not feasible in the real world. However, it seems to be able to recognize and acknowledge key vulnerabilities in its suggested code.

"obviously, it's just an algorithm," Rapha ë l Khoury, a professor of computer science and engineering at the University of Quebec and a co-author of the paper, told The Register. "it doesn't know anything, but it can identify unsafe behavior." He said that the initial ChatGPT response to security issues was to recommend only valid input, which was clearly unreasonable. It provides useful guidance only when asked to improve the problem later.

The researchers believe that this behavior of ChatGPT is not ideal because users who know what questions to ask require some knowledge of specific vulnerabilities and coding techniques.

The researchers also point out that there are moral inconsistencies in ChatGPT. It refuses to create attack code, but it creates vulnerable code. They cite an example of a Java deserialization vulnerability. "the chatbot generated the vulnerable code and gave advice on how to make it more secure, but said it could not create a more secure version of the code."

Khoury believes that ChatGPT is a risk in its current form, but this is not to say that there is no proper use of this unstable, underperforming AI assistant. "We have seen students use this tool, and programmers will use it in real life." "so it's very dangerous to have a tool that generates unsafe code," he said. we need to make students realize that if the code is generated with this type of tool, it is likely to be unsafe. " He also said he was surprised that when they asked ChatGPT to generate code in different languages for the same task, sometimes it generated secure code for one language and vulnerable code for another, "because this language model is a bit like a black box, and I really don't have a good explanation or theory to explain it."

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.