Shulou(Shulou.com)11/24 Report--

Sex, age and skin color are all variables.

Author | Cao Tingting

An embarrassing discovery:

The autopilot system also discriminates against people.

In a study conducted by researchers at King's College London, a loophole was found after examining more than 8000 images:

The AI-driven pedestrian detection system used in self-driving cars is 19.67% less accurate for children than adults, and 7.53% lower for dark skin than for light skin.

On the other hand, there is no significant difference in the detection accuracy between the sexes, with a gap of only 1.1%.

This means that for self-driving cars, children and dark-skinned pedestrians will be more difficult to detect than adults and light-skinned pedestrians.

Why is this?

01. Unfriendly to children and dark-skinned people, let's take a look at the experimental process.

Using the method of data analysis, the research team first found eight pedestrian detection systems that are most commonly used by self-driving companies and are common on the market.

Then use these pedestrian detection systems to collect real scene test data, including different brightness, contrast, weather conditions and other actual scenes, these data sets are mainly composed of real street images.

Source: Waymo they got a total of 8311 images in four real scenes, showing pedestrians in different poses, sizes, and occlusion scenes. The researchers also specifically tagged the pedestrians in the image, with 16070 personality tags, 20115 age tags and 3513 skin color tags.

The focus of the study is whether the response of the autopilot pedestrian detection system to different pedestrians is the same, especially in terms of sex, age and skin color.

The detection systems used include ALFNet, CSP, MGAN and PRNet, among which ALFNet uses multi-step prediction for asymptotic location, which solves the limitation of single-step detection in pedestrian detection.

CSP introduces an anchor-free method by locating the center and zooming the pedestrian, while MGAN uses the bounding box information of the visible region to guide attention generation, which is mainly used to detect pedestrians in the case of occlusion.

Source: after collecting the images of the paper, the research team used a difference formula to solve the question of whether the autopilot system is unfair to the group. MR generally refers to the most commonly used performance indicator in pedestrian detection research, MR=1-TP/ (TP+FN), where TP (true positive) refers to the number of true bounding boxes successfully deleted, and FN (false negative) refers to the number of undetected true bounding boxes.

After calculation, the failure rate of pedestrian detectors to female and male pedestrians is similar, with a difference of 1.1%, while there is a big difference in age and skin color, reaching 19.67% and 7.52% respectively!

This means that driverless pedestrian detection systems are more difficult to distinguish between children and darker-skinned people, who will also be at greater risk.

Moreover, it is particularly noted that these figures have increased to a certain extent at night, with the EOD (difference between children and adults) of children from day to night, the rate of misconduct increased from 22.05% to 26.63%, and the difference rate of skin color group (dark and light skin) increased from 7.14% during the day to 9.68% at night.

In addition, compared with men, the failure rate of women is higher than that of men in the three factors.

In addition, the research team studied the data under different brightness and contrast, and these variables also had a great impact on the failure rate.

Source: among the 8 pedestrian detection systems selected in this paper, with the decrease of brightness, the performance of the primary detection system is the worst, especially in the skin color, the difference between dark skin and light skin is the highest.

"A fair AI should treat all groups equally, but that doesn't seem to be the case with driverless cars at the moment." Said Dr Jie Zhang, author of the study.

Why did this happen?

This is mainly because artificial intelligence systems need a lot of data training, and once these data are insufficient, they will inevitably be reflected in the performance of artificial intelligence. That is to say, the lack of training data leads to some bias in some artificial intelligence AI.

02. There are still many unsolved problems. In fact, there is a certain degree of unfairness in artificial intelligence systems, and it is not the first time for researchers to study it.

As early as 2019, a study by the Georgia Institute of Technology showed that people with darker skin on the road were more likely to be hit by driverless cars than those with whiter skin. The researchers analyzed how driverless cars monitor objects. A total of 3500 photos of people with different skin colors were analyzed.

In the end, it is concluded that the accuracy of self-driving technology in identifying dark-skinned people is 5% lower on average.

Source: Cruise, although these studies do not cover self-driving cars that are already on the road, they will undoubtedly make people more alert to self-driving technology.

A large part of the difficulty of self-driving landing is that it cannot really replace the timely response of human beings to pedestrians and road conditions.

In 2018, a driverless car owned by ride-hailing service giant Uber crashed to death in Tempe, Arizona. It was the first self-driving accident, and "too late to respond" was a big problem.

Some time ago, California voted to allow two self-driving taxis, Cruise and Waymo, to operate 24 hours a day in San Francisco, which caused dissatisfaction among the American public because driverless taxis often cause accidents.

Source: the driverless system of Cruise vehicles can identify road conditions in many ways, such as lidar installed on the roof, which can produce three-dimensional images of the environment around the car many times per second, mainly using infrared laser pulses to reflect objects and transmit signals to sensors, which can detect stationary and moving objects.

However, in extreme weather, such as thick fog or heavy rain, the accuracy of lidar will be greatly reduced.

Short-range and long-range optical cameras can actually read signals, judge the color of objects and other more detailed objects, and can make up for the shortcomings of lidar.

In order to increase the recognition ability, many domestic self-driving systems adopt hybrid perception route, which is realized by lidar and camera vision technology, and visual perception takes precedence over radar perception, mainly visual perception, supplemented by radar perception.

But Tesla is a big fan of "pure visual perception", and Musk once said that lidar is like a human appendix. However, this also caused Tesla to go to court many times because of the accident.

Source: Tesla actually needs to overcome many challenges even if he is a mixed perceptual route.

For example, pedestrians who take long-distance images usually have small targets, which leads to low resolution and low positioning accuracy, which is one of the reasons for the high rate of misdetection in children. Secondly, different pedestrian posture will also lead to inaccurate algorithm detection, and pedestrian detection will be affected by the background, such as the intensity of light, changes in the weather will affect the judgment.

Finally, there are the causes of obstacles, target overlap and occlusion also have a great impact on the algorithm recognition.

03. Chinese scholars led the study of the paper on the fairness of the autopilot system, entitled "Dark skin faces more risks on the Street: exposing the Fairness of the Autopilot system", published in the journal New Scientist.

Source: the New Scientist thesis Research team is from King's College London. There are six authors listed in the paper, including Xinyue Li, Ying Zhang, Xuanzhe Liu, Peking University in China, Zhenpeng Chen and Fedry Cassaro from the University of London, and Jie M.Zhang from King's College London.

Jie M.Zhang, currently an assistant professor at King's College London, focuses on combining software engineering research with artificial intelligence research to improve the credibility of the software. She was a researcher at the University of London and received a doctorate in computer science from Peking University in China.

As a scholar of Chinese nationality, Jie M.Zhang 's achievements in China are also commendable. in March this year, she was named one of the "Top 15 Young Chinese female Scholars". She has also been invited many times to give keynote speeches on the credibility of machine translation, and she and her team have also conducted research and analysis on the learning ability of artificial intelligence for many times.

With regard to the lack of fairness in pedestrian detection systems, Jie M.Zhang said automakers and the government need to work together to develop regulations to ensure the safety and fairness of autopilot systems.

In fact, there have been artificial intelligence recruitment software and facial recognition software before, and black women are not as accurate as white men, but now the consequences of self-driving cars may be more serious once there is a recognition misunderstanding.

In the past, ethnic minorities may have been deprived of their due convenience because of some software. Jie M.Zhang says they may now face more serious injuries, even personal injuries.

This article is from the official account of Wechat: Superelectricity Lab (ID:SuperEV-Lab), author: Cao Tingting

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.