The problem with self-driving cars is that they cannot detect pedestrians if they are children or people with dark skin.

The future of the automotive industry is moving towards autonomous cars. And we already have some glimpses of it in the present; either as a robotic taxi like Waymo or Cruise in the US, or with systems like Tesla’s fully autonomous driving or Ford’s Blue Cruise. However, a major problem has been found in technologies that use artificial intelligence to enable hands-free driving: pose a danger to children and black people.

The paper, signed by six researchers from Peking University, King’s College and University College London, came to a disturbing conclusion. Driving systems for unmanned vehicles with artificial intelligence have a bias in detecting pedestrians. What does it mean? That its accuracy is lower when the street is trying to cross, as we have said, children or people with a darker skin tone.

Experts argue that this problem is much more important than is commonly believed, because it has not been studied as widely as others related to artificial intelligence. Studies show that self-driving vehicles are 19.67% more accurate in detecting adult pedestrians compared to children. While when comparing different skin tones, the difference in the effectiveness of the systems is 7.52% in favor of lighter ones.

AI of self-driving cars has a bias affecting children and black people

Expert research based on four datasets which have been “thoroughly studied” to evaluate the performance of pedestrian detection in autonomous vehicles. These are CityPersons, Berkeley Driving and EuroCityPersons; the latter is divided into two parts: one with images taken during the day and the other at night. It is worth noting that the information was collected from 8 advanced detection systems.

Thus, the researchers gained access to more than 8300 images taken by the cameras of unmanned vehicles in real traffic situations. But they did not stop there, as they manually classified the information in order to have more accurate material. For example, managed to mark over 20,000 ages and 3,500 skin tones. They were also able to indicate the gender of the pedestrians. Although they came to the conclusion that AI does not have serious shortcomings in identifying passers-by by the specified parameter.

Another interesting point is that we studied situations recorded at different times of the day and in different climatic conditions. However, the latter had to be limited to general scenarios—with or without rain—because not all datasets contained images of the environment with snow or fog. Curious that the rain negligible and statistically insignificant impact in detecting problems of artificial intelligence used in autonomous vehicles.

The problem of artificial intelligence escalates at night

AI offset that self-driving cars use to detect pedestrians worsens at night. And also in scenarios where the brightness or contrast is low. If we talk about children, the data is very disturbing. The difference in failure rates between adults and children increases from 22.05% (daytime) to 26.63% (nighttime).

The situation does not improve with day and night failures between people with fair skin and people with darker tones: they increase from 7.14% to 9.68%, to the detriment of the latter. Summing up, then, Pedestrians are more likely to be hit by self-driving cars when they are children or people with dark skin. A hazard that increases as the sun sets or lighting conditions deteriorate for cameras built into vehicles.

As artificial intelligence becomes more and more integrated into our daily lives, experts’ words should not be taken lightly. After all, the problems are already in sight when self-driving cars still make up a minimal part of the world’s car fleet.

Unless the engineers developing these systems take immediate countermeasures, the risk of death for bystanders can be greatly increased. But not only that. Automotive manufacturers may also be subject to scrutiny by legislative and regulatory scrutiny, hindering or stopping the development and deployment of technology that was supposed to theoretically reduce the number of road traffic accidents.