ADAS systems need “eyes” that see what is happening around the car and collect that information so that the vehicle’s processors can perform reliable recognition of the environment and act accordingly.
Those “eyes” are cameras and sensors, most of which are mounted on the windshield. Do not forget that when you replace a windshield, you have to remove the cameras from the broken glass and mount them on the new one.
Once installed, these sensors need to be recalibrated to ensure they point exactly where they need to, work with the highest precision and provide the correct information.
But how does a car really see the world around it, what is the process that allows it to identify, for example, a pedestrian crossing the road? Xavier Martínez, explains it to us.
“The front camera that is installed in the windshield has a key role and allows the car to detect objects and people reliably at all times, through classic image processing, combined with artificial intelligence methods. For it to be useful and to help the driver, it has to be faster than the driver in detecting and alerting obstacles”, says Xavier Martínez.
To get a car to make an efficient and fast recognition of risk situations in its environment, this process is carried out through three routes.
This multi-route approach seeks, on the one hand, to achieve the highest recognition speed, because every tenth of a second that passes, the car advances a distance; which is achieved with the least generation of data and calculations possible.
Keep in mind that a 1MP RGB camera running at 15 FPS generates approximately 59 megabytes per second to process.
On the one hand, the camera recognizes, through pre-programmed algorithms, the appearance of categories of typical objects such as cars, motorcycles, pedestrians, cyclists or road markings.
For each object to be detected, traditional computer vision algorithms are elaborated by hand, which analyze the directions of the edges within an image to try to describe objects.
On the other hand, the camera uses the SfM (Structure from Motion) movement structure, to recognize objects by tracking the movement of the associated pixels.
The role of artificial intelligence
The third route is carried out with artificial intelligence, with deep learning techniques through the use of convolutional neural networks (CNN), which manage to perform a semantic segmentation of the elements of an image.
The CNN is a type of artificial neural network with supervised learning, which mimics the vision of the human eye to identify objects through their characteristics. This neural network applies hierarchical layers to an image. The first layers detect the simplest elements, such as lines and curves, which gradually become more specialized until they recognize more complex shapes.
This neural network has been trained with tens of thousands of images (in the case of ADAS systems, from all kinds of traffic situations) to recognize the unique characteristics of each object and to be able to generalize it.
Thanks to this, it can recognize a person as a pedestrian, regardless of their posture (standing, sitting, facing to the side…), position (standing, jumping, walking…), clothing…
The great advantage of artificial intelligence is that it is not programmed, it is trained. Object detection is not done through manual coding of an algorithm, rather deep neural networks allow the characteristics of an object to be automatically learned from training examples.
This allows the car to detect, for example, vehicles parked at “strange” angles or pedestrians with parts of their bodies hidden between cars. It can also differentiate the road surface, when lines are not visible, by “reading” other indicative features such as changing color and surface from asphalt to gravel, vertical markings, or parked vehicles.
Source: Carglass