June 26, 2020
John Sullivan, Office of Engineering Communications
Using radar commonly deployed to track speeders and fastballs, researchers have developed an automated system that will allow cars to peer around corners and spot oncoming traffic and pedestrians.
Researchers combined artificial intelligence and radar used to track speeders to develop a system that will allow vehicles to spot hazards hidden around corners. Image courtesy of the researchers
The system, easily integrated into today’s vehicles, uses Doppler radar to bounce radio waves off surfaces such as buildings and parked automobiles. The radar signal hits the surface at an angle, so its reflection rebounds off like a cue ball hitting the wall of a pool table. The signal goes on to strike objects hidden around the corner. Some of the radar signal bounces back to detectors mounted on the car, allowing the system to see objects around the corner and tell whether they are moving or stationary.
“This will enable cars to see occluded objects that today’s lidar and camera sensors cannot record, for example, allowing a self-driving vehicle to see around a dangerous intersection” said Felix Heide, an assistant professor of computer science at Princeton University and one of researchers. “The radar sensors are also relatively low-cost, especially compared to lidar sensors, and scale to mass production.”
In a paper presented June 16 at this Conference on Computer Vision and Pattern Recognition (CVPR), the researchers described how the system is able to distinguish objects including cars, bicyclists and pedestrians and gauge their direction and oncoming speed.
“The proposed approach allows for collision warning for pedestrians and cyclists in real-world autonomous driving scenarios — before seeing them with existing direct line-of-sight sensors,” the authors write.
In recent years, engineers have developed a variety of sensor systems that allow cars to detect other objects on the road. Many of them rely on lidar or cameras using visible or near-infrared light, and such sensors preventing collisions are now common on modern cars. But optical sensing is difficult to use to spot items out of the car’s line of sight. In earlier research, Heide’s team has used light to see objects hidden around corners. But those efforts currently are not practical for use in cars both because they require high-powered lasers and are restricted to short ranges.
In conducting that earlier research, Heide and his colleagues wondered whether it would be possible to create a system to detect hazards out of the car’s line of sight using imaging radar instead of visible light. The signal loss at smooth surfaces is much lower for radar systems, and radar is a proven technology for tracking objects. The challenge is that radar’s spatial resolution — used for picturing objects around corners such as cars and bikes — is relatively low. However, the researchers believe that they could create algorithms to interpret the radar data to allow the sensors to function.
“The algorithms that we developed are highly efficient and fit on current generation automotive hardware systems,” Heide said. “So, you might see this technology already in the next generation of vehicles.”
Sophisticated algorithms allow the system to process radio waves (left) to identify potential hazards such as cyclists and pedestrians (right.). Video by the researchers
To allow the system to distinguish objects, Heide’s team processed part of the radar signal that standard radars consider background noise rather than usable information. The team applied artificial intelligence techniques to refine the processing and read the images. Fangyin Wei, a graduate student in computer science and one of the paper’s lead authors, said the computer running the system had to learn to recognize cyclists and pedestrians from a very sparse amount of data.
“First we have to detect if something is there. If there is something there, is it important? Is it a cyclist or a pedestrian?” she said. “Then we have to locate it.”
Wei said the system currently detects pedestrians and cyclists because the engineers felt those were the most challenging objects because of their small size and varied shape and motion. She said the system could be adjusted to detect vehicles as well.
Heide said the researchers plan to follow the research in a number of directions for applications involving both radar and refinements in signal processing. He said the system has the potential to radically improve automotive safety. Since it relies on existing radar sensor technology, readying the radar system for deployment in the next generation of automobiles should be possible.
“It would certainly go through the very rigorous automotive development cycles,” he said. “In terms of integration and bringing it to market, it requires a lot of engineering. But the technology is there, so there is the potential for seeing this very soon in vehicles.”
Besides Heide and Wei, the paper’s authors include: Jürgen Dickmann, Florian Krause, Werner Ritter, and Nicolas Schiener of Mercedes-Benz AG; Buu Phan and Fahim Mannan of Algolux; Klaus Dietmayer of Ulm University; and Bernard Sick of the University of Kassel. Support for the research was provided in part by the European Union’s H2020 ECSEL program.