A system to track a driver’s activity based on image data that could be used to determine how quickly they could take control of an autonomous vehicle has been developed by researchers at the Fraunhofer Institute.
While vehicles with limited autonomous functionality are already hitting roads in the US, the technology is not mature enough to completely do away with human drivers. Current rules mandate that a driver must be ready to take control of an semi-autonomous vehicle within a moment’s notice.
While driver monitoring systems already exist, modern examples are typically limited to detecting driver fatigue and make minimal use of camera image data.
“Our technology not only enables us to identify facial features but also the current poses of the driver and the passengers,” said Fraunhofer researcher Michael Voit. “We can then determine from these poses what the driver and passengers are currently doing.”
The team developed machine-learning algorithms that can analyse the camera data in real time to find out whether the driver is on the phone, playing with children or looking at a smartphone.
The technology is capable of going beyond simple image recognition and can interpret the activities in context.
The researchers first trained the system by manually annotating numerous camera shots with data such as the location of people’s hands, feet and shoulders as well as the location of objects such as smartphones, books and other items.
The system abstracts images of the driver or passengers to form a digital skeleton – a type of stick figure that replicates the person’s body poses. It then deduces a driver’s activity using the skeletal movement alongside object recognition.
“The algorithms can thereby tell whether someone is sleeping or looking at the street, how distracted the person is and how long it will take them to focus back on the road,” Voit explained.
The system supports both traditional video cameras and infrared cameras that can see in the dark, as well as 3D cameras that measure the distance between objects and the camera. The system also gives interior designers freedom in terms of camera placement.
“We can not only detect the activities of the driver but those of all passengers, too – both in the front and back of the vehicle,” Voit added, while noting that the system is ready for pilot production. “We are already in contact with companies who want to use our technology.”
Pascal Birnstill, a senior Fraunhofer scientist, said the system had always been designed to prioritise data protection and security. “The camera data are analysed in real time, not saved, and do not leave the vehicle at any point,” he said. “Personalised models are also not needed for this, so no personal data is collected.”
New EU regulations will soon mandate driver monitoring in automated cars, regardless of their level of automation.
A new report from the IET has recommended that a road crash investigations branch is created in the UK as automation becomes more commonplace in the transport sector.
It called on the Department for Transport (DfT) to establish a road crash investigations branch, like current practices in rail, air and maritime in order to contribute to safety improvements as the automotive industry moves towards autonomy.
Other recommendations to improve safety consider the use and storage of data collected by AI systems such as Fraunhofer’s, as well as robust cyber security standards.
Sign up to the E&T News e-mail to get great stories like this delivered to your inbox every day.