Harman – Holistic approach to managing driver distraction

Global research has identified driver distraction as a major contributor to road accidents.

In the United States alone, the National Highway Traffic Safety Administration (NHTSA,) estimates that one in 10 fatal crashes involves distraction. Technology can help drivers to remain focused by triggering alerts when their attention is diverted.

HARMAN’s “Ready Care” is a set of solutions – cognitive distraction, stress-free routing, and personalized comfort – that can work together or independently to improve safety and reduce driver stress. Ready Care measures cognitive load for drivers in real-time, responding with tailored interventions to instances of distraction, drowsiness, or anxiety.

It is powered by state-of-the-art machine learning algorithms, which gather and process data from in-vehicle cameras and sensors to monitor a driver’s state, and generate customized in-vehicle cabin prompts to trigger a range of behavioural responses — increasing awareness, alleviating stress or stimulating ease and relief.

Automotive Industries (AI) asked Marcus Futterlieb – Principal Engineer, SBU ADAS, HARMAN what the main driver is behind the Ready Care suite.

Futterlieb: I would say the strongest driver is regulations. The legal requirements for increasing driver safety are becoming so strict that passive systems alone will no longer be sufficient nor compliant. In addition to meeting all these new regulatory demands, we also want to please the end customer with better in-cabin user experiences that live up to their expectations. Large studies on consumer demand in the next five to ten years have found that one of the main requests is digital health measurement and care. Our HARMAN Ready Care solution addresses this.

AI: How does it work?

Futterlieb: With Ready Care, we aggregate input from different sensors that we then use to determine the driver’s state. The core is the near infrared (NIR) camera, operating at around 940 nanometers wavelength, which makes it completely invisible to the driver. A standard RGB imager is sometimes unable to detect key drowsiness measurement parameters at night, which is when the driver is most likely to become drowsy.  By using infrared light, the NIR sensor can reliably provide this information even in low light conditions.

While it is obviously important to detect whether a driver’s eyes are on the road, we use input from different sensors to go beyond this, as eyes on the road does not necessarily mean you are not mentally dis

tracted. Therefore, one of our unique selling points is measuring what we call the cognitive load of the driver. Our camera solution monitors involuntary facial movements, by which it can tell whether you are visually engaged in driving or maybe thinking about your kids, being part of a conference call, or are cognitively engaged in any other task.

In addition, with the acquisition of Israel-based startup CAARESYS in September 2022, we will be able to integrate a radar-based monitoring system to track the health and vital signs of vehicle occupants. This is in particular of advantage where no direct line of site is available between a camera sensor and an object, e.g., a child in a rear-facing child seat.

Another feature of Ready Care is what we call vital sensing based analytics. We can monitor your key vital signs like heartrate and breathing information, with the previously mentioned NIR camera, which is a game changer. From measuring this information we strive to gain insights into, for example, stress levels, so that the car can support you to de-stress. Ready Care can also integrate with any smart devices, like a smart watch or phone, to bring in more contextual intelligence while tracking the car driver’s state.

This information about the driver’s stress level can be fed into our Ready Care stress-free routing feature. This is like another navigation layer, which uses cloud-based information on factors like weather, road construction work or traffic jams to calculate an alternative route that might take you a bit longer to get to your destination but provides a less stressful drive.

AI: What about comfort?

Futterlieb: Ready Care is all about increasing driver and passenger safety. If the system detects that you are getting drowsy, Ready Care can trigger personalized intervention strategies, and may, for example, lower the cabin temperature by a degree or two in order to keep you more alert. Studies have found that the lower temperature will keep you alert for longer.

AI: Do OEMs have to take the full Ready Care package, or can they select individual elements?

Futterlieb: They can select the options they feel their customers want. We obviously have a package that delivers the full feature system. It includes hardware, software and the compute unit. However, t

here are several other business models built around options, where carmakers may choose just the hardware or just the software. Furthermore, there are different software packages and features available.

AI: What has been the OEM reaction to Ready Care?

Futterlieb: We are in constant discussions with our existing but also new customers. I am leading the team that manages the customer demo kits. It is a set of cameras and sensors we can physically carry to our customers so they can experience all the functions in a vehicle of their choice. The initial feedback has been very positive and promising. From an engineering perspective, we have always ranked at the top of the companies in our sector. The more stringent regulations are providing support for the business case as well.

AI: What is next for HARMAN and Ready Care?

Futterlieb: While we demonstrate Ready Care to interested carmakers, we continue to develop the solution with the goal of presenting the next generation at HARMAN’s ExPLORE event in January, coinciding with CES. We have established a very agile way of working and are making significant progress in developing products that deliver on the promise of consumer experiences through automotive-grade solutions today. So,

we have lots of concepts that are going into proof-of-concept phase, and from there into full feature readiness.

Key features of Ready Care

• Eyes and Mind on Road: Ready Care measures both driver eye activity and state of mind. An infra-red global shutter camera with active illumination surveys the driver’s face in low and high light environments to provide real-time data into facial expressions, eye gaze, eyelid openings and more. The HARMAN Cognitive Distraction feature deciphers when the driver is mentally distracted and can initiate multiple and simultaneous in-cabin vehicle intervention responses. It measures real-time driver cognitive load and separates driving tasks from mental as well as visual and manual distractions, enabling tailored intervention strategies to maximize safety.
• Stress-free Routing: Compatible with multiple navigation engines, Ready Care provides an alternate route selection to lower the inevitable driver stress levels while on the road. Through detecting real-time stress factors like traffic jams or weather and adapting accordingly, this solution helps reduce the driver’s state of stress.
• Personalized Experiences: Ready Care leverages advanced machine learning technology to provide personalized intervention strategies tailored to the individual and their driving experience. Ready Care delivers closed-loop interventions via an SDK, allowing OEMs and third parties to integrate any vehicle feature or function – such as HVAC control, media content and volume, or seat sensors – to become part of the intervention solution.