Xperi Corporation – In-car safety systems a priority for new vehicle buyers

Recent research confirms the importance of in-car systems that are able to assess the safety status of vehicle drivers and all occupants (pets/children/passengers) in order to trigger alerts to prevent the development of dangerous situations.

The study was part of a national CARAVAN® survey commissioned by Xperi and conducted by ENGINE Insights. It identified car owners’ interest (especially Gen Z and Millennials) in a vehicle future made possible by secure in-vehicle computer vision technology that provides increasing personalization of the in-cabin experience. Some 98% of respondents said that safety informed their choice of vehicle.

Other features that would attract sales include in-cabin sensing of the vehicle occupant/driver to automatically customize playlists, lighting and HVAC systems, hands-free selfies and pay-on-the-go capabilities. “The safety benefits for drivers and occupants, especially children, of secure in-vehicle sensing are profound, and these survey results are testament to that. Supported by global safety legislation, we are focused on innovating computer vision technology to make a safer in-cabin vehicle experience no matter where in the world car buyers are,” said Jeff Jury, Xperi SVP and general manager, Connected Car.

Some 86% reported that their choice of vehicle would be influenced by the inclusion of a safety-focused computer vision system. This was particularly true of younger consumers: Millennials (92%) overwhelmingly said they were more likely to purchase a vehicle model/brand if it had a computer vision system that could detect children/pets left inside a hot vehicle, seatbelt status, sudden sickness, driver drowsiness and other driver distractions.

In addition, the survey found that consumers also look for personalization features such as automatically adjusting infotainment, and wheel, and chair positions.

Automotive Industries asked Jeff Jury whether the findings of the survey were aligned with the known market trends.

Jury: They were aligned, but also hinted towards a potentially interesting new market direction. We all know how important safety is, for both driver and passengers: Road traffic crashes are a leading cause of death in the U.S. for people aged 1 to 54 and kill 1.35 million people globally each year.  So, the importance of in-vehicle driver and occupancy sensing systems in preventing accidents has become a focus both in the US and abroad – and an increasing imperative for automakers, spurring a series of actions globally to encourage automakers’ implementation of driver and occupancy sensing systems, including the U.S. Infrastructure Investment and Jobs Act, the U.S. Hot Cars Act and the Euro NCAP.

That being said, as the pandemic reinforced the importance of the personal vehicle, increasingly, consumers are viewing the vehicle as a third space – an extension of the office or living room – and in fact as an extension of themselves and their digital identity. So, increasingly, vehicles are not just about the utility of going from A to B, but more about the way you feel while you are going from A to B.

For example, when you go to a restaurant, a coffee-shop, or the movies, it is all about the experience, not about the safety – safety is taken for granted. Likewise, with new driver and occupancy sensing technologies, which have the capability to provide both the very safe and the very elevated in-cabin experience that consumers want, safety can be taken for granted in the same way you would assume the car you just bought has state-of-the-art brake pads and airbags.

AI: Does DTS AutoSense meet the safety expectations of respondents?

Jury: We know from our research that safety ranks number one in importance to car buyers and owners and in-cabin safety is our number one priority. While there is no official “rule book” yet, various regulatory bodies across the world such as NHTSA in the US, or NCAP and CLEPA in Europe, are focusing initiatives and regulation on the kind of in-vehicle safety technology that DTS AutoSense™ provides (and, in some cases, is the only one to provide). We are pleased to be helping lead the way with our technology and are committed to advancing safety in vehicles in alignment with the driver distraction and passenger safety objectives outlined in the Euro NCAP roadmap published in December 2021.

Xperi pioneered in-cabin sensing (ICS), an Advanced Driver-Assistance System (ADAS) that

combines computer vision technologies with computing devices that enable cars to detect changes in drivers and occupants – their states, actions, and intentions – to create safer and richer journey experiences. We believe that ICS systems such as DTS AutoSense will become standard in the vehicles of tomorrow and can form the basis of new motor vehicle safety standards required by Euro NCAP.

We also have achieved ISO9001 and ASPICE (Automotive Software Process Improvement and Capability) Level 1 certifications. The ISO9001 standard is based on quality management principles including a strong customer focus, leadership, process approach, and continual improvement. ASPICE is a process model that defines best practices for software and embedded systems development for the automotive industry.

And, what is perhaps most exciting about this new technology is that while car owners today may not be able to imagine what this safety technology means for elevating the experience inside the cabin, the future possibilities are extraordinary.

AI: How important is a computer vision system in the buying decision?

Jury: Very important!! 86% of car owners surveyed said that their likelihood to purchase a vehicle would increase if it had a safety-focused computer vision system.

That survey also uncovered car owners’ interest (especially Gen Z and Millennials) in a vehicle future made possible by secure in-vehicle computer vision technology that provides increasing personalization of the in-cabin experience. From sensing the vehicle occupant/driver to automatically customizing playlists, lighting and HVAC systems – to hands-free selfies and pay-on-the-go capabilities.

Millennials (92%) overwhelmingly say that they are more likely to purchase a vehicle model/brand if it has a computer vision system that can detect children/pets left in a hot vehicle, seatbelt status, sudden sickness, driver drowsiness and other distracted driving indicators that trigger alerts to the driver.

The number of accidents, and associated casualties, generated by any form of distraction or by leaving a child or a pet behind in a hot car, has made car owners especially conscious and interested in technologies that contribute to in-cabin safety.

For example, 90% positively viewed a vehicle feature that could sense, in real-time, driver and occupant immediate health, and act on it to prevent accidents. Over half would like a sensing system that adapts their vehicle interior to driver/occupant preferences, such as wheel and chair position.

Just as appealing to respondents is a feature that detects if a child is present in the car to personalize in-cabin infotainment to be appropriate, chosen by 52% of respondents. 37% would like a feature that identifies, and reacts to, driver emotions to adjust settings, such as infotainment and lighting.

Unsurprisingly, child safety is a key priority for car owners. In-vehicle computer vision systems that detect if a child is present and properly secured in the vehicle – and then trigger alerts to the driver – are considered beneficial by nine out of ten adults. Seven in ten ranked the ability to detect a child left in a hot vehicle and trigger alerts as their top, or second most important, feature enabled by computer vision systems. Pet safety (alerts if left in a hot vehicle), while carrying less weight than children, is of major importance to over a third of respondents (36%).

AI: What is the role that privacy plays when it comes to in-cabin technologies?

Jury: Privacy is critical to in-cabin sensing technology because of its link to a camera. Fortunately, technology also provides the solution to privacy. Xperi’s product designs have privacy at their core. ‘Privacy by definition’ is key in enabling personalized experiences through sensing technologies, while eliminating the distrust associated with the concept of “monitoring.”

This can be deployed through edge computing, where data is processed and stored temporarily by, or near, the devices that collect it. The data stays within the vehicle, and is not transmitted to the cloud or stored or processed by a remote server. In an in-cabin sensing system, edge computing means improved consumer privacy, enhanced data security, faster response times, and reduced power and bandwidth usage – and it can function without an internet connection.

In 2021, Xperi launched the first of its kind neuromorphic in-cabin sensing system to explore further the “privacy by definition” concept for all vehicle occupants.

AI: From concept to the computer vision enabled vehicle, what does the journey look like? 

Jury: There are many parameters that need to align perfectly to ensure a successful journey. Research, data acquisition and marking, neural networks, engineering and processes that go hand in hand are all key to enabling the features that define DTS AutoSense.

DTS AutoSense has a foundation of over 1,200 computer vision/ML patents. At Xperi, we combine the creativity of our researchers, with the rigor of automotive standards, to implement robust technologies that enhance the safety and experience inside the cars of tomorrow.

At the core of DTS AutoSense in-cabin solutions – and any next generation computer vision / machine learning system – is data: from in-cabin data acquisition to synthetic data capture to mapping the complex motion in 3D of a given object or data annotation.

Data can be static (stills) or moving (videos). Our motion capture efforts enable a wide range of realistic video production, both in-car and for other use-cases. Motion capture complements our hand-crafted and AI-enhanced software animation solutions. Like scanning, this method provides the algorithm team with ‘real-as-it-gets’ performance.

Our success is directly related to the way we acquire, mark, test and use data for training. This means meeting both quality and diversity asks/requirements to satisfy our internal or external clients. We work exclusively with custom systems that are fine-tuned, especially for the project-to-project requirements, assuring maximum flexibility and reaction speed.

Our engineering is defined equally by the way we develop software and by the tools we use to develop it. Our goal is to deliver the highest quality code for production. We have developed a specific way to approach low-cost platforms that allows us to use all the resources available on that platform.

There are two areas of focus for our processes: One is centered around compliance through the quality management system, with an emphasis on integration of business processes; the other is focused on engineering best practices and automotive standards compliance, such as automotive SPICE (Software Process Improvement and Capability).

The reality is that there is no ‘one size fits all’ deployment, and rather than the journey being a single one, it is a multitude of individual ones. Each deployment is unique and each feature set needs to consider both the platform specifics and the needs of the partner.

We take great pride in our approach which is centered around flexibility and always focuses on the best way to understand, improve, adapt, test and deploy features that will make a difference for both the OEM as well as the end-user.

AI: When will see the first vehicles with DTS AutoSense?

Jury: Our DTS AutoSense Driver Monitoring System has been on the road since 2019 while our world’s first Occupancy Monitoring System (OMS) hit the road in 2021 – including a super set of technologies which contains occupant detection, emotion detection, child and child seat presence detection as well as pet detection.

Our OMS uses a single camera and leverages our extensive experience with image processing (20+ years) and artificial intelligence. Its advanced computer vision and machine learning techniques enable vehicles to sense, in real time, the presence of occupants and objects (for example, a laptop accidentally left in the vehicle). The technology can also enable personalization of infotainment recommendations, such as customizing playlists and content, volume of music, choice of radio station options, in-cabin temperature adjustments or any setting that can be adapted to a user’s specific taste.

The solution is deployed using edge computing, without a need for cloud connectivity, meaning it is designed to enable all data to remain within the vehicle. Powered by artificial intelligence, each feature relies on proprietary neural networks designed, trained, and tested by our team of over 100 AI/ML engineers, artists, scientists, and problem solvers.

AI: How do you envision the in-cabin experience of tomorrow?

Jury: Because computer vision and sensing capabilities can also be combined with entertainment, we see a future where this technology can help define preferences for the driver and passengers – and go many steps further – creating an even better, and safer, in-cabin future. DTS’ in-cabin products, including DTS AutoStage, which combines over the air broadcast with IP-delivered content for a robust, rich, personalized in-cabin infotainment experience, is well-aligned to deliver just that.

We believe that tomorrow’s vehicle can become a (very safe) third space rather than just a transportation device – with the cockpit a place of relaxation, where content consumption (music, movies), working or socializing can take place – one that is personalized and allows a seamless transition from ‘at-home’ to ‘in-vehicle,’ with the vehicle sensing all the music and comfort preferences of the driver/passengers, then automatically adjusting.

Taking this to the next level, we envision never before seen use-cases that could monitor driver/passengers’ wellness, health parameters and state of mind, all with privacy by definition as a critical component.

As vehicle automation and self-driving vehicles advance, the interior of the vehicle can transform into a mobile conference room, or an extension of the driver/passenger’s mobile device, enabling them to generate and interact with social media content. Imagine passengers watching cinema quality (audio and video) movies while on the go, or gaming with friends while casting their gameplay on the social media platform of your choice?

Xperi enables extraordinary experiences, and this is what we will do for the in-cabin space with our innovative technology.