Building advanced human-machine-interfaces with in-car haptic technologies

Car-makers are increasingly incorporating haptics (touch feedback technology) into automotive touchscreens and touchpads. Drivers and passengers need realtime effective feedback when accessing in-vehicle entertainment systems, which often have flat screens. Haptics brings the sensation of the real world, such as buttons and knobs, to the car interface, according to Immersion Corporation, one of the leaders in haptic technology. Immersion has been the leading innovator of haptics for over 25 years.

The company provides technology solutions for creating immersive and realistic experiences that enhance digital interactions by engaging users’ sense of touch. With more than 3,500 issued or pending patents, Immersion’s technology has been adopted in more than three billion digital devices, including mobile, automotive, gaming, medical and consumer electronics products. In April 2019 the Silicon Valley-based company signed a license agreement with Alpine Electronics, enabling Alpine to access to Immersion’s patented haptic technology.

This is the latest in a series of licensing agreements, with Tier 1 suppliers, including Seoyeon, Preh, Toyodenso, Calsonic Kansei, Panasonic, Bosch and Marquardt. “Touchscreens are becoming more intuitive with the power of touch, and we look forward to seeing how these marquee companies embrace haptics to enhance their in-vehicle systems,” said Chris Ullrich, Immersion’s VP of Technology. “The power of touch elevates the driving experience.” As the automotive industry transitions to autonomous cars, drivers and passengers alike will interact with in-vehicle human-machineinterfaces (HMIs) in new ways. Already, there are some cars that dedicate a large portion of their dashboards to HMI systems. In the future, new haptic technologies will be needed to enable surfaces that are not only bigger, but also conform to a certain shape to display dynamic content. Future automotive controls may well be programmable, and surfaces could feature on-demand controls depending on the activity performed in the car. New technological developments in automotive, as well as entertainment content and gaming will lead to new ways that people interact with their systems and devices. These interactions might only be possible through new haptic use case models. For example, new actuation technologies to create small, flexible, and low-power haptic solutions are needed for devices that a user can wear as part of their everyday life and work as a feedback device (ring, band, watch, etc.).

This can also enable handheld devices that are power efficient and not only provide vibration feedback, but also temperature, deformation, or even force-feedback. “The haptics market is still in its infancy and our touch technology is just beginning to emerge in the marketplace. It has a unique capability to bring people closer, create immersive and intimate experiences, and enable our customers’ products to be more engaging,” says Immersion. There is not a single “killer application,” but rather a wide range of interactions that benefit from haptic feedback. “The question is how fast these technologies can be created and be ready for prime time. In fact, this is the challenge that everybody working on haptics faces today. It is a great quest motivated by our knowledge that, through the power of touch, haptics can make an impressive impact on our lives,” says the company.

Automotive Industries (AI) asked Ullrich what applications designers are most interested in using. Ullrich: The most consistent ask that we get from Tier 1s and OEMs is around haptic touchpads. For a long while, the trend was toward adding haptics to the touchscreens found in most premium and mid-tier vehicles, but automotive HMI trends have largely moved to teleoperation style interfaces. We see this in the form of touchpads, rotary controls or other novel controller style interfaces located close to where the driver’s hand is located during driving. Touchscreens are largely for passenger usage. The other big trend we see is the general replacement of buttons throughout the vehicle by capacitive touch surfaces which are then haptically-enhanced. Recent vehicles like the 2019 Audi A6 show this trend clearly with several button clusters being replaced with touch surfaces and haptics. In general, the haptic feedback on these surfaces is primarily in the form of confirmation, similar to the feedback from a standard mechanical button. However, we also see some interest in gesture interactions that have haptic textures or other dynamic effects associated with them. This type of feedback is particularly useful for interactions where the driver is navigating through a list or adjusting volume or temperature and the design goal is to provide meaningful feedback about the interaction.

AI: How do you see haptics changing in-vehicle experiences? Ullrich: We see a lot of possibilities in the next generation of touch interfaces for operations such as cruise control, stability control, entertainment, navigation, and climate control. We’re working with automotive suppliers and OEMs to create unique user experiences that are intuitive and make vehicles safer to operate. Primarily, I see buttons being replaced by capacitive sensors on the dashboard, door panel and primary controls. These sensors can be very distracting and confusing for the driver if there is no feedback when touched. Haptics is a natural type of feedback to provide in these instances – it’s clear, it generates confidence, and interference is low (unlike audio feedback). By moving to these types of interactions, HMI and interior design staff become unencumbered by the mechanical constraints imposed by historical button clusters. It’s possible to integrate control functions into nearly every surface and to utilize these control functions in ways that mechanical buttons simply cannot support. Gesture and pressure driven interactions are more functional for the driver once there is an appropriate haptic confirmation or other tactile response to guide the user through the interaction. Change the radio station while driving? With capacitive sensors, you’re just “touching glass.” With haptics, the glass comes alive and provides feedback, providing confirmation the system acknowledged your request and will make the adjustment, i.e. change the channel.

AI: What are the emerging technologies that will impact automotive OEMs? Ullrich: There’s a consistent trend in automotive interior design toward seamless surface interiors that can serve multiple functions. These design aspirations are likely to be enabled by the adoption of new driver sensing technologies and new surface sensing and display technologies. In addition, voice driven intelligent assistant technologies are getting increasingly sophisticated and becoming more capable of usefully supporting a driver.

AI: What technologies is Immersion looking at next for the automotive industry? Ullrich: From a haptics perspective, existing commercial embodiments typically rely on electromagnetic solenoid style actuators to generate feedback. Although these do provide a type of confirmation haptics, they are both inelegant and do not have high fidelity for creating truly realistic sensations and textures.

One way that these limitations can be mitigated is to integrate haptic motors that are related to the highly refined linear motors found in premium mobile phones such as the iPhone or Pixel. These high-end linear motors can create very strong sensations that feel identical to mechanical switches as well as create rich textures. Beyond this, new haptic technologies such as piezoelectric actuators will enable high fidelity haptics in form factors that directly replace existing solenoid style solutions with very high accelerations and ultra-wideband response – able to synthesize rich surface textures and sharp mechanical feeling clicks. As Tier 1s and OEMs begin to realize the expressive design capabilities of these new actuator technologies, vehicle HMI designers will begin to deploy highly intuitive and realistic interactions that seamlessly integrate gesture, voice and texture to enable intuitive and luxurious interfaces.

AI: How enthusiastic has the automotive industry been about adopting haptics in cars of the future? Ullrich: In the premium segment, where Immersion has typically operated, there is a significant growth trend toward usage of haptics in the automotive HMI. Since 2000, Immersion has led the market in thinking about how haptics might benefit vehicle interfaces. We see this increasing as the technologies that enable haptics in vehicles become more standard and costeffective to utilize not just in premium vehicles but in mid-range vehicles. High-quality haptic feedback is a cost-effective way for OEMs to enable advanced functionality and industrial design in a way that creates delight and confidence for their customers.