Automated Vehicle Safety Consortium Publishes Best Practice for Evaluating Autonomous Vehicle Safety Performance

The Automated Vehicle Safety Consortium™ (AVSC) announced the availability of a new best practice titled, Evaluation of Behavioral Competencies for Automated Driving System Dedicated Vehicles (ADS-DVs). The new guidelines integrate the AVSC’s Metrics and Methods for Assessing Safety Performance of Automated Driving Systems (ADS) and best practice for Operational Design Domain (ODD) to provide a framework for manufacturers to evaluate the behavioral competency of automated vehicles.

“As automated driving system testing advances, it is important that manufacturers communicate information about the safety of their vehicles in a consistent way, while still using testing protocols specific to their systems and ODDs,” said Amy Chu, director of the AVSC. “With the framework in our new best practice, manufacturers can evaluate the safety performance of their vehicles based on an elemental set of behavioral competencies such as maintaining proper lane position or responding appropriately in work zones. Linking behavioral competencies to a key set of scenarios for a given ODD provides relevant evidence for ADS safety performance and increases confidence in the safety performance of an AV.”

The new best practice provides an approach to specify testable AV behavior by:

* Clarifying a lexicon surrounding automated vehicle behaviors,
* Enumerating an elemental set of behaviors, and
* Demonstrating how to derive metrics to evaluate AV behavioral competence.

Real-world driving involves complex interactions among numerous systems. In order to evaluate as many ADS subtasks within the DDT as possible, developers align them to a generalized set of behaviors. Developers then use system engineering techniques to map these behaviors to an elemental set of behavioral competencies. In other words, competence is considered as a collection of behaviors that cover a context-relevant and predictable part of the broader driving task. Proficiency across a set of behavioral competencies provides directional indication of the ADS-DV safety performance. For example, in the context of driving on a highway while maintaining a lane and approaching another vehicle, the ADS-DV would be expected to keep a safe distance, maintain the speed limit, and monitor the behavior of the vehicle in front of it while preparing to react if the scenario changes.

Human driver tests evaluate candidates against a small number of tasks which represent driving across a variety of different, related scenarios. Although human driving tests reflect a person’s ability to execute key driving tasks and knowledge of driving law, they do not reflect a human’s driving skill in every situation.

“Providing a framework for manufacturers to link behavioral competencies to key scenarios provides evidence for ADS safety performance that manufacturers can use to build their case for safety. This is another important step in engendering public trust and confidence in the testing and deployment of automated vehicles,” added Chu.

For more information about the AVSC, visit avsc.sae-itc.org.

The Automated Vehicle Safety Consortium™ (AVSC) is an industry program of SAE Industry Technologies Consortia® (SAE ITC) building on principles that will inform and help lead to industrywide standards for advancing automated driving systems. The members of this consortium have long been focused on the development of safe, reliable and high-quality vehicles, and are committed to applying these same principles to SAE Level 4 and Level 5 automated vehicles so communities, government entities and the public can be confident that these vehicles will be deployed safely.

SAE Industry Technologies Consortia (SAE ITC®) is an affiliate of SAE International. The SAE ITC team specializes in establishing and managing consortia by providing proven processes, tools and resources. SAE ITC enables public, private, academic and government organizations to connect and collaborate in neutral, pre-competitive forums thus empowering the setting and implementation of strategic business improvements in highly engineered industries globally. http://www.sae-itc.com