Illustration of a yellow car using advanced sensors and radar technology to navigate and detect other vehicles on the road at night.

reports

Getting safety metrics right for connected and autonomous vehicles

Connected and Autonomous Vehicles (CAV) will represent a step-change in the complexity and sophistication of vehicle technology. But to enable their economic benefits to be realised, while maintaining public confidence in safety, assurance processes will need to be altered.

May 11, 2021

The CertiCAV project, led by Connected Places Catapult on behalf of the Department for Transport, has proposed a technical approach for assuring the safety of these vehicles.

This report discusses the challenges and practicalities of implementing methods to test an Automated Driving System (ADS) in a range of simulated scenarios. It relates to the CertiCAV Software Framework, a proof-of-concept tool developed by CertiCAV – which has been released as open-source code.

The preceding work on CertiCAV explored important areas of safety assurance. This included defining a set of high-level driving performance criteria and proposing a structure for requirements definition (to ensure that requirements are measurable and useful). The outcomes of this work package were taken as the starting point for the software framework report and for framework design in general. Analysis in the report covers:

The role of automated testing and its limitations
Examples of tests which could be used to demonstrate that certain types of requirement are met
The key challenges to be overcome when implementing different types of test
Design of a software framework capable of downloading scenarios from a database, running them in a driving simulator, and evaluating the results
Storage of simulator data output in a standards-aligned format

This report was written for the Department for Transport’s International Vehicle Standards team. It is likely to be of interest to others involved in designing, specifying, testing or assuring Connected and Autonomous Vehicles.

“It is becoming clear that collisions or even TTC (time-to-collision) are, by themselves, inadequate metrics for assessing the safety of autonomous vehicles. While stakeholders are starting to explore more sophisticated metrics, you get a deep understanding of the issues by actually implementing them, and this is what we’ve done and reported on here.” Zeyn Saigol, Principal Technologist at CPC and Technical Lead for CertiCAV

* The CertiCAV project, and the document shared here, is intended to be a thought-piece on the challenges of automated vehicle safety. The document does not intend to be exhaustive, present authoritative recommendations or final solutions. Where assumptions have been made, for the purpose of project delivery, these have been set without prejudice to alternatives. This document does not necessarily represent the views of the Department for Transport or any UK government body.