Multi User Scenario Catalogue for Connected and Autonomous Vehicles (MUSICC)
As connected and autonomous vehicles (CAVs) come closer to reality on public roads, attention is turning to how we can validate these vehicles are safe before they are made available. MUSICC provides a way of sharing a library of scenarios against which vehicles can be tested.
The 2019/2020 MUSICC project has now been completed, but the code is still maintained by CPC and we are still active in the area of CAV certification. If you have feedback or questions about MUSICC, or would like to be involved in future work in this area, please send an email to:
musicc-support@cp.catapult.org.uk.
Publications
A live demo of the MUSICC system, containing a small number of example scenarios, is available to interested parties. Documentation can be found in the GitLab repository linked to below.
The MUSICC system has been released as open source software, made available under the AGPL licence.
As a result of this release, you can install a ‘private’ instance of MUSICC within your own organisation. This means you can store and manage your own test scenarios, without them ever leaving your corporate firewall, and still have the ability to easily synchronise your private instance with the regulatory scenarios from the master instance.
The release is intended to allow the community to contribute to the development of MUSICC, encourage the industry to converge on a common scenario format, and demonstrate MUSICC’s effectiveness as a scenario management tool.
Pass-Fail Criteria for Scenario-Based Testing of Automated Driving Systems
Scenario testing for an Automated Driving System must include a highly automated means of evaluating test results, as manual review at the scale required is impractical. This paper proposes a framework for assessing an ADS’s behavioural safety in normal operation.
Design considerations for ODD ontology
MUSICC currently supports a basic form of ODD description in the form of a search query. However, it has become clear that this simple approach will not support all likely use cases, and we believe that development of a more sophisticated language needs to be tackled by the community. This document sets out some of the design considerations for this language and is intended to help inform development efforts.
Background
As automated and highly-automated vehicles (HAVs) are gradually deployed on public roads, regulators globally are faced with the challenge of ensuring that these systems are safe for the public that are exposed to them. This challenge requires coordination of validation activities between national regulators and system developers – with common (or at least aligned) approaches to validation desirable.
One of the big challenges for regulators is achieving enough coverage of real-world situations to know that Automated Driving Systems (ADS) are acceptably safe. Automated Driving System (ADS) developers are now indicating that testing by driving vehicles on public roads alone is not sufficient, due to:
- The large number of miles needed to ensure sufficient coverage of the situations which will be encountered by a fleet of HAVs
- Issues over the lack of control of the test parameters
The solution is likely to involve more than one technology, but will almost certainly include an element of simulation.
Aims
MUSICC is a system to store and share a library of scenarios: in future, an ADS may be expected to demonstrate its performance in these before release to market. As far as we know, MUSICC is the first project of its type to build a proof of concept system specifically designed for regulatory use. This gives it a strong potential to become a de facto standard. MUSICC will help to shape the future direction of the industry, positioning the UK as the “go-to” place for Connected and Autonomous Vehicles certification testing backed by government.
Organisations involved
The MUSICC development project was led by a regulator (Department for Transport) and an impartial, neutral mediator (Connected Places Catapult). The work has been influenced by a highly credible industrial advisory group, including representatives of several major OEMs, tool providers, ADS developers, research and innovation organisations, CAV testbeds and insurers.
As part of MUSICC, Connected Places Catapult has contributed to the development of:
ASAM OpenSCENARIO version 1.0, a standardised file format for scenarios
BSI PAS 1883, which defines a taxonomy for describing ODDs
MUSICC symposium, 24th June 2019
The Multi User Scenario Catalogue for Connected and Autonomous Vehicles (MUSICC) Symposium brought together industry experts who share an interest in the verification and certification of CAVs / Highly Automated Vehicles (HAVs), to inform the direction of the MUSICC project.
List of speakers (with slides where available):
Keynote speech by Donald Macdonald (Department for Transport)
Connor Champ (Law Commission) The challenges of regulating automated vehicles
Zeyn Saigol (Connected Places Catapult) Vision for MUSICC
Iain Whiteside (FiveAI) Towards a Digital Highway Code
Tim Edwards (Horiba MIRA) The role of physical testing in the CAV engineering lifecycle
Alex Grandjean (AV simulation) AV systems validation using SCANeR™studio simulation SW
MUSICC Symposium 2019 – Outcomes from interactive sessions
Further Reading & Listening
