Validating Behavioral Requirements, Conditions, and Rules of Autonomous Systems with Scenario-Based Testing

Abstract

Assuring the safety of autonomous vehicles is more and more approached by using scenario-based testing. Relevant driving situations are utilized here to fuel the argument that an autonomous vehicle behaves correctly. Many recent works focus on the specification, variation, generation, and execution of individual scenarios. However, it is still an open question if operational design domains, which describe the environmental conditions under which the system under test has to function, can be assessed with scenario-based testing. In this paper, we present open challenges and resulting research questions in the field of assuring the safety of autonomous vehicles. We have developed a toolchain that enables us to conduct scenario-based testing experiments based on scenario classification with temporal logic and driving data obtained from the CARLA simulator. We discuss the toolchain and present first results using analysis metrics like class coverage or distribution.