Regulating Autonomous Systems: Beyond Standards

2020

You are free to share the material in any form and adapt the material for any purpose providing you attribute the material to the Safety Critical Systems Club (SCSC) Safety of Autonomous Systems Working Group (SASWG), reference the source material, include the licence details above, and indicate if any changes were made. See the license for full details. Cover photo by Markus Spiske, temporausch.com. Obtained from . The Safety Critical Systems Club (SCSC) is the professional network for sharing knowledge about safety-critical systems. It brings together: engineers and specialists from a range of disciplines working on safety-critical systems in a wide variety of industries; academics researching the arena of safety-critical systems; providers of the tools and services that are needed to develop the systems; and the regulators who oversee safety. Through publications, seminars, workshops, tutorials, a web site and, most importantly, at the annual Safety-critical Systems Symposium (SSS), it provides opportunities for these people to network and bene t from each other's experience in working hard at the accidents that don't happen. It focuses on current and emerging practices in safety engineering, software engineering and product and process safety standards. This document was written by the Safety of Autonomous Systems Working Group (SASWG), which is convened under the auspices of the SCSC. The goal of the SASWG is to produce clear guidance on how autonomous systems and autonomy technologies should be managed in a safety related context, throughout the lifecycle, in a way that is tightly focused on challenges unique to autonomy. The document was formally released at SSS '19, 5-7 February 2019.

Autonomous Driving, 2016

Sensor technology and data processing are constantly improving in their performance. This enables both: continuous further development of driver assistance systems and increasing automation of the driving task, right up to self-driving vehicles [1]. In the following chapter the author traces the technical improvements in vehicle safety over recent decades, factoring in growing consumer expectations. Considering Federal Court of Justice rulings on product liability and economic risks, he depicts requirements that car manufacturers must meet. For proceedings from the first idea until development to sign, he recommends interdisciplinary, harmonized safety and testing procedures. He argues for further development of current internationally agreed-upon standards including tools, methodological descriptions, simulations, and guiding principles with checklists. These will represent and document the practiced state of science and technology, which has to be implemented in a technically viable and economically reasonable way. 28.1.1 Motivation In the course of this development, technical, especially electrical/electronic systems and software are becoming far more complex in the future. Therefore, safety will be one of the key issues in future automobile development and this results in a number of major new challenges, especially for car manufacturers and their developers. In particular, changing vehicle guidance from being completely human-driven, as it has so far been, to being

2003

: As unmanned systems become widely employed in various applications, it is critical to have a set of standard definitions and metrics for specifying and evaluating the systems in terms of their levels of autonomy. Developing autonomy levels for unmanned systems is a complex issue that has to take into account many factors such as task complexity, human interaction, environmental difficulty, mission and system dependence, and quality factors. We report on a workshop that addresses this issue.

Autonomous Industrial Vehicles: From the Laboratory to the Factory Floor, 2016

A workshop was held at the IEEE International Conference on Robotics and Automation, called: "Autonomous Industrial Vehicles: From the Laboratory to the Factory Floor". Nine research papers were presented followed by a discussion session summarized in this paper. The workshop findings are intended to be useful for developing standards within the ASTM F45 Committee for Driverless Automatic Industrial Vehicles. This paper provides feedback from the discussion listing the example organizations in attendance at the workshop and suggests recommendations for standards that evolved from the discussion.

2003

As unmanned systems become widely employed in various applications, it is critical to have a set of standard definitions and metrics for specifying and evaluating the systems in terms of their levels of autonomy. Developing autonomy levels for unmanned systems is a complex issue that has to take into account many factors such as task complexity, human interaction, environmental difficulty, mission and system dependence, and quality factors. We report on a workshop that addresses this issue.

2018

2007

There is growing interest in many sectors in developing highly autonomous systems such as Unmanned Air Vehicles (UAVs) with significant in-mission executive power. Such systems have the potential to replace humans in a variety of dangerous tasks, but there is concern that the combination of novel technologies with demanding tasks and unpredictable environments will lead to new safety challenges. This paper reviews proposed scenarios of autonomous system applications and identifies the safety concerns that they raise. It then explores how autonomous systems can be certified as safe to operate within the terms of the existing safety standards that are used by the UK military. The combination of difficulties arising from the nature of autonomous systems and the provisions of the existing safety standards raise serious concerns about the practicality of certification. With this in mind, promising work on new techniques for analysing and ensuring the safety of autonomous systems is revie...

3rd IET International Conference on System Safety 2008, 2008

Def Stan 00-56 requires a safety case to be built before an autonomous system can be certified, but there is no current guidance on how such a case should be structured. The authors have reviewed several plausible approaches to structuring a safety case, including arguing human equivalence, deriving necessary capabilities from a Level of Autonomy scheme, and by deriving an explicit rationale for the Unmanned Systems Safety Guide recently published by the US Department of Defense. From this, we have produced an initial recommended approach. The process of deriving it has revealed that much of the published advice on autonomous system safety is either of very low value or potentially dangerous.

2018

Autonomous Systems are seeing increasing use and increasingly safety-significant application. Consequently, the safety of autonomous systems is an important topic. To reflect this importance the Safety Critical Systems Club (SCSC) has established the Safety of Autonomous Systems Working Group (SASWG). This paper introduces the SASWG and describes (and justifies) the approach it is taking. A running example is used to illustrate challenges, which are organised against three “difficulty horizons”. Potential solutions to some of the challenges are outlined; possible research directions are suggested for other challenges. Some proposed but invalid solutions are also identified. Overall, whilst the SASWG acknowledges the very significant benefits that could accrue from autonomous systems, it believes their development and implementation should be pursued carefully and thoughtfully.

2019

The introduction of automation in a wide range of activities has changed how society interacts with machines. For years, automation was applied only to physical activities, rather than cognitive aspects, such as, situation assessment, sense-making and decision-taking. The advent of artificial intelligence, machinelearning, and easier access to powerful software and sophisticated hardware have brought a new revolution into how we interact with automated systems, both as users as well as operators. The outcome of this revolution are highly automated and autonomous systems.