Artificial Intelligence and Autonomous Systems
The Global Centre for Risk and Innovation (GCRI)
Autonomous vehicles, drones, industrial and service robots sense their surroundings and act in the physical world, where errors can injure people and damage property. Their safety depends on perception, decision-making, fallback behaviour and the environments they are allowed to operate in. This page sets out how autonomous physical systems fail, how their safety is assessed, and what evidence serves the regulators, operators and communities involved.
GCRI · In development · service level R0
Autonomous vehicles and robot safety is the field that covers machines that sense and act in the physical world without continuous human control, where errors can injure people. Safety depends on perception in rare situations, staying within the operating domain, handover to humans and structured safety cases.
For autonomous physical systems, in GCRI's spatial-intelligence reading, the map opens on true-colour imagery, population density and night lights with true colour from NASA GIBS, framed on East Asia; 3 further NASA layers wait in the list.
The interactive map needs JavaScript; the layer list and sources are below.
Layers 3
Connected activity, for this page
Keyboard, with the map focused
Hold Ctrl or ⌘ and scroll to zoom
Latest available
NASA GIBS / EOSDIS · ASTER GDEM (METI, NASA) · Blue Marble (NASA) · © OpenStreetMap contributors (ODbL) · Natural Earth
Loading the latest NASA imagery.
Drag to move. Ctrl or ⌘ and scroll, pinch or the buttons to zoom; hover for the values under the pointer; reset returns to the region.
For autonomous physical systems, in GCRI's spatial-intelligence reading, the map opens on true-colour imagery, population density and night lights with true colour from NASA GIBS, framed on East Asia; 3 further NASA layers wait in the list.Satellite and model products as NASA publishes them, each with its own date; indicators, not assessments of any place; claim ceiling INTEGRATED.
Layers and dates
The working screen behind this subject: its structure, terms and states. Names are placeholders and values illustrative.
Autonomous physical systems · Agent sandbox · Session AG-7276
Robots, sensors and controls. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Sensors and models misread rare situations: unusual objects, poor weather, glare or confusing road layouts. Rare cases dominate risk because they are hard to test.
Systems are designed for an operational design domain of roads, weather, speeds or spaces. Failures increase when systems are used outside it.
Supervisors lose attention when automation works most of the time, and cannot take over instantly when it fails.
Showing that a system is acceptably safe needs a structured argument combining design, testing, simulation and field data, because driving or operating enough to see rare failures is impractical.
Each term as this page uses it. Where a standard-setting body defines the term, the wording follows that body.
| Term | Definition |
|---|---|
| Operational design domain | The conditions under which an automated system is designed to function. |
| Disengagement | A transfer of control from an automated system to a human, reported in some jurisdictions. |
| Collaborative robot | A robot designed to work in shared space with people under safety limits. |
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Crashes involving automated driving systems | count by severity | monthly | Mandatory crash reporting | OFFICIAL |
| Disengagements or interventions | count per distance or hour | annual | Operator reports to regulators | REPORTED |
| Distance driven autonomously | kilometres | annual | Operator reports | DECLARED |
| Industrial robot injuries | count | annual | Occupational injury statistics | OFFICIAL |
| Operating domain exceedances | count of events outside design domain | per deployment | Operator logs | OBSERVED |
| Drone incidents near aircraft | count | annual | Aviation safety reporting | REPORTED |
The topic as a computational graph: named sources feed the methods, an evidence review gates the outputs, and the decisions stay with the authorities that hold them.
Fig. 02Nodes carry their type, two metric rows and a state; connectors show what feeds what; dashed frames name the owners; the timeline is the cadence. Design of the pipeline, not a record of a run.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Source | What it gives | Licence class |
|---|---|---|
| NHTSA Standing General Order crash reports | Crashes involving automated driving and driver assistance systems in the US | Public domain, US government |
| California DMV autonomous vehicle reports | Collision and disengagement reports from permit holders | Public records |
| UNECE regulations on automated driving, including UN Regulation No. 157 | Type approval requirements for automated lane keeping and related systems | Public, UNECE terms |
| ISO 10218 and ISO/TS 15066 | Safety requirements for industrial and collaborative robots | Licensed standards, purchase required |
| National occupational injury statistics | Injuries involving machinery and robots | Public records, licence per government |
Artificial Intelligence and Autonomous Systems prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Who decides | The question | Evidence this page serves |
|---|---|---|
| Transport and aviation regulators | Whether and where to permit autonomous operation | Crash data with exposure; safety case reviews |
| Workplace safety regulators | What safety requirements to set for robots | Injury statistics; standards |
| Operators and manufacturers | How to define and enforce operating domains | Domain documentation; exceedance logs |
Perception is the weak link: cameras, lidar and radar each fail differently in rain, glare, dust or darkness, and learned perception models can misclassify rare objects. Planning must handle uncertain predictions of other agents' behaviour. Control must remain stable as conditions change.
Handover to humans is difficult because people monitoring automation lose situational awareness and may not respond in time. Classical safety engineering assumes components whose behaviour can be fully specified, which learned components are not.
Defining the conditions a system is designed for and detecting when it leaves them.
Testing across large numbers of simulated scenarios, using robotics and automation and high-performance, cloud and edge computing. Simulation fidelity limits what it can reveal.
Structured arguments, supported by evidence, that a system is acceptably safe for a use, drawing on AI evaluation and assurance engineering. A safety case is only as strong as its weakest evidence.
Collecting incidents and near misses after deployment. Under-reporting is common.
The current service level is R0: information and enquiries. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.