GCRI · 人工智能与自主系统
自主物理系统
Autonomous vehicles, drones, industrial and service robots sense their surroundings and act in the physical world, where errors can injure people and damage property. Their safety depends on perception, decision-making, fallback behaviour and the environments they are allowed to operate in. This page sets out how autonomous physical 系统 fail, how their safety is assessed, and what evidence serves the regulators, operators and communities involved.
Autonomous physical systems: how the risk develops.
Robots, sensors and controls. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Perception meets the unexpected
Sensors and models misread rare situations: unusual objects, poor weather, glare or confusing road layouts. Rare cases dominate risk because they are hard to test.
Operating domains set limits
系统 are designed for an operational design domain of roads, weather, speeds or spaces. Failures increase when systems are used outside it.
Handover to humans is hard
Supervisors lose attention when automation works most of the time, and cannot take over instantly when it fails.
Safety cases must be argued
Showing that a system is acceptably safe needs a structured argument combining design, testing, simulation and field data, because driving or operating enough to see rare failures is impractical.
Agent oversight console.
团队跟踪该主题时会使用的界面：其指标、状态与决策。名称为示例，数值仅作示意。
Agent oversight console
Autonomous physical systems · Agent sandbox · Session AG-7276
Tool calls
- 10:27信息search(documents) · read-only
- 10:20信息run_model(flood_nowcast) · sandbox
- 10:39关口send_message(external) · needs approval
- 10:45Errorwrite_file(outside workspace) · denied
政策
- 范围
- workspace only
- External actions
- human approval
- Spend limit
- 11 units
- Kill switch
- Armed
Actions by approval class
Autonomous machines operate in physical domains with physical risks.
Findings are designed for the Labs 证据系列. SAE J3016 defines levels of driving automation, and the EU Machinery Regulation (2023) addresses machines with self-evolving behaviour.
- Field safety incident data.
- Test protocols for autonomy in unstructured settings.
How autonomous physical systems is modelled, measured and tested.
模型、其假设、数据和不确定性，以科学家或工程师会核查的形式呈现。结构遵循已发表的方法。所取数值仅为示意，不是研究发现。
系统模型
The couplings between the systems in autonomous physical systems: signs of influence, a reinforcing and a balancing loop, and cross-links where stress jumps.
衡量什么，以及衡量得如何。
每个指标都注明其单位、频率、背后的来源类型以及数值将附带的证据标签。此处不显示数值：数值由计划依据其自身权限提供。
| 指标 | 单位 | 节奏 | 来源类型 | 标签 |
|---|---|---|---|---|
| Crashes involving automated driving 系统 | count by severity | 每月 | Mandatory crash reporting | 已发布统计 |
| Disengagements or interventions | count per distance or hour | 年度 | Operator reports to regulators | 已报告 |
| Distance driven autonomously | kilometres | 年度 | Operator reports | 已声明 |
| Industrial robot injuries | 计数 | 年度 | Occupational injury statistics | 已发布统计 |
| Operating domain exceedances | count of events outside design domain | per deployment | Operator logs | 观测 |
| Drone incidents near aircraft | 计数 | 年度 | Aviation safety reporting | 已报告 |
证据来自何处。
具名公共数据集与机构记录，各自注明许可类别。依协议获得的来源仅按其条款使用，绝不再次发布。
| 来源 | 它提供什么 | 许可类别 |
|---|---|---|
| NHTSA Standing General Order crash reports | Crashes involving automated driving and driver assistance 系统 in the US | Public domain, US government |
| California DMV autonomous vehicle reports | Collision and disengagement reports from permit holders | 公共记录 |
| UNECE regulations on automated driving, including UN Regulation No. 157 | Type approval requirements for automated lane keeping and related systems | Public, UNECE terms |
| ISO 10218 和 ISO/TS 15066 | Safety requirements for industrial and collaborative robots | Licensed standards, purchase required |
| 国家 occupational injury statistics | Injuries involving machinery and robots | Public records, licence per government |
由谁决定，以及证据服务于什么。
人工智能与自主 系统 prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| 谁来决定 | 问题 | 本页面服务的证据 |
|---|---|---|
| Transport and aviation regulators | Whether and where to permit autonomous operation | Crash data with exposure; safety case reviews |
| Workplace safety regulators | What safety requirements to set for robots | Injury statistics; standards |
| Operators and manufacturers | How to define and enforce operating domains | Domain documentation; exceedance logs |
该主题产生的产出
- Safety case review frameworks
- Analyses of crash and intervention reports with exposure stated
- Operational design domain documentation templates
- Evidence notes for regulators
What safety engineers ask.
- Does the system perceive its environment correctly across weather, lighting and unusual objects?
- How does it behave at the edges of its operating conditions, and does it know when it is outside them?
- Can it fail safely, handing over to a person or stopping without creating new hazards?
- How is safety demonstrated when learned components cannot be fully specified?
Where physical autonomy fails.
Perception is the weak link: cameras, lidar and radar each fail differently in rain, glare, dust or darkness, and learned perception models can misclassify rare objects. Planning must handle uncertain predictions of other agents’ behaviour. Control must remain stable as conditions change.
Handover to humans is difficult because people monitoring automation lose situational awareness and may not respond in time. Classical safety engineering assumes components whose behaviour can be fully specified, which learned components are not.
Assurance for systems that move.
- 01
运行设计域
Defining the conditions a system is designed for and detecting when it leaves them.
- 02
Simulation and scenario testing
Testing across large numbers of simulated scenarios, using 机器人与自动化 and high-performance, cloud and edge computing. Simulation fidelity limits what it can reveal.
- 03
Safety cases
Structured arguments, supported by evidence, that a system is acceptably safe for a use, drawing on AI 评估与保证工程. A safety case is only as strong as its weakest evidence.
- 04
Field monitoring
Collecting incidents and near misses after deployment. Under-reporting is common.
Where this page’s references lead.
阅读本页时所对照的机构和文件。状态和版本会变化；每个链接都指向发布机构。
问题与解答。
What is an operational design domain?
Are self-driving cars safer than human drivers?
What is a safety case?
Are autonomous vehicles safer than human drivers?
Can learned components be safety-assured?
Does the platform test products?
从本页贡献证据。
报告 is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- 电子邮件网关 GW-GCRI从本页发送证据、更正、案例、方法、审查、翻译或概况投稿。它会带上本页标签送达，由负责的编辑接收。
- Zenodo开放存缴存缴并提交到社区用您的 ORCID iD 在 Zenodo 上发布您自己的记录，然后提交到那里的 Reports 社区。由策展人审核；DOI 仍归您所有。
- 报告GCRI查看所有贡献方式贡献类型、各自的评审方式、您获得的署名、许可协议，以及绝不接受的内容。
切勿发送个人数据、敏感基础设施细节或您无权分享的任何内容。贡献在使用前会经过审查；发送贡献不赋予署名权或编辑权，赞助也绝不能换取研究结论。
法律文件。
本网站上的地图、数据和图表仅作为背景和一般信息展示。它们不是建议、预测或预警；信息的局限见免责声明和数据说明。