GCRI · هوش مصنوعی و سامانه‌های خودمختار
سامانه‌های فیزیکی خودگردان
Autonomous vehicles, drones, industrial and service robots sense their surroundings and act in the physical world, where errors can injure people and damage property. Their safety depends on perception, decision-making, fallback behaviour and the environments they are allowed to operate in. This page sets out how autonomous physical سامانه‌ها fail, how their safety is assessed, and what evidence serves the regulators, operators and communities involved.
Autonomous physical systems on the record.
Autonomous physical systems: how the risk develops.
Robots, sensors and controls. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Perception meets the unexpected
Sensors and models misread rare situations: unusual objects, poor weather, glare or confusing road layouts. Rare cases dominate risk because they are hard to test.
Operating domains set limits
سامانه‌ها are designed for an operational design domain of roads, weather, speeds or spaces. Failures increase when systems are used outside it.
Handover to humans is hard
Supervisors lose attention when automation works most of the time, and cannot take over instantly when it fails.
Safety cases must be argued
Showing that a system is acceptably safe needs a structured argument combining design, testing, simulation and field data, because driving or operating enough to see rare failures is impractical.
Agent oversight console.
The screen a team would use to follow this subject: its measures, states and decisions. Names are examples and values are illustrative.
Agent oversight console
Autonomous physical systems · Agent sandbox · Session AG-7276
Tool calls
- 10:27اطلاعاتsearch(documents) · read-only
- 10:20اطلاعاتrun_model(flood_nowcast) · sandbox
- 10:39دروازهsend_message(external) · needs approval
- 10:45Errorwrite_file(outside workspace) · denied
سیاست‌گذاری
- دامنه
- workspace only
- External actions
- human approval
- Spend limit
- 11 units
- Kill switch
- Armed
Actions by approval class
Autonomous machines operate in physical domains with physical risks.
Findings are designed for the مجموعهٔ Labs Evidence. SAE J3016 defines levels of driving automation, and the EU Machinery Regulation (2023) addresses machines with self-evolving behaviour.
- Field safety incident data.
- Test protocols for autonomy in unstructured settings.
How autonomous physical systems is modelled, measured and tested.
مدل، مفروضات، داده‌ها و عدم قطعیت آن، به شکلی که دانشمند یا مهندس بررسی می‌کند. ساختارها از روش‌های منتشرشده پیروی می‌کنند. مقادیر ترسیم‌شده تبیینی‌اند، نه یافته.
مدل سامانه
The couplings between the systems in autonomous physical systems: signs of influence, a reinforcing and a balancing loop, and cross-links where stress jumps.
What is measured, and how well.
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | دورهٔ تکرار | Source type | برچسب |
|---|---|---|---|---|
| Crashes involving automated driving سامانه‌ها | count by severity | monthly | Mandatory crash reporting | آمارهای منتشرشده |
| Disengagements or interventions | count per distance or hour | annual | Operator reports to regulators | گزارش‌شده |
| Distance driven autonomously | kilometres | annual | Operator reports | DECLARED |
| Industrial robot injuries | count | annual | Occupational injury statistics | آمارهای منتشرشده |
| Operating domain exceedances | count of events outside design domain | per deployment | Operator logs | مشاهده‌شده |
| Drone incidents near aircraft | count | annual | Aviation safety reporting | گزارش‌شده |
Where the evidence comes from.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| منبع | What it gives | Licence class |
|---|---|---|
| NHTSA Standing General Order crash reports | Crashes involving automated driving and driver assistance سامانه‌ها in the US | Public domain, US government |
| California DMV autonomous vehicle reports | Collision and disengagement reports from permit holders | Public records |
| UNECE regulations on automated driving, including UN Regulation No. 157 | Type approval requirements for automated lane keeping and related systems | Public, UNECE terms |
| ISO 10218 و ISO/TS 15066 | Safety requirements for industrial and collaborative robots | Licensed standards, purchase required |
| ملی occupational injury statistics | Injuries involving machinery and robots | Public records, licence per government |
Who decides, and what the evidence serves.
هوش مصنوعی و سامانه‌ها prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| چه کسی تصمیم می‌گیرد | The question | Evidence this page serves |
|---|---|---|
| Transport and aviation regulators | Whether and where to permit autonomous operation | Crash data with exposure; safety case reviews |
| Workplace safety regulators | What safety requirements to set for robots | Injury statistics; standards |
| Operators and manufacturers | How to define and enforce operating domains | Domain documentation; exceedance logs |
Outputs this topic produces
- Safety case review frameworks
- Analyses of crash and intervention reports with exposure stated
- Operational design domain documentation templates
- Evidence notes for regulators
What safety engineers ask.
- Does the system perceive its environment correctly across weather, lighting and unusual objects?
- How does it behave at the edges of its operating conditions, and does it know when it is outside them?
- Can it fail safely, handing over to a person or stopping without creating new hazards?
- How is safety demonstrated when learned components cannot be fully specified?
Where physical autonomy fails.
Perception is the weak link: cameras, lidar and radar each fail differently in rain, glare, dust or darkness, and learned perception models can misclassify rare objects. Planning must handle uncertain predictions of other agents’ behaviour. Control must remain stable as conditions change.
Handover to humans is difficult because people monitoring automation lose situational awareness and may not respond in time. Classical safety engineering assumes components whose behaviour can be fully specified, which learned components are not.
Assurance for systems that move.
- 01
Operational design domain
Defining the conditions a system is designed for and detecting when it leaves them.
- 02
Simulation and scenario testing
Testing across large numbers of simulated scenarios, using رباتیک و خودکارسازی and high-performance, cloud and edge computing. Simulation fidelity limits what it can reveal.
- 03
Safety cases
Structured arguments, supported by evidence, that a system is acceptably safe for a use, drawing on مهندسی ارزیابی و اطمینان‌بخشی هوش مصنوعی. A safety case is only as strong as its weakest evidence.
- 04
Field monitoring
Collecting incidents and near misses after deployment. Under-reporting is common.
How it is studied, and what the methods cannot say.
- 01رباتیک و خودکارسازیScenario-based testing · Collaborative robot safety · Operating domain monitoringApplied here
- 02مهندسی ارزیابی و اطمینان‌بخشی هوش مصنوعیAssurance cases for AI systems · Structured red teaming · Post-deployment monitoringApplied here
- 03احتمال و آمارUncertainty on benchmark scores · Calibration assessment · Audit samplingApplied here
- 04هوش مصنوعی و یادگیری ماشینDistribution shift analysis · Fine-tuning for a specific use · Interpretability methodsApplied here
Where this page’s references lead.
نهادها و اسنادی که این صفحه بر پایهٔ آن‌ها خوانده می‌شود. وضعیت و ویراست‌ها تغییر می‌کنند؛ هر پیوند به نهاد صادرکننده می‌رود.
پرسش و پاسخ.
What is an operational design domain?
Are self-driving cars safer than human drivers?
What is a safety case?
Are autonomous vehicles safer than human drivers?
Can learned components be safety-assured?
Does the platform test products?
شواهد را از همین صفحه ارسال کنید.
گزارش‌ها is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- ایمیلدرگاه GW-GCRIاز همین صفحه ارسال کنیدشواهد، یک اصلاحیه، یک مورد، یک روش، یک داوری، یک ترجمه یا ورودی نمایه. با برچسب این صفحه میرسد تا ویراستار مسئول آن را دریافت کند.
- Zenodoسپردهٔ بازآن را واگذار کنید و به جامعه ارائه دهیدسابقهٔ خود را با شناسهٔ ORCID خود در Zenodo منتشر کنید و سپس آن را در همانجا به جامعهٔ Reports ارسال کنید. گزینشگران آن را داوری می‌کنند؛ DOI از آنِ شما میماند.
- گزارش‌هاGCRIهمهٔ راههای مشارکت را ببینیدانواع مشارکت، نحوهٔ بررسی هر یک، اعتباری که دریافت میکنید، مجوز و آنچه هرگز پذیرفته نمیشود.
هرگز داده شخصی، جزئیات حساس زیرساخت یا چیزی را که حق اشتراکش را ندارید نفرستید. هر مشارکت پیش از استفاده داوری می‌شود؛ ارسال آن حق مؤلفی یا ویرایشی نمی‌دهد و حمایت مالی هرگز یافته‌ای را نمی‌خرد.
یک روش، یک مجموعه‌داده یا یک پرسش بیاورید.
اطلاعات و درخواست‌ها امروز باز است. هر درخواست یک شمارهٔ پیگیری می‌گیرد و گام بعدی را مشخص می‌کند؛ درخواست‌های عضویت، پرداخت‌ها، تخصیص رایانش و ثبت‌نام‌ها تنها زمانی باز می‌شوند که تیم، شرایط و ظرفیت داوری آن‌ها فراهم باشد.
اسناد حقوقی.
نقشه‌ها، داده‌ها و ارقام این سایت برای زمینه و اطلاعات عمومی نشان داده می‌شوند. توصیه، پیش‌بینی یا هشدار نیستند؛ حدود این اطلاعات در سلب مسئولیت‌ها و اعلامیه‌های داده بیان شده است.