GCRI · Kecerdasan Buatan dan Sistem Otonom
Sistem fisik otonom
Autonomous vehicles, drones, industrial and service robots sense their surroundings and act in the physical world, where errors can injure people and damage property. Their safety depends on perception, decision-making, fallback behaviour and the environments they are allowed to operate in. This page sets out how autonomous physical sistem fail, how their safety is assessed, and what evidence serves the regulators, operators and communities involved.
Autonomous physical systems on the record.
Autonomous physical systems: how the risk develops.
Robots, sensors and controls. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Perception meets the unexpected
Sensors and models misread rare situations: unusual objects, poor weather, glare or confusing road layouts. Rare cases dominate risk because they are hard to test.
Operating domains set limits
Sistem are designed for an operational design domain of roads, weather, speeds or spaces. Failures increase when systems are used outside it.
Handover to humans is hard
Supervisors lose attention when automation works most of the time, and cannot take over instantly when it fails.
Safety cases must be argued
Showing that a system is acceptably safe needs a structured argument combining design, testing, simulation and field data, because driving or operating enough to see rare failures is impractical.
Agent oversight console.
Layar yang akan digunakan sebuah tim untuk mengikuti subjek ini: ukuran, status, dan keputusannya. Nama adalah contoh dan nilai bersifat ilustratif.
Agent oversight console
Autonomous physical systems · Agent sandbox · Session AG-7276
Tool calls
- 10:27Infosearch(documents) · read-only
- 10:20Inforun_model(flood_nowcast) · sandbox
- 10:39Gerbangsend_message(external) · needs approval
- 10:45Errorwrite_file(outside workspace) · denied
Kebijakan
- Lingkup
- workspace only
- External actions
- human approval
- Spend limit
- 11 units
- Kill switch
- Armed
Actions by approval class
Autonomous machines operate in physical domains with physical risks.
- PanganKeterkaitanAutonomous farm machinery operates near people and animals.What to watchSafety incidents
- EnergiKeterkaitanDrones and robots inspect and repair grids.What to watchInspection reliability
- KesehatanKeterkaitanRobots in hospitals need safety and hygiene controls.What to watchIncident reports
Findings are designed for the seri Bukti Labs. SAE J3016 defines levels of driving automation, and the EU Machinery Regulation (2023) addresses machines with self-evolving behaviour.
- Field safety incident data.
- Test protocols for autonomy in unstructured settings.
How autonomous physical systems is modelled, measured and tested.
Model, asumsi, data, dan ketidakpastiannya, dalam bentuk yang akan diperiksa ilmuwan atau insinyur. Strukturnya mengikuti metode yang dipublikasikan. Nilai yang digambar bersifat ilustratif, bukan temuan.
Model sistem
The couplings between the systems in autonomous physical systems: signs of influence, a reinforcing and a balancing loop, and cross-links where stress jumps.
Apa yang diukur, dan seberapa baik.
Setiap indikator membawa satuan, frekuensi, jenis sumber di baliknya, dan label bukti yang akan dibawa suatu nilai. Tidak ada nilai yang ditampilkan: sebuah program menyediakannya berdasarkan wewenangnya sendiri.
| Indikator | Unit | Kadensi | Jenis sumber | Label |
|---|---|---|---|---|
| Crashes involving automated driving sistem | count by severity | setiap bulan | Mandatory crash reporting | Statistik terbitan |
| Disengagements or interventions | count per distance or hour | tahunan | Operator reports to regulators | DILAPORKAN |
| Distance driven autonomously | kilometres | tahunan | Operator reports | DINYATAKAN |
| Industrial robot injuries | jumlah | tahunan | Occupational injury statistics | Statistik terbitan |
| Operating domain exceedances | count of events outside design domain | per deployment | Operator logs | TERAMATI |
| Drone incidents near aircraft | jumlah | tahunan | Aviation safety reporting | DILAPORKAN |
Asal bukti.
Set data publik bernama dan catatan kelembagaan, masing-masing dengan kelas lisensinya. Sumber berdasarkan perjanjian hanya digunakan sesuai ketentuannya dan tidak pernah diterbitkan ulang.
| Sumber | Apa yang diberikannya | Kelas lisensi |
|---|---|---|
| NHTSA Standing General Order crash reports | Crashes involving automated driving and driver assistance sistem in the US | Public domain, US government |
| California DMV autonomous vehicle reports | Collision and disengagement reports from permit holders | Catatan publik |
| UNECE regulations on automated driving, including UN Regulation No. 157 | Type approval requirements for automated lane keeping and related systems | Public, UNECE terms |
| ISO 10218 dan ISO/TS 15066 | Safety requirements for industrial and collaborative robots | Licensed standards, purchase required |
| Nasional occupational injury statistics | Injuries involving machinery and robots | Public records, licence per government |
Siapa yang memutuskan, dan apa yang dilayani bukti.
Studio Kecerdasan Buatan dan Sistem prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Siapa yang memutuskan | Pertanyaannya | Bukti yang dilayani halaman ini |
|---|---|---|
| Transport and aviation regulators | Whether and where to permit autonomous operation | Crash data with exposure; safety case reviews |
| Workplace safety regulators | What safety requirements to set for robots | Injury statistics; standards |
| Operators and manufacturers | How to define and enforce operating domains | Domain documentation; exceedance logs |
Keluaran yang dihasilkan topik ini
- Safety case review frameworks
- Analyses of crash and intervention reports with exposure stated
- Operational design domain documentation templates
- Evidence notes for regulators
What safety engineers ask.
- Does the system perceive its environment correctly across weather, lighting and unusual objects?
- How does it behave at the edges of its operating conditions, and does it know when it is outside them?
- Can it fail safely, handing over to a person or stopping without creating new hazards?
- How is safety demonstrated when learned components cannot be fully specified?
Where physical autonomy fails.
Perception is the weak link: cameras, lidar and radar each fail differently in rain, glare, dust or darkness, and learned perception models can misclassify rare objects. Planning must handle uncertain predictions of other agents’ behaviour. Control must remain stable as conditions change.
Handover to humans is difficult because people monitoring automation lose situational awareness and may not respond in time. Classical safety engineering assumes components whose behaviour can be fully specified, which learned components are not.
Assurance for systems that move.
- 01
Domain desain operasional
Defining the conditions a system is designed for and detecting when it leaves them.
- 02
Simulation and scenario testing
Testing across large numbers of simulated scenarios, using robotika dan otomasi and high-performance, cloud and edge computing. Simulation fidelity limits what it can reveal.
- 03
Safety cases
Structured arguments, supported by evidence, that a system is acceptably safe for a use, drawing on Evaluasi AI dan rekayasa penjaminan. A safety case is only as strong as its weakest evidence.
- 04
Field monitoring
Collecting incidents and near misses after deployment. Under-reporting is common.
Cara hal ini dikaji, dan apa yang tidak dapat dinyatakan metodenya.
- 01Robotika dan otomasiPengujian berbasis skenario · Keselamatan robot kolaboratif · Pemantauan domain operasiDiterapkan di sini
- 02Evaluasi AI dan rekayasa penjaminanKasus penjaminan untuk sistem AI · Red teaming terstruktur · Pemantauan pascapenerapanDiterapkan di sini
- 03Probabilitas dan statistikaKetidakpastian pada skor tolok ukur · Penilaian kalibrasi · Sampel auditDiterapkan di sini
- 04Kecerdasan buatan dan pembelajaran mesinAnalisis pergeseran distribusi · Fine-tuning untuk penggunaan tertentu · Metode interpretabilitasDiterapkan di sini
Where this page’s references lead.
Badan dan dokumen yang menjadi acuan halaman ini. Status dan edisi berubah; setiap tautan menuju badan penerbitnya.
Tanya jawab.
What is an operational design domain?
Are self-driving cars safer than human drivers?
What is a safety case?
Are autonomous vehicles safer than human drivers?
Can learned components be safety-assured?
Does the platform test products?
Kontribusikan bukti dari halaman ini.
Laporan is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- SurelGerbang GW-GCRIKirim dari halaman iniBukti, koreksi, kasus, metode, tinjauan, terjemahan, atau masukan profil. Kiriman tiba dengan label halaman ini, sehingga editor pemiliknya menerimanya.
- ZenodoDeposit terbukaDeposit dan ajukan ke komunitasTerbitkan catatan Anda sendiri di Zenodo dengan ORCID iD Anda, lalu ajukan ke komunitas Reports di sana. Kurator meninjaunya; DOI tetap milik Anda.
- LaporanGCRILihat semua cara berkontribusiJenis kontribusi, cara masing-masing ditinjau, kredit yang Anda terima, lisensinya, dan apa yang tidak pernah diterima.
Jangan pernah mengirim data pribadi, rincian infrastruktur sensitif, atau apa pun yang tidak berhak Anda bagikan. Kontribusi ditinjau sebelum digunakan. Mengirimkannya tidak memberikan hak kepenulisan atau editorial, dan sponsor tidak pernah membeli sebuah temuan.
Bawa metode, dataset, atau pertanyaan.
Informasi dan pertanyaan sudah terbuka hari ini. Sebuah pertanyaan menerima nomor referensi dan menyebutkan langkah berikutnya; lamaran, pembayaran, alokasi komputasi, dan pendaftaran hanya dibuka ketika tim, ketentuan, dan kapasitas tinjauannya tersedia.
Dokumen hukum.
Peta, data, dan gambar di situs ini ditampilkan untuk konteks dan informasi umum. Semua itu bukan nasihat, prakiraan, atau peringatan; batasan informasinya dijelaskan dalam Penafian dan Pemberitahuan Data.