GCRI · Kecerdasan Buatan dan Sistem Otonom
Evaluasi model
Model evaluation asks what an AI model can do, how reliably, and where it fails. Benchmarks, red teaming, human studies and field monitoring each measure something different, and none fully predicts behaviour in deployment. This page sets out how evaluations are designed and where they mislead, how results should be reported, and what evidence serves the developers, deployers and regulators who rely on them.
Model evaluation in seven parts.
- 01PertanyaannyaModel evaluation: how the risk develops
- 02Di seluruh domainEvaluation must reflect the domain where a model is used
- 03IndikatorApa yang diukur, dan seberapa baik
- 04SumberDari mana bukti berasal
- 05KeputusanSiapa yang memutuskan, dan apa yang dilayani bukti
- 06Pertanyaan intiWhat evaluators ask
- 07MekanismeWhy scores mislead
Model evaluation: how the risk develops.
Validity, calibration and generalisation. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Benchmarks saturate and leak
Models quickly reach high scores on fixed benchmarks, and test items can leak into training data. High scores then say more about the benchmark than about general capability.
Test conditions differ from use
Evaluations use set prompts, settings and tasks. Real users phrase things differently, combine tools and face new situations, so measured performance may not transfer.
Adversarial testing finds hidden failures
Red teaming and adversarial prompts reveal unsafe or unreliable behaviour that average-case metrics miss. Coverage depends on the testers’ skill and the time allowed.
Performance drifts after deployment
Model updates, changing inputs and new uses alter behaviour over time. Without monitoring, a model evaluated at release can behave differently months later.
Registri model.
Layar yang akan digunakan sebuah tim untuk mengikuti subjek ini: ukuran, status, dan keputusannya. Nama adalah contoh dan nilai bersifat ilustratif.
Registri model
Model evaluation · Model registry · Release gate
Model
| M-184 triage assistant | Layanan publik | Tinggi | 19 of 24 suites | Terhambat |
| M-320 flood nowcast | Sains | Sedang | 12 dari 12 | Dirilis |
| M-157 document search | Internal | Rendah | 8 of 8 | Dirilis |
| M-563 agent planner | Agentik | Tinggi | 12 of 16 | Sedang dievaluasi |
Kartu model
- Penggunaan yang dimaksudkan
- dukungan triase, manusia memutuskan
- Di luar cakupan
- keputusan kelayakan
- Batasan yang diketahui
- akurasi lebih rendah pada kasus langka
- Pemilik
- Pemilik model
Log insiden
- D-21INCKutipan halusinasi dalam ringkasan
- D-9MITLandasan pengambilan data ditambahkan
- D-2INCInjeksi prompt melalui berkas yang diunggah
- D-1MITIzin alat dipersempit
Evaluation scores
vs release threshold
Evaluation must reflect the domain where a model is used.
- KesehatanKeterkaitanClinical AI must be evaluated on the populations it serves.What to watchPerformance by subgroup
- AirKeterkaitanHydrological ML must be tested out of sample in new basins.What to watchPerformance in ungauged basins
- PanganKeterkaitanCrop monitoring models need local ground truth.What to watchAccuracy against field data
Evaluations are designed for the seri Bukti Labs. The International AI Safety Report (2025) describes the limits of current evaluations; NIST’s AI RMF and its generative AI profile (2024) set measurement practice. Models would carry cards under NS-12 of the proposed Nexus Standards.
- Held-out test sets for a domain.
- Evaluation protocols with known failure cases.
How model evaluation is modelled, measured and tested.
Model, asumsi, data, dan ketidakpastiannya, dalam bentuk yang akan diperiksa ilmuwan atau insinyur. Strukturnya mengikuti metode yang dipublikasikan. Nilai yang digambar bersifat ilustratif, bukan temuan.
Model, persamaan, dan asumsi
The model structure used for model evaluation, the assumptions it makes and how each is tested.
Ri = f(Hi, Ei, Vi, Ci)bahaya, keterpaparan, kerentanan, dan kapasitas penanggulanganEAL = ∑k pk · Lkatas peristiwa k dengan probabilitas tahunan pkxt+1 = A xt + B ut + εtA carries the couplings, u the shock, ε the error| Asumsi | Mengapa ini dibuat | Bagaimana ini diuji |
|---|---|---|
| Bahaya dan eksposur bersifat independen dalam satu peristiwa | Menjaga integral kerugian tetap dapat dihitung | Bandingkan dengan pencocokan kopula pada catatan gabungan |
| Keterkaitan bersifat linear di dekat titik operasi | Membiarkan guncangan merambat melalui A | Uji di luar rentang linear lalu bandingkan |
| Kurva kerentanan dapat dipindahkan antartempat | Kurva lokal jarang tersedia | Validasi terhadap kerugian teramati jika tersedia |
- Masukan
- dataset berversi dengan asal-usul yang tercatat
- Keluaran
- distribusi, bukan angka tunggal
- Kode
- dirilis bersama kartu model dan pengujiannya
- Reproduksi
- data, kode, lingkungan, dan seed yang sama
Apa yang diukur, dan seberapa baik.
Setiap indikator membawa satuan, frekuensi, jenis sumber di baliknya, dan label bukti yang akan dibawa suatu nilai. Tidak ada nilai yang ditampilkan: sebuah program menyediakannya berdasarkan wewenangnya sendiri.
| Indikator | Unit | Kadensi | Jenis sumber | Label |
|---|---|---|---|---|
| Benchmark score with uncertainty | score with confidence interval | per evaluation | Published benchmark results | DILAPORKAN |
| Contamination checks reported | percent of evaluations reporting checks | per evaluation | Evaluation reports | DINYATAKAN |
| Red-team findings by severity | count by severity class | per exercise | Red-team reports | DILAPORKAN |
| Performance gap between subgroups | difference in error rates | per evaluation | Disaggregated evaluation | TERAMATI |
| Post-deployment incident reports | count by type | setiap bulan | Incident monitoring | DILAPORKAN |
| Galat kalibrasi | expected calibration error | per evaluation | Evaluation on held-out data | TERAMATI |
Asal bukti.
Set data publik bernama dan catatan kelembagaan, masing-masing dengan kelas lisensinya. Sumber berdasarkan perjanjian hanya digunakan sesuai ketentuannya dan tidak pernah diterbitkan ulang.
| Sumber | Apa yang diberikannya | Kelas lisensi |
|---|---|---|
| Stanford HELM (Holistic Evaluation of Language Models) | Open, multi-metric evaluations of language models | Open source, Apache 2.0 for code; results public |
| MLCommons benchmarks, including AILuminate | Standardised performance and safety benchmarks | Public results, MLCommons terms (to check) |
| Laporan Keselamatan AI Internasional | Synthesis of evidence on capabilities, risks and evaluation limits | Public, terms per publisher |
| Model cards and system cards from developers | Developer-reported evaluations and limitations | Public documents, terms per developer |
| Epoch AI benchmark and model databases | Compiled benchmark results and model characteristics | Terbuka, CC BY 4.0 |
Siapa yang memutuskan, dan apa yang dilayani bukti.
Studio Kecerdasan Buatan dan Sistem prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Siapa yang memutuskan | Pertanyaannya | Bukti yang dilayani halaman ini |
|---|---|---|
| Pengembang | Whether a model is ready for release and what limits to state | Evaluation protocols; red-team findings |
| Deploying organisations | Whether a model is fit for a specific use | Use-specific evaluations; subgroup performance |
| Regulators and AI safety institutes | What evaluation evidence to require | Evaluation standards and reporting templates |
Keluaran yang dihasilkan topik ini
- Evaluation protocols with uncertainty and contamination checks
- Disaggregated evaluation designs for subgroup performance
- Red-team methods for public sector use cases
- Evaluation reporting templates
What evaluators ask.
- Does the test measure the capability or property that matters for the intended use (construct validity)?
- When the model reports high confidence, is it right as often as that confidence implies (calibration)?
- Does performance hold on data from different places, times or populations (generalisation)?
- Has test data leaked into training, inflating results (contamination)?
Why scores mislead.
Models learn statistical regularities in their training data, including spurious ones. A medical imaging model may learn the hospital’s scanner signature rather than the disease; a language model may recognise benchmark questions it saw during training. When deployment data differ from test data, a problem known as pergeseran distribusi, performance can fall sharply and without warning.
Large general-purpose models add new difficulties. Their capabilities are broad and emergent, outputs vary with phrasing, and the line between capability and memorisation is blurred. Aggregate scores can hide failures concentrated in particular groups or tasks.
How evaluation is done, and its limits.
- 01
Held-out and out-of-distribution testing
Testing on data deliberately different from training, analysed with probabilitas dan statistika. No test set covers every deployment condition.
- 02
Calibration analysis
Comparing stated confidence with observed accuracy, and checking how it changes under shift.
- 03
Disaggregated evaluation
Reporting performance by subgroup and task rather than only on average. Small subgroups give uncertain estimates.
- 04
Human and expert evaluation
Structured assessment of open-ended outputs, using ilmu sosial dan perilaku methods for rater agreement. Human judgement is costly and can be inconsistent.
Cara hal ini dikaji, dan apa yang tidak dapat dinyatakan metodenya.
- 01Evaluasi AI dan rekayasa penjaminanKasus penjaminan untuk sistem AI · Red teaming terstruktur · Pemantauan pascapenerapanDiterapkan di sini
- 02Probabilitas dan statistikaKetidakpastian pada skor tolok ukur · Penilaian kalibrasi · Sampel auditDiterapkan di sini
- 03Kecerdasan buatan dan pembelajaran mesinAnalisis pergeseran distribusi · Fine-tuning untuk penggunaan tertentu · Metode interpretabilitasDiterapkan di sini
- 04Ilmu sosial dan perilakuKajian pengawasan manusia · Riset masyarakat terdampak · Kajian adopsi di tempat kerjaDiterapkan di sini
Where this page’s references lead.
Badan dan dokumen yang menjadi acuan halaman ini. Status dan edisi berubah; setiap tautan menuju badan penerbitnya.
Ke mana halaman ini mengarah di seluruh Konsorsium.
Tanya jawab.
Why are benchmark scores not enough?
What is red teaming in AI?
What is data contamination?
Is a high benchmark score evidence a model is safe to deploy?
What is benchmark saturation?
Why evaluate after deployment too?
Will the platform publish evaluation protocols?
Kontribusikan bukti dari halaman ini.
Laporan is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- SurelGerbang GW-GCRIKirim dari halaman iniBukti, koreksi, kasus, metode, tinjauan, terjemahan, atau masukan profil. Kiriman tiba dengan label halaman ini, sehingga editor pemiliknya menerimanya.
- ZenodoDeposit terbukaDeposit dan ajukan ke komunitasTerbitkan catatan Anda sendiri di Zenodo dengan ORCID iD Anda, lalu ajukan ke komunitas Reports di sana. Kurator meninjaunya; DOI tetap milik Anda.
- LaporanGCRILihat semua cara berkontribusiJenis kontribusi, cara masing-masing ditinjau, kredit yang Anda terima, lisensinya, dan apa yang tidak pernah diterima.
Jangan pernah mengirim data pribadi, rincian infrastruktur sensitif, atau apa pun yang tidak berhak Anda bagikan. Kontribusi ditinjau sebelum digunakan. Mengirimkannya tidak memberikan hak kepenulisan atau editorial, dan sponsor tidak pernah membeli sebuah temuan.
Bawa metode, dataset, atau pertanyaan.
Informasi dan pertanyaan sudah terbuka hari ini. Sebuah pertanyaan menerima nomor referensi dan menyebutkan langkah berikutnya; lamaran, pembayaran, alokasi komputasi, dan pendaftaran hanya dibuka ketika tim, ketentuan, dan kapasitas tinjauannya tersedia.
Dokumen hukum.
Peta, data, dan gambar di situs ini ditampilkan untuk konteks dan informasi umum. Semua itu bukan nasihat, prakiraan, atau peringatan; batasan informasinya dijelaskan dalam Penafian dan Pemberitahuan Data.