GCRI · 人工智能与自主系统
模型评估
Model evaluation asks what an AI model can do, how reliably, and where it fails. Benchmarks, red teaming, human studies and field monitoring each measure something different, and none fully predicts behaviour in deployment. This page sets out how evaluations are designed and where they mislead, how results should be reported, and what evidence serves the developers, deployers and regulators who rely on them.
Model evaluation: how the risk develops.
Validity, calibration and generalisation. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Benchmarks saturate and leak
Models quickly reach high scores on fixed benchmarks, and test items can leak into training data. High scores then say more about the benchmark than about general capability.
Test conditions differ from use
Evaluations use set prompts, settings and tasks. Real users phrase things differently, combine tools and face new situations, so measured performance may not transfer.
Adversarial testing finds hidden failures
Red teaming and adversarial prompts reveal unsafe or unreliable behaviour that average-case metrics miss. Coverage depends on the testers’ skill and the time allowed.
Performance drifts after deployment
Model updates, changing inputs and new uses alter behaviour over time. Without monitoring, a model evaluated at release can behave differently months later.
模型登记册。
团队跟踪该主题时会使用的界面：其指标、状态与决策。名称为示例，数值仅作示意。
模型注册库
Model evaluation · Model registry · Release gate
模型
| M-184 triage assistant | 公共服务 | 高 | 19 of 24 suites | 已阻止 |
| M-320 flood nowcast | 科学 | 中 | 12 / 12 | 已发布 |
| M-157 document search | 内部 | 低 | 8 of 8 | 已发布 |
| M-563 agent planner | 智能体式 | 高 | 12 of 16 | 评估中 |
模型卡
- 预期用途
- 分诊辅助，由人决定
- 不在范围内
- 资格决定
- 已知局限
- 在罕见案例上的准确率较低
- 负责人
- 模型所有者
事件日志
- D-21INC摘要中出现虚构引用
- D-9MIT已增加检索依据
- D-2INC通过上传文件的提示注入
- D-1MIT工具权限已收窄
Evaluation scores
vs release threshold
Evaluation must reflect the domain where a model is used.
Evaluations are designed for the Labs 证据系列. The International AI Safety Report (2025) describes the limits of current evaluations; NIST’s AI RMF and its generative AI profile (2024) set measurement practice. Models would carry cards under NS-12 of the proposed Nexus Standards.
- Held-out test sets for a domain.
- Evaluation protocols with known failure cases.
How model evaluation is modelled, measured and tested.
模型、其假设、数据和不确定性，以科学家或工程师会核查的形式呈现。结构遵循已发表的方法。所取数值仅为示意，不是研究发现。
模型、方程与假设
The model structure used for model evaluation, the assumptions it makes and how each is tested.
Ri = f(Hi, Ei, Vi, Ci)危害、风险暴露、脆弱性和应对能力EAL = ∑k pk · Lk对年概率为 pkxt+1 = A xt + B ut + εtA carries the couplings, u the shock, ε the error| 假设 | 为何制作 | 如何检验 |
|---|---|---|
| 在一次事件中，危害与暴露相互独立 | 使损失积分保持可计算 | 与基于联合记录拟合的 copula 进行比较 |
| 耦合在工作点附近是线性的 | 让冲击通过 A 传播 | 在线性范围之外施加扰动并比较 |
| 脆弱性曲线可以在不同地点之间迁移使用 | 很少有本地曲线可用 | 在有观测损失的地方据此进行验证 |
- 输入
- 带溯源信息的版本化数据集
- 成果
- 分布，绝不是单一数字
- 代码
- 连同模型卡和测试一起发布
- 复现
- 相同的数据、代码、环境和随机种子
衡量什么，以及衡量得如何。
每个指标都注明其单位、频率、背后的来源类型以及数值将附带的证据标签。此处不显示数值：数值由计划依据其自身权限提供。
| 指标 | 单位 | 节奏 | 来源类型 | 标签 |
|---|---|---|---|---|
| Benchmark score with uncertainty | score with confidence interval | per evaluation | Published benchmark results | 已报告 |
| Contamination checks reported | percent of evaluations reporting checks | per evaluation | Evaluation reports | 已声明 |
| Red-team findings by severity | count by severity class | per exercise | Red-team reports | 已报告 |
| Performance gap between subgroups | difference in error rates | per evaluation | Disaggregated evaluation | 观测 |
| Post-deployment incident reports | count by type | 每月 | Incident monitoring | 已报告 |
| 校准误差 | expected calibration error | per evaluation | Evaluation on held-out data | 观测 |
证据来自何处。
具名公共数据集与机构记录，各自注明许可类别。依协议获得的来源仅按其条款使用，绝不再次发布。
| 来源 | 它提供什么 | 许可类别 |
|---|---|---|
| Stanford HELM (Holistic Evaluation of Language Models) | Open, multi-metric evaluations of language models | Open source, Apache 2.0 for code; results public |
| MLCommons benchmarks, including AILuminate | Standardised performance and safety benchmarks | Public results, MLCommons terms (to check) |
| 《国际人工智能安全报告》 | Synthesis of evidence on capabilities, risks and evaluation limits | Public, terms per publisher |
| Model cards and system cards from developers | Developer-reported evaluations and limitations | Public documents, terms per developer |
| Epoch AI benchmark and model databases | Compiled benchmark results and model characteristics | 开放，CC BY 4.0 |
由谁决定，以及证据服务于什么。
人工智能与自主 系统 prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| 谁来决定 | 问题 | 本页面服务的证据 |
|---|---|---|
| 开发者 | Whether a model is ready for release and what limits to state | Evaluation protocols; red-team findings |
| Deploying organisations | Whether a model is fit for a specific use | Use-specific evaluations; subgroup performance |
| Regulators and AI safety institutes | What evaluation evidence to require | Evaluation standards and reporting templates |
该主题产生的产出
- Evaluation protocols with uncertainty and contamination checks
- Disaggregated evaluation designs for subgroup performance
- Red-team methods for public sector use cases
- Evaluation reporting templates
What evaluators ask.
- Does the test measure the capability or property that matters for the intended use (construct validity)?
- When the model reports high confidence, is it right as often as that confidence implies (calibration)?
- Does performance hold on data from different places, times or populations (generalisation)?
- Has test data leaked into training, inflating results (contamination)?
Why scores mislead.
Models learn statistical regularities in their training data, including spurious ones. A medical imaging model may learn the hospital’s scanner signature rather than the disease; a language model may recognise benchmark questions it saw during training. When deployment data differ from test data, a problem known as 分布偏移, performance can fall sharply and without warning.
Large general-purpose models add new difficulties. Their capabilities are broad and emergent, outputs vary with phrasing, and the line between capability and memorisation is blurred. Aggregate scores can hide failures concentrated in particular groups or tasks.
How evaluation is done, and its limits.
- 01
Held-out and out-of-distribution testing
Testing on data deliberately different from training, analysed with 概率与统计. No test set covers every deployment condition.
- 02
Calibration analysis
Comparing stated confidence with observed accuracy, and checking how it changes under shift.
- 03
Disaggregated evaluation
Reporting performance by subgroup and task rather than only on average. Small subgroups give uncertain estimates.
- 04
Human and expert evaluation
Structured assessment of open-ended outputs, using 社会与行为科学 methods for rater agreement. Human judgement is costly and can be inconsistent.
Where this page’s references lead.
阅读本页时所对照的机构和文件。状态和版本会变化；每个链接都指向发布机构。
问题与解答。
Why are benchmark scores not enough?
What is red teaming in AI?
What is data contamination?
Is a high benchmark score evidence a model is safe to deploy?
What is benchmark saturation?
Why evaluate after deployment too?
Will the platform publish evaluation protocols?
从本页贡献证据。
报告 is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- 电子邮件网关 GW-GCRI从本页发送证据、更正、案例、方法、审查、翻译或概况投稿。它会带上本页标签送达，由负责的编辑接收。
- Zenodo开放存缴存缴并提交到社区用您的 ORCID iD 在 Zenodo 上发布您自己的记录，然后提交到那里的 Reports 社区。由策展人审核；DOI 仍归您所有。
- 报告GCRI查看所有贡献方式贡献类型、各自的评审方式、您获得的署名、许可协议，以及绝不接受的内容。
切勿发送个人数据、敏感基础设施细节或您无权分享的任何内容。贡献在使用前会经过审查；发送贡献不赋予署名权或编辑权，赞助也绝不能换取研究结论。
法律文件。
本网站上的地图、数据和图表仅作为背景和一般信息展示。它们不是建议、预测或预警；信息的局限见免责声明和数据说明。