GCRI · Labs
科学工作流中的人工智能
Machine learning now enters science at every step: screening literature, designing experiments, analysing images and sequences, emulating simulations and drafting text. Each use can speed discovery and each can introduce errors that look like findings. This page sets out how models are evaluated for scientific use, where leakage, benchmark overfitting and silent drift arise, and what evidence editors, funders and regulators need before they rely on a model-assisted result.
工作如何开展。
Testbed operations
AI in scientific workflows · Field lab coast · Campaign 5
Sensor deployment
示意海岸线
Sensors
| S-01 tide | 94 | 0.0 | Nominal |
| S-04 salinity | 76 | 0.4 | Recalibrate |
| S-07 wave | 95 | 0.1 | Nominal |
Sensor vs reference
图Testbed operations: example names and illustrative values, not a record or a live system.
AI in scientific workflows: how the risk develops.
Model evaluation and scientific assistance. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Leakage inflates performance
When information from the test set reaches training, through duplicates, shared patients, neighbouring sites or preprocessing fitted on all the data, a model appears more accurate than it is. Leakage has been documented in many fields that adopted machine learning.
Benchmarks get overfitted
Repeated tuning against the same public benchmark selects models that fit its quirks. Gains on a leaderboard can vanish on new data, and contamination of training corpora with benchmark items makes scores hard to interpret.
Drift and distribution shift
Instruments, protocols, populations and software change. A model validated on last year’s data or another lab’s scanner can degrade without any visible error. Monitoring after deployment is part of the evaluation, not an extra.
Generated text and fabricated sources
Language models can produce fluent text, citations and code that are wrong. Unchecked, these errors enter manuscripts, peer reviews and proposals, and spread through citation. Disclosure and verification policies exist because such errors are hard to see.
How AI in scientific workflows is modelled, measured and tested.
模型、其假设、数据和不确定性，以科学家或工程师会核查的形式呈现。结构遵循已发表的方法。所取数值仅为示意，不是研究发现。
模型、方程与假设
The model structure used for AI in scientific workflows, the assumptions it makes and how each is tested.
Ri = f(Hi, Ei, Vi, Ci)危害、风险暴露、脆弱性和应对能力EAL = ∑k pk · Lk对年概率为 pkxt+1 = A xt + B ut + εtA carries the couplings, u the shock, ε the error| 假设 | 为何制作 | 如何检验 |
|---|---|---|
| 在一次事件中，危害与暴露相互独立 | 使损失积分保持可计算 | 与基于联合记录拟合的 copula 进行比较 |
| 耦合在工作点附近是线性的 | 让冲击通过 A 传播 | 在线性范围之外施加扰动并比较 |
| 脆弱性曲线可以在不同地点之间迁移使用 | 很少有本地曲线可用 | 在有观测损失的地方据此进行验证 |
- 输入
- 带溯源信息的版本化数据集
- 成果
- 分布，绝不是单一数字
- 代码
- 连同模型卡和测试一起发布
- 复现
- 相同的数据、代码、环境和随机种子
AI in scientific workflows changes each domain’s methods.
Findings feed the 方法与技术系列. AI use follows NIST’s AI RMF and is disclosed; models carry NS-12 cards under the proposed Nexus Standards.
- Workflow evaluations.
- Disclosed AI use cases.
衡量什么，以及衡量得如何。
每个指标都注明其单位、频率、背后的来源类型以及数值将附带的证据标签。此处不显示数值：数值由计划依据其自身权限提供。
| 指标 | 单位 | 节奏 | 来源类型 | 标签 |
|---|---|---|---|---|
| Held-out evaluation design stated | percent of model-based studies | 年度 | Audit against the REFORMS checklist | 观测 |
| Code and model weights available | percent of studies with runnable code and weights | 年度 | Repository and article audit | 观测 |
| External validation | percent of models tested on data from another site or period | 年度 | Literature review | 估算 |
| 校准误差 | expected calibration error, dimensionless | per model | Held-out evaluation | 观测 |
| Performance gap under shift | difference in the primary metric between internal and external test sets | per model | Validation reports | 观测 |
| AI use disclosure | percent of submissions with a disclosure statement | 年度 | Journal submission records | 已声明 |
| Contamination checks reported | percent of benchmark evaluation reports | 年度 | Evaluation reports | 已报告 |
证据来自何处。
具名公共数据集与机构记录，各自注明许可类别。依协议获得的来源仅按其条款使用，绝不再次发布。
| 来源 | 它提供什么 | 许可类别 |
|---|---|---|
| OpenML | Datasets, tasks and evaluation runs with standardised splits | Open, CC BY |
| Hugging Face Hub model and dataset cards | Documented models and datasets with stated training data and limitations | Licence per model or dataset |
| MLCommons benchmarks | Shared benchmarks and measurement rules for model performance | Open, Apache 2.0 |
| AI Incident Database (Responsible AI Collaborative) | Reported failures and harms of deployed AI 系统 | 开放，须署名 |
| Software Heritage | Archived source code with persistent intrinsic identifiers | Open archive, licence per repository |
| arXiv metadata | Preprints across physics, computing and quantitative biology | Metadata CC0; licence per paper |
| Zenodo (CERN) | Citable deposits of code, models, data and evaluation outputs | Open, licence per record |
由谁决定，以及证据服务于什么。
实验室 prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| 谁来决定 | 问题 | 本页面服务的证据 |
|---|---|---|
| Journal editors and reviewers | Whether a model-based claim rests on an evaluation that excludes leakage and tests generalisation | REFORMS or TRIPOD+AI checklist, split design, external validation, code availability |
| 研究资助方 | Whether applicants and reviewers may use generative AI, and on what disclosure terms | Funder policy, disclosure records, documented verification of generated content |
| Medicines and device regulators | Whether an AI component in a medical product is validated for its intended use | Prespecified evaluation, external validation, predetermined change control plan |
| Research integrity offices | Whether an allegation involving fabricated or generated content is substantiated | Provenance of text, data and code; version history; tool logs |
该主题产生的产出
- Evaluation protocols that fix splits, metrics and baselines before training
- Model cards and datasheets with intended use, limits and known failure modes
- Leakage and contamination audits of model-assisted studies
- Reproducible pipelines with code, weights, environment and data versions archived
What experts ask about AI in research.
- What task is the model doing, and what is it compared against?
- Was the model evaluated on data independent of the data it was trained or tuned on?
- How does performance change when conditions shift away from the training data?
- Can the output be checked by a person or a simpler method, and is it?
- Which steps of the workflow were assisted, and is that disclosed in the result?
Where AI helps and where it misleads.
Machine learning models learn statistical regularities from data. They can be very effective where those regularities are stable and data are plentiful, for example classifying land cover from satellite imagery or emulating a climate model to explore many scenarios. They are weaker where data are sparse, where the system changes over time or where rare events matter most, which describes much of risk science.
Common failure modes are well understood: leakage, where information from the test set reaches training; 分布偏移, where conditions in use differ from those in training; shortcut learning, where a model relies on a spurious cue; and overconfident outputs without calibrated uncertainty. Generative models add the risk of fluent but fabricated content, including citations and code that looks correct but is not.
Workflow effects matter as much as model accuracy. When a tool is used at several stages, small errors can accumulate, and reviewers may give automated outputs more trust than they deserve.
How AI tools are evaluated.
- 01
Held-out and out-of-distribution tests
Evaluation on data separated by time, place or source. This is the minimum for any claim of generalisation.
- 02
Baselines
Comparison with simple statistical methods and expert judgement. A complex model that does not beat a simple baseline is not adding value.
- 03
Calibration and uncertainty
Checks that stated confidence matches observed accuracy, using 不确定性量化.
- 04
Provenance and disclosure
A record of which model, version and prompts or settings were used at each step, so the workflow can be re-run or audited.
Benchmark scores rarely transfer directly to a new research setting, and a model’s behaviour can change between versions. Evaluation is repeated when either the task or the model changes.
Where this page’s references lead.
阅读本页时所对照的机构和文件。状态和版本会变化；每个链接都指向发布机构。
问题与解答。
What is data leakage?
Is a high benchmark score enough to publish a method?
Can a generative AI tool be listed as an author?
Why publish model weights and code?
Does Labs use generative AI to write findings?
Can Labs evaluate a commercial AI model?
Why compare with simple baselines?
从本页贡献证据。
报告 is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- 电子邮件网关 GW-GCRI从本页发送证据、更正、案例、方法、审查、翻译或概况投稿。它会带上本页标签送达，由负责的编辑接收。
- Zenodo开放存缴存缴并提交到社区用您的 ORCID iD 在 Zenodo 上发布您自己的记录，然后提交到那里的 Reports 社区。由策展人审核；DOI 仍归您所有。
- 报告GCRI查看所有贡献方式贡献类型、各自的评审方式、您获得的署名、许可协议，以及绝不接受的内容。
切勿发送个人数据、敏感基础设施细节或您无权分享的任何内容。贡献在使用前会经过审查；发送贡献不赋予署名权或编辑权，赞助也绝不能换取研究结论。
法律文件。
本网站上的地图、数据和图表仅作为背景和一般信息展示。它们不是建议、预测或预警；信息的局限见免责声明和数据说明。