带空间验证的地球观测模型
Land cover, building and damage models are trained and tested on spatially separated blocks, so reported accuracy reflects new places rather than the neighbours of training pixels.
GCRI · Foundry
Artificial intelligence and machine learning extend detection, classification and forecasting across large archives, and agents extend them into action. In this field the method is also a source of risk: models that fail when conditions shift, outputs that sound certain when they are not, and tools that turn an error into an effect.
Artificial intelligence and machine learning develop algorithms that learn patterns from data to classify, predict or generate outputs. They are increasingly used for hazard detection, forecasting and decision support, and are themselves a source of risk. In data and compute and methods and modelling, machine learning can scale detection and forecasting across interconnected systems. Its own failure modes must be weighed when automated outputs inform consequential decisions. The canonical description is on the STEM Commons.
Land cover, building and damage models are trained and tested on spatially separated blocks, so reported accuracy reflects new places rather than the neighbours of training pixels.
Each model version carries its intended use, training data, evaluation by region and subgroup, known limits and red-team results, and is promoted only with a recorded review.
Agents run analyses with a declared tool set, data classes, budget and kill-switch enforced at the policy gate, and every action produces a receipt for a reviewer.
团队遵循该方法时会使用的界面:其指标、状态与决策。名称为示例,数值仅作示意。
Foundry · AI and machine learning at Foundry · Run R-6794
| 全部 | 0.83 | 0.02 | 5712 | 通过 |
| 区域 A | 0.74 | 0.05 | 1098 | 审核 |
| 区域 B | 0.76 | 0.05 | 458 | 审核 |
| 稀有类别 | 0.64 | 0.07 | 160 | 低于标准 |
| 数据 | 角色 | 标签 |
|---|---|---|
| 带标注的训练集 | 监督学习 | 已报告 |
| Satellite image archives | 分类的输入 | 观测 |
| 留出测试集和分布外测试集 | 评估 | 观测 |
| Evaluation and red-team records | Promotion decisions | 已报告 |
| 训练数据的数据表 | Origin, consent and licence | 已声明 |
模型、其假设、数据和不确定性,以科学家或工程师会核查的形式呈现。结构遵循已发表的方法。所取数值仅为示意,不是研究发现。
Foundry 用于人工智能和机器学习的模型结构、其所作的假设,以及每项假设如何测试。
Ri = f(Hi, Ei, Vi, Ci)危害、风险暴露、脆弱性和应对能力EAL = ∑k pk · Lkover events k with annual probability pkxt+1 = A xt + B ut + εtA carries the couplings, u the shock, ε the error| 假设 | 为何制作 | 如何检验 |
|---|---|---|
| 在一次事件中,危害与暴露相互独立 | 使损失积分保持可计算 | 与基于联合记录拟合的 copula 进行比较 |
| 耦合在工作点附近是线性的 | 让冲击通过 A 传播 | 在线性范围之外施加扰动并比较 |
| 脆弱性曲线可以在不同地点之间迁移使用 | 很少有本地曲线可用 | 在有观测损失的地方据此进行验证 |
模型在尊重空间、时间与群体结构的留出数据上进行评估。在测试准确性的同时,也测试稳健性、校准与公平性。
| 核查 | 如何开展 |
|---|---|
| 空间与时间留出验证 | 测试数据在空间和时间上与训练数据分离,以防止数据泄漏 |
| Probability calibration | Predicted probabilities are compared with observed frequencies on held-out data |
| 漂移监测 | 将使用中的输入和输出分布与验证时的分布进行比较,出现偏离即触发审查 |
| 对抗性测试 | 模型和智能体在晋级前须经过提示注入和滥用测试 |
当部署条件与训练数据不同时,性能会下降。模型可能固化偏见,并在罕见事件上毫无明显预警地失效。
Process explorer · Process
9 steps·5 stages·1 gate
流程
AI RMF 核心包含四项职能:GOVERN、MAP、MEASURE 和 MANAGE,其中 GOVERN 贯穿其他各项;图中所示顺序只是一种常见的解读方式,并非检查清单。
对已发布的外部标准或方法的中立概述,内容来自其发布机构。并非运行记录。Nexus 不执行、认证或批准所示的任何步骤。关卡或决定归属于来源所指明的一方。这些行动不一定构成有先后顺序的步骤。本流程图未纳入欧盟《人工智能法案》。
Reports 是联合体的季刊。您对该主题的了解可以成为下一期中的证据、案例或更正,署名归您,并在属于您本人作品时以 CC BY 4.0 发布。
切勿发送个人数据、敏感基础设施细节或您无权分享的任何内容。贡献在使用前会经过审查;发送贡献不赋予署名权或编辑权,赞助也绝不能换取研究结论。
本网站上的地图、数据和图表仅作为背景和一般信息展示。它们不是建议、预测或预警;信息的局限见免责声明和数据说明。
The Global Centre for Risk and Innovation (GCRI Canada) is a Canadian not-for-profit organisation, not a registered charity, with its registered office in Ontario.