GCRI · 人工智能与自主系统
AI 网络安全
AI and cybersecurity meet in three ways: AI 系统 can be attacked, AI can help attackers, and AI can help defenders. Models can be poisoned, stolen, manipulated or made to leak data; their supply chains of data, weights and libraries carry new risks. This page sets out how these threats work, how AI systems are secured, and what evidence serves the security teams and regulators responsible.
AI cyber security: how the risk develops.
Adversarial methods and model exposure. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Models are attack surfaces
Adversarial inputs, prompt injection, data extraction and model theft target the model itself. Classic application vulnerabilities still apply around it.
The AI supply chain is long
Training data, pretrained weights, libraries and plugins come from many sources. Poisoned data or tampered models can introduce hidden behaviour.
AI lowers the cost of some attacks
Generative AI can draft phishing, translate lures and help write code, lowering the skill barrier for some attacks. The size of this effect is studied and debated.
AI assists defence
Machine learning detects anomalies, triages alerts and finds vulnerabilities. Defenders must also secure the AI tools they adopt.
在相互依存的网络中的一次级联。
A 危害足迹 会摧毁每一层中位于同处的资产；随后，碎片化、层间依赖和幸存资产的过载使失效扩散开来。3D 视图按深度分开各层。这是带有说明性取值的教学模型，不是预测。
控制措施
- 每个图层的资产
- 18
- 危害足迹
- 占面积的 8%
- 依赖其他图层的资产
- 60%
- 备用容量（容差）
- 30%
读数
- 受危害冲击的资产
- 7%
- 仍在运行的资产
- 50%
- 初始损害的放大
- 6.8 times
- 结算步数
- 3
数字、方法与来源
| 步骤 | 运行中的数据中心（%） | 运行中的网络（%） | 电力运行（%） |
|---|---|---|---|
| 0 | 83 | 94 | 100 |
| 1 | 72 | 78 | 94 |
| 2 | 61 | 67 | 50 |
| 3 | 61 | 67 | 22 |
方法
- 相互依存网络 Buldyrev 等（2010），Catastrophic cascade of failures in interdependent networks，Nature 464
- 过载级联 Motter and Lai (2002), Physical Review E 66
- 介数负载 Brandes (2001), Journal of Mathematical Sociology 25
- 多层网络 Kivela and others (2014), Journal of Complex Networks 2
- 基础设施相互依赖 Rinaldi, Peerenboom and Kelly (2001), IEEE Control Systems 21(6)
- 不确定性扇形图（参数蒙特卡罗） Spiegelhalter, Pearson and Short (2011), Visualising uncertainty about the future, Science 333
- 单因素敏感性（龙卷风图） Saltelli and others (2008), Global Sensitivity Analysis: The Primer, Wiley
数据
The model runs on the teaching values set by the controls. To calibrate it for a place, read it against these admitted layers and series, which this page’s map can draw:
- 主要道路 Natural Earth
- 铁路 Natural Earth
- 海港 Natural Earth
- 机场 Natural Earth
- 城市范围 Natural Earth
- 今晚的夜间灯光、停电（VIIRS 昼夜波段） NASA/NOAA VIIRS（Suomi NPP）· 登记 nasa.gibs.context-live-layers
- 自然事件（NASA EONET） NASA EONET · 登记 nasa.eonet.events
贯穿六大支柱，由 Nexus Consortium 牵头
AI changes cyber risk in every domain’s control systems.
Findings are designed for the 指数级技术系列. ISO/IEC 23894 (2023) sets AI risk management guidance; the page publishes no attack techniques.
- Defensive evaluations of AI tools.
- Incident case studies.
How AI cyber security is modelled, measured and tested.
模型、其假设、数据和不确定性，以科学家或工程师会核查的形式呈现。结构遵循已发表的方法。所取数值仅为示意，不是研究发现。
系统模型
The couplings between the systems in AI cyber security: signs of influence, a reinforcing and a balancing loop, and cross-links where stress jumps.
衡量什么，以及衡量得如何。
每个指标都注明其单位、频率、背后的来源类型以及数值将附带的证据标签。此处不显示数值：数值由计划依据其自身权限提供。
| 指标 | 单位 | 节奏 | 来源类型 | 标签 |
|---|---|---|---|---|
| Vulnerabilities reported in AI frameworks and libraries | count of CVE entries | 每月 | Vulnerability databases | 已报告 |
| Adversarial robustness | accuracy under a stated attack | per evaluation | Robustness testing | 观测 |
| Models with signed provenance | percent of deployed models | per inventory | Model registries | 已声明 |
| AI-related security incidents | count by technique | 年度 | Incident reports | 已报告 |
| Time to detect AI system compromise | hours | per incident | Incident response records | 观测 |
| Organisations with AI security policies | percent of surveyed organisations | per survey | Industry surveys | 已报告 |
证据来自何处。
具名公共数据集与机构记录，各自注明许可类别。依协议获得的来源仅按其条款使用，绝不再次发布。
| 来源 | 它提供什么 | 许可类别 |
|---|---|---|
| MITRE ATLAS | Adversarial tactics and techniques against AI 系统, with case studies | Public, licence to check |
| NIST adversarial machine learning taxonomy | Terminology and taxonomy of attacks and mitigations | Public domain, US government |
| National Vulnerability Database and CVE records | Vulnerabilities in AI software | Public domain, US government |
| ENISA reports on AI cybersecurity | Threat landscape and good practice for securing AI | Public, ENISA terms |
| Joint guidance from national cyber agencies on secure AI system development | Secure design, development, deployment and operation | Public, terms per agency |
由谁决定，以及证据服务于什么。
人工智能与自主 系统 prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| 谁来决定 | 问题 | 本页面服务的证据 |
|---|---|---|
| Chief information security officers | How to secure AI systems the organisation builds or buys | Threat models; supply chain checklists |
| National cybersecurity agencies | What guidance to issue on AI security | Incident and vulnerability evidence |
| 开发者 | Which defences to build into models and applications | Robustness testing; attack taxonomies |
该主题产生的产出
- Threat models for AI deployments
- AI supply chain security checklists
- Robustness testing protocols
- Evidence notes on AI-enabled attack trends
What security researchers ask.
- How can inputs be crafted to make a model misbehave, and how robust are defences?
- Can an attacker poison training data or tamper with model weights in the supply chain?
- Can private training data or a model itself be extracted through its outputs?
- How does AI change attackers’ and defenders’ capabilities?
The AI attack surface.
Adversarial examples are inputs altered in ways imperceptible to people that cause models to err. Data poisoning inserts malicious examples into training data to plant backdoors. Membership inference and extraction attacks recover information about training data or replicate a model through repeated queries. For language models, jailbreaks and prompt injection bypass intended restrictions.
On the other side, AI can help attackers find vulnerabilities, write malware variants and craft convincing phishing at scale, and help defenders triage alerts, analyse code and detect anomalies. The balance between the two is an open research question.
Security engineering for AI.
- 01
Red-teaming
Structured attempts to break a system, combining 网络安全与密码学 and artificial intelligence and machine learning expertise. Red-teaming finds problems but cannot prove their absence.
- 02
威胁建模
Mapping assets, adversaries and attack paths across data, training, deployment and supply chain.
- 03
Robustness testing
Measuring performance under adversarial perturbation. Robustness to known attacks does not imply robustness to new ones.
- 04
Supply-chain integrity
Provenance of datasets, models and dependencies, with signing and verification. Provenance tooling is still maturing.
Where this page’s references lead.
阅读本页时所对照的机构和文件。状态和版本会变化；每个链接都指向发布机构。
问题与解答。
What is data poisoning?
What is MITRE ATLAS?
Does AI make cyberattacks easier?
Does the platform publish attack techniques?
How do I report a vulnerability?
Is open release of model weights a security risk?
Can AI defend against AI-enabled attacks?
从本页贡献证据。
报告 is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- 电子邮件网关 GW-GCRI从本页发送证据、更正、案例、方法、审查、翻译或概况投稿。它会带上本页标签送达，由负责的编辑接收。
- Zenodo开放存缴存缴并提交到社区用您的 ORCID iD 在 Zenodo 上发布您自己的记录，然后提交到那里的 Reports 社区。由策展人审核；DOI 仍归您所有。
- 报告GCRI查看所有贡献方式贡献类型、各自的评审方式、您获得的署名、许可协议，以及绝不接受的内容。
切勿发送个人数据、敏感基础设施细节或您无权分享的任何内容。贡献在使用前会经过审查；发送贡献不赋予署名权或编辑权，赞助也绝不能换取研究结论。
法律文件。
本网站上的地图、数据和图表仅作为背景和一般信息展示。它们不是建议、预测或预警；信息的局限见免责声明和数据说明。