GCRI · Artificial Intelligence and Autonomous Systems
AI cyber security
AI and cybersecurity meet in three ways: AI systems can be attacked, AI can help attackers, and AI can help defenders. Models can be poisoned, stolen, manipulated or made to leak data; their supply chains of data, weights and libraries carry new risks. This page sets out how these threats work, how AI systems are secured, and what evidence serves the security teams and regulators responsible.
AI cyber security on the record.
AI cyber security: how the risk develops.
Adversarial methods and model exposure. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Models are attack surfaces
Adversarial inputs, prompt injection, data extraction and model theft target the model itself. Classic application vulnerabilities still apply around it.
The AI supply chain is long
Training data, pretrained weights, libraries and plugins come from many sources. Poisoned data or tampered models can introduce hidden behaviour.
AI lowers the cost of some attacks
Generative AI can draft phishing, translate lures and help write code, lowering the skill barrier for some attacks. The size of this effect is studied and debated.
AI assists defence
Machine learning detects anomalies, triages alerts and finds vulnerabilities. Defenders must also secure the AI tools they adopt.
A cascade through interdependent networks.
A hazard footprint takes out co-located assets in every layer; fragmentation, dependency between layers and overload of the survivors then spread the failure. The 3D view separates the layers by depth. Teaching model with stated illustrative values, not a prediction.
Controls
- Assets per layer
- 18
- Hazard footprint
- 8% of area
- Assets that depend on another layer
- 60%
- Spare capacity (tolerance)
- 30%
Readouts
- Assets hit by the hazard
- 7%
- Assets still working
- 50%
- Amplification of the initial damage
- 6.8 times
- Steps to settle
- 3
Numbers, methods and sources
| Step | Data centres working (%) | Networks working (%) | Power working (%) |
|---|---|---|---|
| 0 | 83 | 94 | 100 |
| 1 | 72 | 78 | 94 |
| 2 | 61 | 67 | 50 |
| 3 | 61 | 67 | 22 |
Methods
- Interdependent networks Buldyrev and others (2010), Catastrophic cascade of failures in interdependent networks, Nature 464
- Overload cascades Motter and Lai (2002), Physical Review E 66
- Betweenness load Brandes (2001), Journal of Mathematical Sociology 25
- Multilayer networks Kivela and others (2014), Journal of Complex Networks 2
- Infrastructure interdependencies Rinaldi, Peerenboom and Kelly (2001), IEEE Control Systems 21(6)
- Uncertainty fan (parameter Monte Carlo) Spiegelhalter, Pearson and Short (2011), Visualising uncertainty about the future, Science 333
- One-at-a-time sensitivity (tornado) Saltelli and others (2008), Global Sensitivity Analysis: The Primer, Wiley
Data
The model runs on the teaching values set by the controls. To calibrate it for a place, read it against these admitted layers and series, which this page’s map can draw:
- Major roads Natural Earth
- Railways Natural Earth
- Seaports Natural Earth
- Airports Natural Earth
- Urban extents Natural Earth
- Night lights tonight, outages (VIIRS Day/Night Band) NASA/NOAA VIIRS (Suomi NPP) · register nasa.gibs.context-live-layers
- Natural events (NASA EONET) NASA EONET · register nasa.eonet.events
Across the six pillars, led by Nexus Consortium
- Experiments Labs
- Built in Artificial Intelligence and Autonomous Systems in Foundry
- Tested at Artificial Intelligence and Autonomous Systems in Universe
- Codified in Digital, Cyber and AI in Standards
- Published in Artificial Intelligence and Autonomous Systems in Reports
- Taught in Faculty of Technology and Digital Systems in Academy
AI changes cyber risk in every domain’s control systems.
Findings are designed for the Exponential Technology series. ISO/IEC 23894 (2023) sets AI risk management guidance; the page publishes no attack techniques.
- Defensive evaluations of AI tools.
- Incident case studies.
How AI cyber security is modelled, measured and tested.
The model, its assumptions, its data and its uncertainty, in the form a scientist or engineer would check. Structures follow published methods. The values drawn are illustrative, not findings.
System model
The couplings between the systems in AI cyber security: signs of influence, a reinforcing and a balancing loop, and cross-links where stress jumps.
What is measured, and how well.
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Vulnerabilities reported in AI frameworks and libraries | count of CVE entries | monthly | Vulnerability databases | REPORTED |
| Adversarial robustness | accuracy under a stated attack | per evaluation | Robustness testing | OBSERVED |
| Models with signed provenance | percent of deployed models | per inventory | Model registries | DECLARED |
| AI-related security incidents | count by technique | annual | Incident reports | REPORTED |
| Time to detect AI system compromise | hours | per incident | Incident response records | OBSERVED |
| Organisations with AI security policies | percent of surveyed organisations | per survey | Industry surveys | REPORTED |
Where the evidence comes from.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Source | What it gives | Licence class |
|---|---|---|
| MITRE ATLAS | Adversarial tactics and techniques against AI systems, with case studies | Public, licence to check |
| NIST adversarial machine learning taxonomy | Terminology and taxonomy of attacks and mitigations | Public domain, US government |
| National Vulnerability Database and CVE records | Vulnerabilities in AI software | Public domain, US government |
| ENISA reports on AI cybersecurity | Threat landscape and good practice for securing AI | Public, ENISA terms |
| Joint guidance from national cyber agencies on secure AI system development | Secure design, development, deployment and operation | Public, terms per agency |
Who decides, and what the evidence serves.
Artificial Intelligence and Autonomous Systems prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Who decides | The question | Evidence this page serves |
|---|---|---|
| Chief information security officers | How to secure AI systems the organisation builds or buys | Threat models; supply chain checklists |
| National cybersecurity agencies | What guidance to issue on AI security | Incident and vulnerability evidence |
| Developers | Which defences to build into models and applications | Robustness testing; attack taxonomies |
Outputs this topic produces
- Threat models for AI deployments
- AI supply chain security checklists
- Robustness testing protocols
- Evidence notes on AI-enabled attack trends
What security researchers ask.
- How can inputs be crafted to make a model misbehave, and how robust are defences?
- Can an attacker poison training data or tamper with model weights in the supply chain?
- Can private training data or a model itself be extracted through its outputs?
- How does AI change attackers’ and defenders’ capabilities?
The AI attack surface.
Adversarial examples are inputs altered in ways imperceptible to people that cause models to err. Data poisoning inserts malicious examples into training data to plant backdoors. Membership inference and extraction attacks recover information about training data or replicate a model through repeated queries. For language models, jailbreaks and prompt injection bypass intended restrictions.
On the other side, AI can help attackers find vulnerabilities, write malware variants and craft convincing phishing at scale, and help defenders triage alerts, analyse code and detect anomalies. The balance between the two is an open research question.
Security engineering for AI.
- 01
Red-teaming
Structured attempts to break a system, combining cybersecurity and cryptography and artificial intelligence and machine learning expertise. Red-teaming finds problems but cannot prove their absence.
- 02
Threat modelling
Mapping assets, adversaries and attack paths across data, training, deployment and supply chain.
- 03
Robustness testing
Measuring performance under adversarial perturbation. Robustness to known attacks does not imply robustness to new ones.
- 04
Supply-chain integrity
Provenance of datasets, models and dependencies, with signing and verification. Provenance tooling is still maturing.
How it is studied, and what the methods cannot say.
- 01Cybersecurity and cryptographyThreat modelling AI applications · Model supply chain integrity · Injection defence testingApplied here
- 02AI evaluation and assurance engineeringAssurance cases for AI systems · Structured red teaming · Post-deployment monitoringApplied here
- 03Artificial intelligence and machine learningDistribution shift analysis · Fine-tuning for a specific use · Interpretability methodsApplied here
- 04High-performance, cloud and edge computingCompute access for public research · Energy measurement of workloadsApplied here
Where this page’s references lead.
The bodies and documents this page is read against. Status and editions change; each link goes to the issuing body.
Questions and answers.
What is data poisoning?
What is MITRE ATLAS?
Does AI make cyberattacks easier?
Does the platform publish attack techniques?
How do I report a vulnerability?
Is open release of model weights a security risk?
Can AI defend against AI-enabled attacks?
Contribute evidence from this page.
Reports is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- EmailGateway GW-GCRISend it from this pageEvidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen depositDeposit it and submit to the communityPublish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsGCRISee every way to contributeThe contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.
Bring a method, a dataset or a question.
Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.
Legal documents.
Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.