Skip to content

GCRI · Foundry

Ajan û otomasyon

AI agents can draft analyses, run pipelines and query data at a speed no team can match, which is why their authority must be explicit. An agent that can call tools can call the wrong one, read data it should not, follow instructions planted in a document, or present a draft as reviewed evidence. This page sets out how agent risk develops and spreads, how agents are evaluated before they receive authority, and the controls that let people stop them.

02Pirs

Agents and automation: how the risk develops.

AI tools, authority and risk controls. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.

Mekanîzma

Tool reach multiplies impact

Each tool an agent can call adds a path from a model error to a real effect: a query, a write, a message, a run. Risk grows with the number of tools and the privileges behind them, not with the model alone.

Mekanîzma

Instructions arrive in data

Text retrieved from a web page, a document or a dataset can carry instructions the agent then follows. Prompt injection turns ordinary content into a control channel, so tool permissions must never depend on the agent recognising it.

Mekanîzma

Fluent error looks like evidence

Language models produce confident, well-formatted output that can be wrong, including invented citations and figures. Once an unlabelled draft enters a report, it borrows the authority of the report and the people who signed it.

Mekanîzma

Automation chains

One agent’s output becomes another’s input, or triggers a pipeline, with no person in between. An early error is copied, transformed and amplified along the chain, and the receipt trail is the only way back to where it started.

03Çavdêrî bike

Konsola cihê kar û karan.

Ekrana ku tîmek dê ji bo şopandina vê mijarê bikar bîne: pîvan, rewş û biryarên wê. Nav mînak in û nirx nîşander in.

Konsola cihê kar û karan

Agents and automation · Project space 12 · Cluster east

  • Kar dixebitin
    31Çalak
  • Li bendê ma, p50
    8minNormal
  • Xebatên îro
    35924
  • Kota hatiye bikaranîn, % vê mehê
    64Çavdêrî
IllustrativeGCRI · Foundry

Kar

J-86161Ansambla modelên lehiyêXwediyê projeyê00:37:12Dixebite
J-42787Xêzkirina kaşêlan, zoom 0 heta 12Cihê xebatê yê GIS00:02:40Dixebite
J-76872Derxistina delîlanBerpirsê modelê00:00:47Qediya
J-55145Berhevoka bangên amûrê yên ajanêQada ceribandinê ya ajanan00:01:16Astengkirina siyasetê

Bûyerên siyasetê

  1. 10:25DerîBangeya ajanê pejirandina mirovî dixwaze
  2. 10:51AgahîTomara xebatê hatiye mohrkirin
  3. 11:34Hişyar bikeKota li 80 ji sedî ji bo projeyê
Workspace and job console: the screen’s structure, terms and states for Agents and automation.Example names and illustrative values; a design, not a record or a live system.
04In the five domains

Automation in Foundry is designed to keep humans in charge in every domain.

05Rêbaz û delîl

How agents and automation is modelled, measured and tested.

Model, texmînên wî, daneyên wî û nediyariya wî, bi şeweyê ku zanyar an endezyar kontrol bike. Avahî rêbazên weşandî dişopînin. Nirxên hatine xêzkirin nimûneyî ne, ne encam in.

Model, hevkêşe û texmîn

The model structure used for agents and automation, the assumptions it makes and how each is tested.

Xetera hêmana iRi = f(Hi, Ei, Vi, Ci)metirsî, rûbirûbûn, lawazî û kapasîteya serederiyê
Windahiya salane ya hêvîkirîEAL = ∑k pk · Lkover events k with annual probability pk
Pergalên girêdayîxt+1 = A xt + B ut + εtA carries the couplings, u the shock, ε the error
TexmînÇima tê çêkirinÇawa tê ceribandin
Metirsî û rûbirûbûn di nav bûyerekê de serbixwe neEntegrala windahiyê ji hesabkirinê re maqûl dihêleBi lihevhatineke copula li ser tomarên hevbeş re bidin ber hev
Girêdan li nêzî xala xebatê xêzik inDihêle şok di A de belav bibinJi derveyî rêza xêzik tevlihev bike û bide ber hev
Xêzên lawaziyê di navbera cihan de têne veguhastinXêzikên herêmî kêm tên dîtinLi hember windahiyên çavdêrîkirî erê bike li cihê ku hene
Têketin
danehevên bi guhertoyan û çavkaniyê
Derketin
belavkirin, tu carî jimarên tenê
Kod
bi qerta modelê û ceribandinan re hat weşandin
Zêdebûn
heman dane, kod, jîngeh û tov
Formulasyonên standard ên xetereyê û qada rewşê; texmînên navborî ji bo rêbazê tîpîk in, ji bo nirxandinê hatine diyarkirin, ne encam in.
FAIR guiding principles · GO FAIR and FORCE11W3C PROV · World Wide Web ConsortiumGuidance on consistent treatment of uncertainties · IPCCCEOS Analysis Ready Data · Committee on Earth Observation Satellites
06Nîşander

What is measured, and how well.

Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.

NîşaneYekîneRîtmSource typeNîşan
Active agents with a current authority recordproportion of active agentsheftaneAuthority registerDECLARED
Tool calls denied at the policy gatecount per agent per perioddailyPolicy gate logsÇAVDÊRÎKIRÎ
Time from kill-switch activation to haltsecondsper drill or eventGate and runtime logsÇAVDÊRÎKIRÎ
Agent outputs accepted after human reviewproportion of outputsmehaneReview recordsÇAVDÊRÎKIRÎ
Red-team findings open at promotioncount by severityper model versionEvaluation recordsHATIYE RAPORKIRIN
Unsupported claims found in sampled outputscount per samplemehaneReviewer auditÇAVDÊRÎKIRÎ
Agent actions with a complete receiptproportion of actionsdailyReceipt storeÇAVDÊRÎKIRÎ
07Çavkanî

Where the evidence comes from.

Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.

ÇavkanîWhat it givesLicence class
AI Incident DatabaseDocumented real-world AI failures and harms for hazard identificationOpen, terms of the database
OECD AI Incidents MonitorReported AI incidents and hazards drawn from news sourcesTerms of the OECD
MITRE ATLASAdversary tactics and techniques against AI systemsFree, terms of use
OWASP Top 10 for large language model applicationsCommon vulnerability classes in language model and agent systemsCC BY-SA
NIST AI RMF PlaybookSuggested actions for the govern, map, measure and manage functionsGiştî
Model cards published with open modelsIntended use, evaluation and known limits of a base modelLicence per model
08Biryar

Who decides, and what the evidence serves.

Foundry prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.

Kî biryarê didePirsEvidence this page serves
Supervising institutionsWhether an agent may act in their project, with which tools and on which dataAuthority record, evaluation record, open red-team findings
Data ownersWhether their data may be read by agents or used to train or tune modelsData classes in scope; the recorded training decision
ReviewersWhether an agent’s draft may be accepted into evidence or a reportReceipts, sources cited and checked, evidence labels
AI governance leads and data protection authoritiesWhether automated processing meets applicable law and the institution’s AI policyImpact assessment, logs, the human oversight record

Outputs this topic produces

  • Authority records per agent: purpose, tools, data classes, budget, kill-switch and supervisor
  • Evaluation and red-team records at each promotion
  • Receipts for every agent action
  • Labelled drafts that remain assertions until a person reviews them
09Core questions

What AI and safety engineers ask.

  • What exactly is this agent permitted to do, on which data, in which project?
  • How was the agent and its model evaluated for this use, and by whom?
  • Can every agent action be attributed to an identity, a configuration and a supervising institution?
  • How quickly can a person stop it, and is that stop enforced outside the agent?
10Mechanisms

Authority records, gates and receipts.

In the design of the NXS-NSF (trust, authorisation and assurance) Sovereignty Framework (NXS-NSF), every agent and tool holds an authority record that states its purpose, the tools it may call, the data classes it may touch, its rate limits and its kill-switch. Requests without an explicit policy decision fail closed. Agent actions pass the same policy gates and receive the same receipts as human actions, and scopes and kill-switches are enforced at the gate rather than left to the agent’s own configuration.

In NXS-LAB (research, experiments and simulations), agents may run analyses and draft artefacts, but each action is bound to an agent identity, model, tool set, budget and supervising institution, and remains an assertion until a person reviews it. In NXS-STU (composition, dashboards and workflows), agent behaviour inside a composition is declared with its purpose, tools and scope.

11Evaluation and limits

Evaluation comes before authority.

  1. 01

    Contract testing

    Integrations and agents are tested against their declared contracts before they receive authority in any tenant.

  2. 02

    Nirxandina modelê

    Promotion of a model requires evaluation records, including red-team results where the use calls for them.

  3. 03

    Data use

    No tenant data is used for training without a recorded decision by the data owner.

  4. 04

    Sînorên naskirî

    Evaluations sample behaviour and cannot exclude every failure; language models can produce fluent, wrong output; and tool-use risks grow with the number of tools an agent can reach. Narrow scopes and human review are the main mitigations.

Agents never take a decision that belongs to an institution or a competent authority, issue a warning, release funds, dispatch resources, sign an acceptance or present a draft as reviewed evidence.

12Rêbaz

How it is studied, and what the methods cannot say.

  1. Artificial intelligence and machine learningEarth observation models with spatial validation · Model cards and promotion recordsApplied here
  2. Software and safety-critical systems engineeringFail-closed policy gates · Build provenance and bills of materials · Assurance casesApplied here
  3. Systems engineeringCanonical system inventory · Contract-first interfaces · Requirements traced to evidenceApplied here
  4. Data engineering, interoperability and stewardshipValidation at the ingest boundary · Frozen data cuts for reproducible productsApplied here
13Where this page’s references

Where this page’s references lead.

Sazî û belgeyên ku ev rûpel li hember wan tê xwendin. Rewş û çap diguherin; her girêdan diçe ba saziya derxistinê.

14Pirs

Pirs û bersiv.

Who is accountable for an agent’s output?
The person who accepts it into evidence, a report or a decision, and the supervising institution named in the agent’s authority record. The agent itself is never the accountable party.
Why are permissions enforced outside the agent?
Because an agent can be manipulated by the content it reads. A policy gate the agent cannot reconfigure denies by default and holds even when the agent’s own reasoning has been subverted.
Is project data used to train models?
Not without a recorded decision by the data owner. Training, tuning and evaluation uses are declared separately in the source terms.
Can I connect my own agent?
Agent and integration access is designed to open with authorised projects, after contract testing against the declared tools and scopes.
rêzeya Pergalên StratejîkTevkarî bike

Ji vê rûpelê delîl bidin.

Reports weşana sê mehane ya Konsorsiyûmê ye. Tiştê ku tu li ser vê mijarê dizanî dikare di çapa pêş de bibe delîl, dosya an rastkirin, bi navê te û di bin CC BY 4.0 de weşandî li cihê ku karê te bi xwe be.

GCRIFoundryFoundry Build and Readiness
Foundry

Rêbazek, danehevek an pirsek bîne.

Agahî û pirs îro vekirî ne. Pirsek referansekê distîne û gava piştî wê dide nasîn; serlêdan, drav, veqetandina hesabkirinê û tomarkirin tenê dema ku tîm, mercên wê û kapasîteya vekolînê hebin vedibin.

The Global Centre for Risk and Innovation (GCRI Canada) is a Canadian not-for-profit organisation, not a registered charity, with its registered office in Ontario.