Artificial Intelligence and Autonomous Systems
The Global Centre for Risk and Innovation (GCRI)
Model evaluation asks what an AI model can do, how reliably, and where it fails. Benchmarks, red teaming, human studies and field monitoring each measure something different, and none fully predicts behaviour in deployment. This page sets out how evaluations are designed and where they mislead, how results should be reported, and what evidence serves the developers, deployers and regulators who rely on them.
GCRI · In development · service level R0
AI model evaluation is the measurement of what a model can do, how reliably and where it fails. Benchmarks, red teaming, human studies and monitoring after deployment each measure different things, and none fully predicts real use, so results carry uncertainty, contamination checks and stated limits.
For model evaluation, in GCRI's spatial-intelligence reading, the map opens on night lights, true-colour imagery and population density with true colour from NASA GIBS, framed on East Asia; 3 further NASA layers wait in the list.
The interactive map needs JavaScript; the layer list and sources are below.
Layers 3
Connected activity, for this page
Keyboard, with the map focused
Hold Ctrl or ⌘ and scroll to zoom
Latest available
NASA GIBS / EOSDIS · ASTER GDEM (METI, NASA) · Blue Marble (NASA) · © OpenStreetMap contributors (ODbL) · Natural Earth
Loading the latest NASA imagery.
Drag to move. Ctrl or ⌘ and scroll, pinch or the buttons to zoom; hover for the values under the pointer; reset returns to the region.
For model evaluation, in GCRI's spatial-intelligence reading, the map opens on night lights, true-colour imagery and population density with true colour from NASA GIBS, framed on East Asia; 3 further NASA layers wait in the list.Satellite and model products as NASA publishes them, each with its own date; indicators, not assessments of any place; claim ceiling INTEGRATED.
Layers and dates
The working screen behind this subject: its structure, terms and states. Names are placeholders and values illustrative.
Model evaluation · Model registry · Release gate
| M-184 triage assistant | Public service | High | 19 of 24 suites | Blocked |
| M-320 flood nowcast | Science | Medium | 12 of 12 | Released |
| M-157 document search | Internal | Low | 8 of 8 | Released |
| M-563 agent planner | Agentic | High | 12 of 16 | Evaluating |
vs release threshold
Validity, calibration and generalization. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Models quickly reach high scores on fixed benchmarks, and test items can leak into training data. High scores then say more about the benchmark than about general capability.
Evaluations use set prompts, settings and tasks. Real users phrase things differently, combine tools and face new situations, so measured performance may not transfer.
Red teaming and adversarial prompts reveal unsafe or unreliable behaviour that average-case metrics miss. Coverage depends on the testers' skill and the time allowed.
Model updates, changing inputs and new uses alter behaviour over time. Without monitoring, a model evaluated at release can behave differently months later.
Each term as this page uses it. Where a standard-setting body defines the term, the wording follows that body.
| Term | Definition |
|---|---|
| Benchmark | A fixed set of tasks and scoring rules used to compare model performance. |
| Benchmark saturation | The point at which models reach scores so high that a benchmark no longer distinguishes between them. |
| Calibration | The match between a model's stated confidence and how often it is actually correct. |
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Benchmark score with uncertainty | score with confidence interval | per evaluation | Published benchmark results | REPORTED |
| Contamination checks reported | percent of evaluations reporting checks | per evaluation | Evaluation reports | DECLARED |
| Red-team findings by severity | count by severity class | per exercise | Red-team reports | REPORTED |
| Performance gap between subgroups | difference in error rates | per evaluation | Disaggregated evaluation | OBSERVED |
| Post-deployment incident reports | count by type | monthly | Incident monitoring | REPORTED |
| Calibration error | expected calibration error | per evaluation | Evaluation on held-out data | OBSERVED |
The topic as a computational graph: named sources feed the methods, an evidence review gates the outputs, and the decisions stay with the authorities that hold them.
Fig. 02Nodes carry their type, two metric rows and a state; connectors show what feeds what; dashed frames name the owners; the timeline is the cadence. Design of the pipeline, not a record of a run.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Source | What it gives | Licence class |
|---|---|---|
| Stanford HELM (Holistic Evaluation of Language Models) | Open, multi-metric evaluations of language models | Open source, Apache 2.0 for code; results public |
| MLCommons benchmarks, including AILuminate | Standardised performance and safety benchmarks | Public results, MLCommons terms (to check) |
| International AI Safety Report | Synthesis of evidence on capabilities, risks and evaluation limits | Public, terms per publisher |
| Model cards and system cards from developers | Developer-reported evaluations and limitations | Public documents, terms per developer |
| Epoch AI benchmark and model databases | Compiled benchmark results and model characteristics | Open, CC BY 4.0 |
Artificial Intelligence and Autonomous Systems prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Who decides | The question | Evidence this page serves |
|---|---|---|
| Developers | Whether a model is ready for release and what limits to state | Evaluation protocols; red-team findings |
| Deploying organizations | Whether a model is fit for a specific use | Use-specific evaluations; subgroup performance |
| Regulators and AI safety institutes | What evaluation evidence to require | Evaluation standards and reporting templates |
Models learn statistical regularities in their training data, including spurious ones. A medical imaging model may learn the hospital's scanner signature rather than the disease; a language model may recognise benchmark questions it saw during training. When deployment data differ from test data, a problem known as distribution shift, performance can fall sharply and without warning.
Large general-purpose models add new difficulties. Their capabilities are broad and emergent, outputs vary with phrasing, and the line between capability and memorisation is blurred. Aggregate scores can hide failures concentrated in particular groups or tasks.
Testing on data deliberately different from training, analysed with probability and statistics. No test set covers every deployment condition.
Comparing stated confidence with observed accuracy, and checking how it changes under shift.
Reporting performance by subgroup and task rather than only on average. Small subgroups give uncertain estimates.
Structured assessment of open-ended outputs, using social and behavioural science methods for rater agreement. Human judgement is costly and can be inconsistent.
The current service level is R0: information and enquiries. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.