Skip to content

GCRI · Engineering

AI evaluation and assurance engineering

AI evaluation and assurance engineering develops methods to test, monitor and provide justified confidence in AI systems. It addresses performance, robustness, safety and societal impact across the system life cycle. Artificial Intelligence and Autonomous Systems stewards this page for the Nexus Consortium.

Make an enquiry Back to Engineering
01Contents

AI evaluation and assurance engineering in six parts.

In this dossier6 parts
  1. 01The methodWhat aI evaluation and assurance engineering covers
  2. 02Reference practiceAI assurance draws on risk frameworks, management systems and evaluation practice
  3. 03In systemic risk workEvidence that an AI system is fit for its use
  4. 04Applied inPlatforms that apply this method
  5. 05StewardshipWho maintains this page
  6. 06SourcesWhere this page’s references lead
02The method

What aI evaluation and assurance engineering covers.

  1. 01

    Core methods

    capability and safety evaluation; red teaming; robustness and adversarial testing; bias and fairness assessment; assurance case development; post-deployment monitoring.

  2. 02

    Data and instruments

    evaluation benchmarks; incident reports; model documentation; deployment logs; human feedback records.

  3. 03

    Validation

    Evaluation methods are checked for reliability and relevance to deployment conditions. Independent replication and third-party audits test claims.

  4. 04

    Limits

    Benchmarks may not reflect real-world use or novel capabilities. General-purpose systems have open-ended behaviour that resists complete testing.

  5. 05

    Use in risk decisions

    AI assurance determines whether automated systems can be trusted in high-consequence settings. It helps manage risks when AI is embedded across finance, health and infrastructure.

03Reference practice

AI assurance draws on risk frameworks, management systems and evaluation practice.

ReferenceWhat it sets
NIST AI RMFMeasure and manage functions
ISO/IEC 42001AI management system
EU AI Act conformity requirementsObligations for high-risk systems, applied in stages
OECD AI Incidents MonitorRecord of reported AI incidents
04Process

How the process runs.

Process

AI evaluation and assurance engineering: AI risk management (NIST)

The AI RMF Core has four functions, GOVERN, MAP, MEASURE and MANAGE, with GOVERN cross-cutting; the order drawn is one common reading, not a checklist.

  1. 01 Govern

    1. 01GOVERN: policies, roles and accountabilityStep · the Global Centre for Risk and Innovation (GCRI) technical officeGOVERN is a cross-cutting function. It is infused throughout AI risk management and enables the other functions.

  2. 02 Map

    1. 02MAP: context and intended useDatasetThe MAP function establishes the context to frame risks related to an AI system, including intended purposes and impacts on people.

  3. 03 Measure

    1. 03MEASURE: evaluate trustworthy characteristicsModelAI systems are evaluated for trustworthy characteristics. They should be tested before deployment and regularly in operation.
    2. 04MEASURE: track identified risks over timeModelMechanisms for tracking identified AI risks over time are in place, including unanticipated and emergent risks.

  4. 04 Manage

    1. 05MANAGE: treat risksStepRisk treatment plans cover response to, recovery from and communication about incidents. Resources go to mapped and measured risks.
    2. 06Proceed or not (MANAGE 1.1)Gate · GCRI technical officeA determination is made whether the system meets its purposes and objectives, and whether to proceed. The framework sets no thresholds.
    3. 07Deployment decisionDecision · GCRI technical officeThe organisation that owns the system decides whether and under what conditions to deploy.
    4. 08Post-deployment monitoring and incidentsModelPost-deployment monitoring plans are implemented. Incidents and errors are shared with relevant AI actors, including affected communities.

  5. 05 Record

    1. 09Inventory of AI systemsDatasetGOVERN 1.6 asks for mechanisms to inventory AI systems. A model card is common practice and is not named in the framework.

Neutral outline of a published external standard or method, from its issuing body. Not a record of operation. Nexus does not perform, certify or approve any step shown. A gate or decision belongs to the party the source names. The actions are not necessarily an ordered set of steps. The EU AI Act is not drawn in this flow.

05In systemic risk work

Evidence that an AI system is fit for its use.

Assurance builds an argument that a system is acceptably safe for a defined use, backed by evaluations, red teaming, monitoring and incident records. For systemic risk the use context matters most: the same model may be acceptable for drafting and unacceptable for triage.

  • Define the use context before evaluating.
  • Monitor after deployment; capability and use drift.
  • Record incidents and near misses.
  • Applications on AI and Autonomous Systems.
06Applied in

Platforms that apply this method.

07Stewardship

Who maintains this page.

Artificial Intelligence and Autonomous Systems stewards aI evaluation and assurance engineering: it keeps this description current and reviews how the method is applied across the platforms.

08Sources

Where this page’s references lead.

The bodies and documents this page is read against. Status and editions change; each link goes to the issuing body.

09Questions

Questions and answers.

Why does one institution host every method page?
GCRI hosts the STEM Commons so that each method has one canonical description. Platforms link to it rather than repeat it, which keeps definitions consistent.
How do I suggest a correction to a method page?
Write to the integrity concern form with the passage and the reason.
ReportsContribute

Contribute evidence from this page.

Reports is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.

GCRI

Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.

The Global Centre for Risk and Innovation

Bring a method, a dataset or a question.

Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.

Propose a method See all contact routes
Legal

Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.

Privacy choices