GCRI · Artificial Intelligence and Autonomous Systems
Methods and uncertainty
The methods Artificial Intelligence and Autonomous Systems relies on: their equations and assumptions, the data they need, how they are validated and the limits of what they can show.
The order things happen in.
- 01Draftsources cited, labels attached
- 02Evidence checkevery value traced
- 03Independent reviewconflicts declared
- 04Editionpublished with its limits
- 05Correctiona new version, never a silent edit
The methods in use.
- 01Artificial intelligence and machine learningDistribution shift analysis · Fine-tuning for a specific use · Interpretability methodsApplied here
- 02AI evaluation and assurance engineeringAssurance cases for AI systems · Structured red teaming · Post-deployment monitoringApplied here
- 03High-performance, cloud and edge computingCompute access for public research · Energy measurement of workloadsApplied here
- 04Probability and statisticsUncertainty on benchmark scores · Calibration assessment · Audit samplingApplied here
- 05Cybersecurity and cryptographyThreat modelling AI applications · Model supply chain integrity · Injection defence testingApplied here
- 06Social and behavioural scienceHuman oversight studies · Affected community research · Workplace adoption studiesApplied here
- 07Robotics and automationScenario-based testing · Collaborative robot safety · Operating domain monitoringApplied here
Each method, its role in this field and how it is checked.
- Ai MLRole hereArtificial intelligence and machine learning is the technology this platform stewards for the network.Checks appliedHeld-out evaluation; Dataset documentation; Drift monitoring
- Ai assuranceRole hereAI evaluation and assurance engineering is the second method this platform stewards.Checks appliedIndependent review; Evidence currency; Coverage reporting
- ComputingRole hereHigh-performance and cloud computing supply the capacity AI needs.Checks appliedMeasurement boundaries; Reproducible configuration; Provider data verification
- Probability statisticsRole hereProbability and statistics make AI evaluation trustworthy: confidence intervals on scores, significance of differences between models, calibration of predicted probabilities and sampling designs for audits.Checks appliedInterval reporting; Multiple comparisons; Sample size planning
- CybersecurityRole hereCybersecurity and cryptography protect AI systems and the data they use.Checks appliedThreat model review; Provenance verification; Regression testing
- Social scienceRole hereSocial and behavioural science studies how people use, trust and are affected by AI.Checks appliedEthics review; Pre-registration; Representative sampling
- RoboticsRole hereRobotics and automation bring AI into physical systems.Checks appliedScenario coverage; Simulation validity; Standards conformity
How the process runs.
Process explorer · Process
Methods and uncertainty: PRISMA 2020 review reporting
10 steps·4 stages
Process
Methods and uncertainty: PRISMA 2020 review reporting
PRISMA 2020 is a reporting guideline for systematic reviews; the steps follow its checklist, not a mandated sequence, and no Nexus document adopts it.
01 Plan
- 01Registration and protocolRecord · The review teamThe guideline asks for registration details, access to the protocol and any amendments.
- 02Eligibility criteriaRecord · The review teamThe guideline asks for the inclusion and exclusion criteria and how studies were grouped.
02 Find and select
- 03Search: sources and strategiesStep · The review teamThe guideline asks for the information sources and the full search strategies. It does not fix which sources are used.
- 04Selection of studiesStep · The review teamThe guideline asks how studies were judged eligible, including how many reviewers screened. It does not fix the number of reviewers.
- 05Flow diagramRecordThe flow diagram shows the number of records found, the number of studies included and the reasons for exclusion.
03 Analyse
- 06Data collectionStep · The review teamThe guideline asks how data were collected from reports and which data items were sought.
- 07Risk of biasStep · The review teamThe guideline asks for the methods and tools used to assess risk of bias.
- 08SynthesisModel · The review teamThe guideline asks for the effect measures and the synthesis methods. It does not prescribe meta-analysis.
- 09Certainty of evidenceStep · The review teamThe guideline asks for the methods used to assess certainty, and reports the result per outcome. It does not name a grading scale.
04 Report
- 10Review reportRecordThe guideline helps authors report why the review was done, what they did and what they found.
ReportsOutline of an externally published reporting guideline, not a Nexus process and not a record of operation. PRISMA 2020 mainly covers reviews of intervention effects. No Nexus page names it. Every programme is at service level R0.
Where uncertainty in AI analysis comes from, and how it is shown.
| Source | Why it matters here | How it is reported |
|---|---|---|
| Benchmark validity | Benchmarks saturate and leak into training data | Several evaluations, held-out tests, dates stated |
| Capability elicitation | Measured capability depends on prompting and tools | Elicitation effort described |
| Deployment context | Risk depends on use, not only the model | Use-specific assessment |
| Incident data | Reporting is voluntary and uneven | Incident sources and coverage stated |
Evaluation results carry the model version, date and setup; AI capability changes quickly and old results expire.
How uncertainty is reported.
- 01
State assumptions
Each analysis lists its assumptions and the data it depends on.
- 02
Quantify where possible
Ranges and confidence are reported alongside central estimates.
- 03
Validate and compare
Models are checked against observations and, where possible, against other models.
- 04
Publish limits
What an analysis cannot show is stated beside what it can.
Where this page’s references lead.
The bodies and documents this page is read against. Status and editions change; each link goes to the issuing body.
Questions and answers.
Are the methods open?
Contribute evidence from this page.
Reports is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- EmailGateway GW-GCRISend it from this pageEvidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen depositDeposit it and submit to the communityPublish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsGCRISee every way to contributeThe contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.
Bring a method, a dataset or a question.
Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.
Legal documents.
Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.