Skip to content

GCRI · Research library

Methods and technical papers

Papers describing the quantitative and qualitative methods of The Global Centre for Risk and Innovation (GCRI), open enough for another team to apply, test or criticise them. Each states the method, its assumptions, its validation, where it fails and the software or data it needs. Methods link to their STEM Commons pages. Part of the research library.

Make an enquiry Back to Research library
01Workspace

The workspace for methods and technical papers.

Evidence grading

Methods and technical papers · Review wave 2 · Example interface, not a record

Two sources downgradedGCRI · GCRI

Certainty by criterion

0 low to 100 high

Higher is more certain. Values are examples.

Review queue

  1. R-1Country programmes activesecond reviewer3 d
  2. R-2Evidence records with labelssecond reviewer2 d
  3. R-3Methods appliedsecond reviewer4 d
Evidence grading.Illustrative interface; not live data.

GCRI · Evidence grading. Evidence grading: example names and illustrative values, not a record.

02Two traditions

Quantitative and qualitative, held to one standard.

Both kinds of method must be described well enough to be checked.

Quantitative

Models and statistics

Statistical, network, simulation and optimisation methods, such as uncertainty quantification.

Qualitative

Structured judgement

Interviews, expert elicitation, scenario workshops and case analysis, with policy and decision science.

Mixed

Combined designs

Studies that combine both, stating how evidence of each kind is weighed.

03Reporting guides

The reporting guides a methods paper follows, by kind of method.

Kind of methodGuideReference
Prediction models, including machine learningTRIPOD+AICollins and others, BMJ, 2024
Agent-based and other simulation modelsODD protocol, second updateGrimm and others, 2020
Computer model calibrationCalibration with explicit model discrepancyKennedy and O’Hagan, 2001
Machine-learning models in deploymentModel cardsMitchell and others, 2019
Land change from remote sensingArea estimation from a reference sampleOlofsson and others, 2014
Structured expert judgementDocumented elicitation protocol with calibration questionsStated in each paper

Reviewed methods are designed for the Methods and Technical series, with each model described under NS-12 of the proposed Nexus Standards. Methods researchers can contribute replications and documented failures; both are published.

04Process

How the process runs.

Process

Methods and technical papers: Source to evidence record

A source is registered and assessed, its record labelled and classified, and a Digital Evidence Passport carries provenance; a correction reaches every dependent record.

  1. 01 Intake

    1. 01Identify the sourceSource · The source stewardDiscovery is not ingestion. Source identification records the source class, steward, institutional role and jurisdiction.
    2. 03Register the sourceStepRegistration makes a discovered source a governed participant in the evidence lifecycle. It implies no endorsement or certification.

  2. 02 Assessment

    1. 04Jurisdiction and rights classificationGateRights classification identifies licence, consent, confidentiality and reuse limits. The result determines access and routing.
    2. 05Evidence quality assessmentStepQuality assessment decides whether a record is fit for a defined use. Quality is multidimensional and never a single score.
    3. 06Evidence labelStepEvery finding carries one of nine evidence labels. Missing data is shown as missing, never as zero.

  3. 03 Record

    1. 07Evidence classificationRecordClassification gives records lifecycle states, such as source-verified, public-safe or restricted. States can coexist and set evidence use.
    2. 08Source release and identifierRecord · NXS-GRX (canonical data, provenance and risk)A source release records rights, temporal coverage and source lineage. A UNI is allocated through a catalogue and never typed by hand.
    3. 09Digital Evidence PassportRecordThe passport records provenance, quality, rights, jurisdiction, permitted use and history, so evidence keeps its context when it travels.

  4. 04 Use and correction

    1. 02Correction reaches dependent recordsStep · NXS-GRX (canonical data, provenance and risk)A source correction propagates to dependent cuts and products without rewriting old results.
    2. 10Public-safe reviewGatePublic-safe review decides whether a record can be shared publicly. It is a distinct stage and is not validation.

Data protocols: evidence lifecycle · Data protocols: core data objects and evidence records · The Global Risks Index (NXS-GRX)Designed programme process, not a record of operation. No cycle has run and every programme is at service level R0. A gate is a programme decision, never a public-authority approval, certification or deployment authorisation. The data-protocol pages describe a design, not a running system.

05What a methods paper contains

Six elements in every paper.

  1. 01

    Purpose

    The question the method answers and where it should not be used.

  2. 02

    Assumptions

    What must be true for results to hold.

  3. 03

    Validation

    How the method was tested, on which data and with what result.

  4. 04

    Failure modes

    Known situations where the method gives misleading results.

  5. 05

    Implementation

    Software, versions and parameters, released where licences allow.

  6. 06

    Review

    Who reviewed the paper and its review state.

06Why openness matters

A method no one can check is not evidence.

Risk decisions often rest on models and judgement that are hard to see. Open methods let independent researchers, source agencies and affected communities test them, find weaknesses and propose improvements. Where licensing or security prevents releasing code or data, the paper says so and describes the method in enough detail to assess.

08Sources

Where this page’s references lead.

The bodies and documents this page is read against. Status and editions change; each link goes to the issuing body.

09Questions

Questions and answers.

Why publish methods separately from results?
So that the method can be reviewed and reused on its own, and so that results in many publications can point to one checked description.
Are qualitative methods treated as less rigorous?
No. They answer different questions. Both must state their design, assumptions and limits, and both are reviewed.
Can I reuse a GCRI method?
Yes, under the licence of the paper and any released software. Please cite the method paper and version.
How are methods improved over time?
Through review, challenge and new evidence. A revised method is published as a new version, and results that depend on it note the change.
Can I propose an improvement to a method?
Yes. Write to the research collaboration form with the method, the proposed change and your evidence. Proposals receive a reference and are reviewed on their merits.
ReportsContribute

Contribute evidence from this page.

Reports is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.

GCRI

Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.

The Global Centre for Risk and Innovation

Bring a method, a dataset or a question.

Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.

Propose a method See all contact routes
Legal

Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.

Privacy choices