GCRI · Systemic Risk Management
Quantitative and qualitative methods
Risk assessment uses numbers where data support them and structured judgement where they do not. Probabilistic models, loss distributions, expert elicitation, bow-tie diagrams and risk matrices each answer different questions and fail in different ways. This page sets out how the main methods work, when each is fit for purpose, and how combining them keeps an assessment honest about what is known and what is assumed.
Quantitative and qualitative methods on the record.
Quantitative and qualitative methods: how the risk develops.
Probability, uncertainty and judgement. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Models need data that fit
Frequency and severity models work where events are numerous and recorded consistently. For rare, novel or changing risks the data are thin, and a precise-looking number can carry more assumption than evidence.
Judgement carries biases
Experts anchor on recent events, overestimate their own precision and converge in groups. Structured elicitation reduces these effects by separating estimates, asking for ranges and scoring calibration.
Matrices compress information
Likelihood and consequence matrices are quick to use, but they rank risks coarsely, can reverse orderings and hide correlated losses. They suit screening, and need other methods before a major decision.
Methods combine through structure
Fault trees, event trees and bow-tie diagrams link causes, controls and consequences. They provide a frame into which quantitative estimates and expert judgement can be placed and checked.
Cascade and stress lab.
The screen a team would use to follow this subject: its measures, states and decisions. Names are examples and values are illustrative.
Cascade and stress lab
Quantitative and qualitative methods · National system · Scenario SC-40
Interdependency graph
red: failed in scenario
Cascade sequence
- T+0HAZHeatwave and drought
- T+6hENGThermal plant derated, cooling water
- T+9hICTData centre on backup power
- T+14hWATPumping reduced, pressure loss
- T+20hHLTHospital water supply at risk
Service loss by system
Methods must be fit for risks that cross domains.
- WaterCouplingHydrological risk is mostly quantified with simulation and extreme value statistics.What to watchUncertainty ranges reported with return periods
- HealthCouplingEpidemic risk combines transmission models with expert judgement early in outbreaks.What to watchStructured elicitation records
- BiodiversityCouplingEcosystem risks often have qualitative evidence only, which matrices flatten.What to watchQualitative evidence kept as such
Method comparisons are designed for the Methods and Technical series. Cox showed in 2008 that risk matrices can rank risks inconsistently; the page sets out when quantitative, semi-quantitative and qualitative methods each fit.
- Comparisons of assessment techniques applied to the same risk.
- Cases where a risk matrix misled a decision.
How quantitative and qualitative methods is modelled, measured and tested.
The model, its assumptions, its data and its uncertainty, in the form a scientist or engineer would check. Structures follow published methods. The values drawn are illustrative, not findings.
Model, equations and assumptions
The model structure used for quantitative and qualitative methods, the assumptions it makes and how each is tested.
Ri = f(Hi, Ei, Vi, Ci)hazard, exposure, vulnerability and coping capacityEAL = ∑k pk · Lkover events k with annual probability pkxt+1 = A xt + B ut + εtA carries the couplings, u the shock, ε the error| Assumption | Why it is made | How it is tested |
|---|---|---|
| Hazard and exposure are independent within an event | Keeps the loss integral tractable | Compare with a copula fit on joint records |
| Couplings are linear near the operating point | Lets shocks propagate through A | Perturb beyond the linear range and compare |
| Vulnerability curves transfer between places | Local curves are rarely available | Validate against observed losses where they exist |
- Inputs
- versioned datasets with provenance
- Outputs
- distributions, never single numbers
- Code
- released with the model card and tests
- Reproduction
- same data, code, environment and seed
What is measured, and how well.
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Risks assessed with quantitative estimates | percent of principal risks | annual | Assessment records | REPORTED |
| Calibration of expert estimates | calibration score on seed questions | per elicitation | Structured expert judgement records | OBSERVED |
| Loss event records | count of events with cause and amount recorded | continuous | Internal loss databases | OBSERVED |
| Assessment uncertainty range | ratio of upper to lower bound | per assessment | Assessment outputs | ESTIMATED |
| Controls with tested effectiveness | percent of key controls | annual | Control testing records | REPORTED |
| Method documentation completeness | percent of assessments with method, data and assumptions recorded | annual | Assessment files | REPORTED |
Where the evidence comes from.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Source | What it gives | Licence class |
|---|---|---|
| IEC 31010 Risk assessment techniques | Descriptions, strengths and limits of more than forty techniques | Licensed standard, purchase required |
| EM-DAT international disaster database (CRED, UCLouvain) | Disaster events with human and economic impacts | Free for non-commercial use with registration |
| Sendai Framework Monitor and national disaster loss databases | Published reporting of disaster losses against Sendai targets | Public, UN terms of use |
| TU Delft expert judgement database | Past structured expert judgement studies with calibration data | Licence to check |
| Published operational loss data consortia summaries | Aggregate operational loss patterns in financial services | Members only; public summaries (to check) |
Who decides, and what the evidence serves.
Systemic Risk Management prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Who decides | The question | Evidence this page serves |
|---|---|---|
| Risk functions and analysts | Which method fits a given risk and decision | Method selection guides; documentation standards |
| Boards and executives | How much confidence to place in an assessment before acting on it | Uncertainty ranges; method documentation; calibration records |
| Regulators and supervisors | Whether an institution’s risk models are fit for the use made of them | Model documentation; validation and back-testing evidence |
Outputs this topic produces
- Method selection guides keyed to the question and the evidence available
- Structured expert elicitation protocols with calibration
- Bow-tie and fault-tree templates linked to registers
- Assessment files that record data, assumptions and ranges
What methods experts ask.
- Is there enough relevant data to estimate probabilities, or would numbers convey a false sense of precision?
- What kind of uncertainty dominates: randomness, lack of knowledge, or disagreement about how the system works?
- How sensitive is the decision to the parts of the analysis that are least certain?
- How can expert judgement be elicited so that it is transparent and less prone to bias?
Kinds of uncertainty.
Randomness
Natural variability that persists even with perfect knowledge, such as weather from year to year. Probability handles it well.
Lack of knowledge
Uncertainty that more data or research could reduce. It should be shown, not hidden in a single number.
Disputed structure
Where experts disagree on models, probabilities or values. Scenarios and robust decision methods suit it better.
A toolkit, matched to the problem.
- 01
Statistical and probabilistic models
Frequency analysis, extreme value methods and Bayesian inference from probability and statistics. Limits: rare events are poorly sampled, and past data may not reflect a changing system.
- 02
Uncertainty quantification
Uncertainty quantification and model validation propagates input uncertainty through models and tests them against data. Limits: it cannot capture what the model leaves out.
- 03
Structured expert judgement
Formal elicitation protocols that calibrate and combine expert views. Limits: time-consuming, and experts share blind spots.
- 04
Qualitative assessment
Scales, matrices and narrative analysis. Useful for communication and screening, but risk matrices can rank risks inconsistently and should not be the only tool.
How it is studied, and what the methods cannot say.
- 01Probability and statisticsFrequency and severity modelling · Dependence between risks · Combining data and judgementApplied here
- 02Uncertainty quantification and model validationPropagating input uncertainty · Separating kinds of uncertainty · Deep uncertainty framingApplied here
- 03Decision analysis under uncertaintyMulti-criteria treatment choice · Decision trees for contingent plansApplied here
- 04Scenario design and stress testingCross-sector scenario design · Reverse stress testing · Second-round effectsApplied here
Where this page’s references lead.
The bodies and documents this page is read against. Status and editions change; each link goes to the issuing body.
Questions and answers.
Are risk matrices reliable?
What is structured expert judgement?
When should risk be quantified?
Are quantitative methods always better?
What is wrong with risk matrices?
How should uncertainty be communicated?
Contribute evidence from this page.
Reports is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- EmailGateway GW-GCRISend it from this pageEvidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen depositDeposit it and submit to the communityPublish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsGCRISee every way to contributeThe contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.
Bring a method, a dataset or a question.
Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.
Legal documents.
Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.