Skip to content

GCRI · Technology

Data engineering, interoperability and stewardship

Data engineering, interoperability and stewardship cover the pipelines, standards and governance that make data usable and trustworthy. They determine whether information from different sectors can be combined for joint risk assessment. Data and Compute stewards this page for the Nexus Consortium.

Make an enquiry Back to Technology
01Register

Data engineering, interoperability and stewardship on the record.

The Global Centre for Risk and Innovation
  1. 01InstitutionThe Global Centre for Risk and Innovation (GCRI)
  2. 02The methodWhat data engineering, interoperability and stewardship covers
  3. 03Reference practiceData engineering follows stewardship, provenance and catalogue standards
  4. 04In systemic risk workThe pipeline is part of the evidence
  5. 05Applied inPlatforms that apply this method
  6. 06StewardshipWho maintains this page
  7. 07SourcesWhere this page’s references lead
02The method

What data engineering, interoperability and stewardship covers.

  1. 01

    Core methods

    data pipeline design; schema and metadata standards; data quality assessment; provenance tracking; master data management; access control and licensing.

  2. 02

    Data and instruments

    data catalogues; application programming interfaces; metadata registries; data lakes and warehouses; open data portals.

  3. 03

    Validation

    Pipelines are tested with automated quality checks, schema validation and lineage audits. Reconciliation against source systems confirms completeness.

  4. 04

    Limits

    Incompatible standards, missing metadata and restrictive licences block integration. Legacy systems and undocumented transformations introduce silent errors.

  5. 05

    Use in risk decisions

    Sound data engineering allows hazard, exposure and financial data to be linked consistently. It supports transparent and repeatable analysis of interconnected risk.

03Reference practice

Data engineering follows stewardship, provenance and catalogue standards.

ReferenceWhat it sets
FAIR principlesFindable, accessible, interoperable, reusable data
W3C PROVProvenance model
W3C DCATData catalogue vocabulary
GSBPMStatistical production process model
04Process

How the process runs.

Process

Data engineering, interoperability and stewardship: Catastrophe model pipeline

Catastrophe models usually have hazard, vulnerability, exposure and financial modules, and produce loss tables, average annual loss, exceedance curves, PML and TVaR.

  1. 01 Hazard

    1. 01Event catalogueDataset · Model developerThe hazard module holds a large catalogue of simulated events that represents a wide range of plausible scenarios.
    2. 03Hazard moduleModel · Model developerFor each event the hazard module gives the local hazard intensity that the vulnerability module reads.

  2. 02 Exposure and damage

    1. 02ExposureDataset · Programme ownerThe exposure module houses the portfolio data, such as location-specific information.
    2. 04Vulnerability moduleModel · Model developerThe vulnerability module quantifies the expected damage to the exposures from an event.

  3. 03 Financial

    1. 05Financial moduleModel · Programme ownerThe financial module calculates losses from all event scenarios. Ground-up, gross and net losses differ as shown below.
    2. 06Event loss tableDataset · Programme ownerAn event loss table lists each event with its annual occurrence probability and its loss for the portfolio.
    3. 07Year loss tableDataset · Programme ownerSome models also sample simulated years from event losses. This outline does not describe their format.

  4. 04 Metrics

    1. 08Exceedance probability curvesModel · Programme ownerThe occurrence curve gives the chance that the largest loss in a year exceeds an amount. The aggregate curve does so for the sum of losses.
    2. 09AAL, PML and TVaRModel · Programme ownerAverage annual loss sums event probability times loss. PML is one point on the occurrence curve.

  5. 05 Use of results

    1. 10Use of resultsDecision · Programme ownerThe tail metrics are used when considering a portfolio’s exposure and the level of catastrophe reinsurance an insurer requires.

Outline of an external standard or method, from its public sources. It describes what the framework says, not a record of any run. It does not show that Nexus or any partner performs, certifies or approves a step. A gate is a step of the method, never a Nexus approval.

05In systemic risk work

The pipeline is part of the evidence.

Every transformation between a source and a published figure (cleaning, joining, imputing, aggregating) is a place where errors and assumptions enter. In cross-system analysis pipelines are long and shared, so provenance records are what make a figure traceable and a correction possible.

  • Record provenance at each transformation.
  • Make imputation visible in outputs.
  • Version datasets so a figure can be regenerated.
  • Applications on Registry and Reports.
06Applied in

Platforms that apply this method.

07Stewardship

Who maintains this page.

Data and Compute stewards data engineering, interoperability and stewardship: it keeps this description current and reviews how the method is applied across the platforms.

08Sources

Where this page’s references lead.

The bodies and documents this page is read against. Status and editions change; each link goes to the issuing body.

09Questions

Questions and answers.

Why does one institution host every method page?
GCRI hosts the STEM Commons so that each method has one canonical description. Platforms link to it rather than repeat it, which keeps definitions consistent.
How do I suggest a correction to a method page?
Write to the integrity concern form with the passage and the reason.
ReportsContribute

Contribute evidence from this page.

Reports is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.

GCRI

Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.

The Global Centre for Risk and Innovation

Bring a method, a dataset or a question.

Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.

Propose a method See all contact routes
Legal

Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.

Privacy choices