The Global Centre for Risk and Innovation (GCRI)
A result can be trusted when others can rerun the same analysis on the same data and get the same answer, and when they can trace each number to its source. Reproducible pipelines combine versioned data and code, recorded environments, workflow systems and provenance. This page sets out why analyses fail to reproduce, how reproducibility is engineered and checked, and what evidence serves the researchers, publishers and agencies that depend on it.
Reproducible pipelines · Public data platform · Catalogue v9
| Flood extents | Open | Complete | Yes | Published |
| Health facility list | Open | Complete | Yes | Published |
| Mobility traces | Restricted | Partial | n/a | Rights review |
| Household survey | Restricted | Complete | Yes | Access by request |
Fig.Data platform and pipelines. Design wireframe with placeholder names and illustrative values, not a record or a live system.
Reproducible data pipelines are analysis workflows built so that others can rerun them on the same data and obtain the same results. They combine versioned data and code, recorded environments, automated workflows and provenance, which also lets errors be traced and corrections reach every affected output.
For reproducible pipelines, in GCRI's spatial-intelligence reading, the map opens on night lights, true-colour imagery and population density with true colour from NASA GIBS, framed on the contiguous United States; every other NASA layer, live feed and forecast waits in the layer list.
The interactive map needs JavaScript; the layer list and sources are below.
Layers 3
Connected activity, for this page
Keyboard, with the map focused
Hold Ctrl or ⌘ and scroll to zoom
Latest available
NASA GIBS / EOSDIS · ASTER GDEM (METI, NASA) · Blue Marble (NASA) · © OpenStreetMap contributors (ODbL) · Natural Earth
Loading the latest NASA imagery.
Drag to move. Ctrl or ⌘ and scroll, pinch or the buttons to zoom; hover for the values under the pointer; reset returns to the region.
For reproducible pipelines, in GCRI's spatial-intelligence reading, the map opens on night lights, true-colour imagery and population density with true colour from NASA GIBS, framed on the contiguous United States; every other NASA layer, live feed and forecast waits in the layer list.Satellite and model products as NASA publishes them, each with its own date; indicators, not assessments of any place; claim ceiling INTEGRATED.
Layers and dates
Pipelines, models and experiment provenance. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Software libraries, operating systems and hardware change. An analysis that ran last year may give different results, or none, without a recorded environment.
Datasets are corrected and updated. Unless the exact version used is recorded, results cannot be traced or rerun.
Steps done by hand in spreadsheets or interactive sessions are not recorded and cannot be repeated exactly.
Recording which inputs, code and parameters produced each output lets errors be traced and corrections propagate.
Each term as this page uses it. Where a standard-setting body defines the term, the wording follows that body.
| Term | Definition |
|---|---|
| Workflow system | Software that defines and runs analysis steps in order and records how outputs were produced. |
| Container | A packaged software environment, with its libraries and settings, that runs consistently across computers. |
| DOI | A digital object identifier that persistently identifies publications, datasets or software. |
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Published analyses with code and data available | percent of outputs | per audit | Publication audits | OBSERVED |
| Analyses rerun successfully by a third party | percent of attempted reruns | per audit | Reproduction studies | OBSERVED |
| Datasets with persistent identifiers | percent of datasets | annual | Repository metadata | OBSERVED |
| Pipelines with recorded environments | percent of production pipelines | per assessment | Pipeline repositories | DECLARED |
| Corrections issued after reproduction checks | count | annual | Correction logs | OBSERVED |
| Software archived with identifiers | count | annual | Software archive records | OBSERVED |
The topic as a computational graph: named sources feed the methods, an evidence review gates the outputs, and the decisions stay with the authorities that hold them.
Fig. 02Nodes carry their type, two metric rows and a state; connectors show what feeds what; dashed frames name the owners; the timeline is the cadence. Design of the pipeline, not a record of a run.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Source | What it gives | Licence class |
|---|---|---|
| DataCite and Crossref metadata | Persistent identifiers for data, software and publications | Open, CC0 metadata |
| Software Heritage archive | Archived source code with intrinsic identifiers | Open, archive terms |
| Zenodo and institutional repositories | Deposited data and code with identifiers | Open, licence per record |
| W3C PROV standard | A model for representing provenance | Open standard, W3C terms |
| Reproducibility studies in published literature | Measured reproduction rates in fields | Licence per publication |
Data and Compute prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Who decides | The question | Evidence this page serves |
|---|---|---|
| Research teams and analysts | How to structure pipelines so results can be rerun | Checklists; pipeline templates |
| Publishers and report editors | What reproducibility evidence to require | Audit methods |
| Funders and agencies | Whether to fund archiving and reproducibility support | Reproduction rate evidence |
Irreproducibility is rarely deliberate. It accumulates through small, reasonable shortcuts made under deadline pressure, each harmless alone. The cost appears later, when a result is questioned, a correction is needed or a team member leaves.
Libraries and dependencies change, so the same code produces different results or stops running.
Datasets are updated in place without versioning, so the original input cannot be recovered.
Undocumented manual cleaning or parameter choices cannot be repeated by others.
Tracking code, configuration and, where possible, data versions. Limits: large datasets need specialized tools.
Packaging software dependencies so they can be rebuilt. Limits: containers themselves depend on underlying systems that change.
Tools that run pipelines and record each step, studied under data engineering, interoperability and stewardship. Limits: adoption requires effort and discipline.
On parallel and accelerated hardware, floating-point results can differ slightly between runs, a matter for computing specialists. Limits: exact bitwise reproducibility is sometimes impractical, so tolerances must be stated.
The current service level is R0: information and enquiries. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.