The Global Centre for Risk and Innovation (GCRI)
A model result should be reproducible: others should be able to rerun the model with the same code, data and settings and get the same result, and inspect every step. Many published models cannot be rerun because code, data or environments are missing. This page sets out how reproducible modelling is practised, how it is checked, and what evidence serves the modellers, publishers and institutions who depend on it.
GCRI · In development · service level R0
Reproducible modelling is the practice of making model results rerunnable and inspectable by others, using the same code, data and settings. It depends on archived code and inputs with identifiers, recorded software environments, workflows that record every step and versioning that lets corrections be traced.
For reproducible modelling, in GCRI's spatial-intelligence reading, the map opens on true-colour imagery, land cover and night lights with relief in 3D, framed on North America; every other NASA layer, live feed and forecast waits in the layer list.
The interactive map needs JavaScript; the layer list and sources are below.
Layers 3
Hazards and exposure, for this page
Keyboard, with the map focused
Hold Ctrl or ⌘ and scroll to zoom
Latest available
NASA GIBS / EOSDIS · ASTER GDEM (METI, NASA) · Blue Marble (NASA) · © OpenStreetMap contributors (ODbL) · Natural Earth
Loading the latest NASA imagery.
Drag to move. Ctrl or ⌘ and scroll, pinch or the buttons to zoom; hover for the values under the pointer; reset returns to the region.
For reproducible modelling, in GCRI's spatial-intelligence reading, the map opens on true-colour imagery, land cover and night lights with relief in 3D, framed on North America; every other NASA layer, live feed and forecast waits in the layer list.Satellite and model products as NASA publishes them, each with its own date; indicators, not assessments of any place; claim ceiling INTEGRATED.
Layers and dates
Versioned models, data lineage and re-runnable results. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Without the exact code and inputs, a result cannot be checked. Archiving them with identifiers makes reruns possible.
Software dependencies change over time. Recording environments with containers or lock files keeps models runnable for longer.
Workflow tools record every step from raw data to final figures, removing undocumented manual steps from the chain.
When errors are found, versioned models and data let corrections be traced and affected results updated.
Each term as this page uses it. Where a standard-setting body defines the term, the wording follows that body.
| Term | Definition |
|---|---|
| Computational reproducibility | Obtaining the same results by rerunning the same code on the same data in the same environment. |
| Lock file | A file recording the exact versions of software dependencies used. |
| Research software | Software created for or used in research, such as models and analysis code. |
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Models with archived code | percent of published models | per review | Code archive checks | OBSERVED |
| Successful independent reruns | percent of attempted reruns | per audit | Reproduction studies | OBSERVED |
| Models with recorded environments | percent | per review | Repository checks | OBSERVED |
| Software with persistent identifiers | count | annual | Archive records | OBSERVED |
| Corrections issued | count with reason | continuous | Correction logs | OBSERVED |
| Models meeting FAIR4RS criteria | percent | per assessment | FAIR assessment | ESTIMATED |
The topic as a computational graph: named sources feed the methods, an evidence review gates the outputs, and the decisions stay with the authorities that hold them.
Fig. 02Nodes carry their type, two metric rows and a state; connectors show what feeds what; dashed frames name the owners; the timeline is the cadence. Design of the pipeline, not a record of a run.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Source | What it gives | Licence class |
|---|---|---|
| Software Heritage archive | Archived source code with identifiers | Open, archive terms |
| Zenodo and institutional repositories | Deposited models and data with DOIs | Open, licence per record |
| FAIR principles for research software (FAIR4RS) | Principles for reusable research software | Open, CC BY 4.0 |
| CoMSES Net and model archives | Archived models with documentation | Open, licence per model |
| Reproducibility studies in computational science | Measured reproduction rates | Licence per publication |
Systems Science and Modelling prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Who decides | The question | Evidence this page serves |
|---|---|---|
| Modelling teams | How to package models so others can rerun them | Checklists; archiving guides |
| Publishers and report editors | What reproducibility evidence to require | Rerun audits |
| Funders | Whether to fund archiving and research software engineering | Reproduction rate evidence |
Models change constantly as code is fixed and improved. Without version control, it may be impossible to know which version produced a published result. Input data are revised, cleaned and transformed through steps that are often undocumented. Software libraries change behaviour between versions, and stochastic models depend on random seeds.
Coupled models multiply these problems, since each component has its own versions, data and settings. A result may depend on a particular combination that is never recorded.
Incentives play a role: documenting and sharing models takes time that is rarely rewarded. But irreproducible results cannot be verified, and errors in widely used models can spread into many decisions.
Recording every change to code and configuration. It requires discipline and shared conventions.
Documenting sources and processing steps for every input, using statistical and data stewardship practice. Some data cannot be shared.
Capturing software dependencies so models can be re-run. Long-term preservation remains difficult.
Having another team reproduce key results before publication. Costly but highly effective.
Reproducibility does not make a model right; it makes it checkable. Replication with independent models and data is a further step, covered in model intercomparison.
The current service level is R0: information and enquiries. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.