The Global Centre for Risk and Innovation (GCRI)
Science corrects itself only when failures are recorded: replications that did not reproduce, experiments that found nothing, errors found after publication. Many of these records never appear, so the published literature overstates effects and certainty. This page sets out how irreproducibility arises, how replication and correction work, which records exist, and what editors, integrity offices and funders rely on when they correct or retract.
GCRI · Established · service level R0
Negative results are findings of no effect or of a failed replication, and science corrects itself only when they are recorded along with errors found after publication. Many never appear, so the published literature overstates effects and certainty. Replication databases, retraction records and repositories for null results help restore the balance.
For reproducibility and negative results, in GCRI's spatial-intelligence reading, the map opens on true-colour imagery, land cover and night lights, framed on World; every other NASA layer, live feed and forecast waits in the layer list.
The interactive map needs JavaScript; the layer list and sources are below.
Layers 3
Hazards and exposure, for this page
Keyboard, with the map focused
Hold Ctrl or ⌘ and scroll to zoom
Latest available
NASA GIBS / EOSDIS · ASTER GDEM (METI, NASA) · Blue Marble (NASA) · © OpenStreetMap contributors (ODbL) · Natural Earth
Loading the latest NASA imagery.
Drag to move. Ctrl or ⌘ and scroll, pinch or the buttons to zoom; hover for the values under the pointer; reset returns to the region.
For reproducibility and negative results, in GCRI's spatial-intelligence reading, the map opens on true-colour imagery, land cover and night lights, framed on World; every other NASA layer, live feed and forecast waits in the layer list.Satellite and model products as NASA publishes them, each with its own date; indicators, not assessments of any place; claim ceiling INTEGRATED.
Layers and dates
The working screen behind this subject: its structure, terms and states. Names are placeholders and values illustrative.
Reproducibility and negative results · Labs · Quarter 1
| PR-594 sensor drift | Field experiment | Yes | Approved | Running |
| PR-714 AI triage | Randomised | Yes | Conditions | Setup |
| PR-551 message recall | Survey experiment | Yes | Approved | Complete |
| PR-896 heat exposure | Cohort | No | Pending | Draft |
Failure records and corrections. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Significant, novel results are more likely to be written up, accepted and cited than null ones. The literature then shows a biased sample of what was studied, and meta-analyses inherit the bias.
A flawed or retracted paper keeps being cited, often without mention of its status. Reviews, guidelines and later studies that build on it carry the error forward, sometimes long after the notice.
Missing code, undocumented software versions, unshared reagents, unclear methods and lost raw data stop others from repeating an analysis or experiment, even when the original work was honest.
Investigations take time, journals differ in how they flag problems, and retraction notices can be vague. Readers may not learn that a paper was corrected, and databases may not carry the notice forward.
Each term as this page uses it. Where a standard-setting body defines the term, the wording follows that body.
| Term | Definition |
|---|---|
| Publication bias | The tendency for studies with positive or significant results to be published more often than others. |
| Computational reproducibility | Re-running the original data and code and obtaining the reported results. |
| Post-retraction citation | Continued citation of a retracted paper as if it were valid. |
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Computational reproducibility | percent of sampled studies regenerated from shared code and data | per audit | Reproducibility audits | OBSERVED |
| Replication outcome | proportion of replications meeting the prespecified criterion | per programme | Coordinated replication projects | OBSERVED |
| Retraction notices | count by field and reason | annual | Retraction Watch database via Crossref | OFFICIAL |
| Time from concern to correction | months | per case | Journal and integrity office records | REPORTED |
| Post-retraction citations | citations per retracted article | annual | Citation data linked to retraction records | OBSERVED |
| Null and negative results deposited | count of records | annual | Repositories and registries | REPORTED |
| Data available on request | percent of articles whose data could be obtained | per audit | Data request audits | OBSERVED |
The topic as a computational graph: named sources feed the methods, an evidence review gates the outputs, and the decisions stay with the authorities that hold them.
Fig. 02Nodes carry their type, two metric rows and a state; connectors show what feeds what; dashed frames name the owners; the timeline is the cadence. Design of the pipeline, not a record of a run.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Source | What it gives | Licence class |
|---|---|---|
| Retraction Watch database (Crossref) | Retractions, corrections and expressions of concern with reasons | Open, attribution |
| Crossref metadata and Crossmark updates | Links from articles to their corrections and retractions | Open metadata, no restrictions |
| OpenAlex | Works, citations, authors and institutions across disciplines | Open, CC0 |
| FORRT Replication Database | Replication attempts and outcomes, mostly in the social and behavioural sciences | Open, attribution |
| OSF projects of coordinated replication programmes | Protocols, data and code of large-scale replications | Open, licence per project |
| PubPeer | Post-publication comments identifying possible problems in papers | Free access, terms of use |
| Zenodo (CERN) | Citable deposits for null results, data and code | Open, licence per record |
Nexus Labs prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Who decides | The question | Evidence this page serves |
|---|---|---|
| Journal editors and publishers | Whether to correct, retract or issue an expression of concern | Evidence of error or misconduct, author response, COPE guidance, reproduction attempts |
| Research integrity offices | Whether an allegation is substantiated and what follows | Raw data, records, code, interviews and an independent analysis |
| Research funders | Whether to fund replications and require data and code sharing | Reproducibility audit results, replication evidence by field, cost of sharing |
| Guideline developers and systematic reviewers | Whether to rely on a study that has been questioned or corrected | Retraction and correction status, replication evidence, risk-of-bias assessment |
Many published findings in several disciplines have proved difficult to replicate. The causes are well understood: small samples, flexible analysis, selective reporting, undocumented data processing and incentives that favour novel positive findings. In computational work, results can also depend on software versions, hardware, random seeds and data that change after publication.
Negative results are under-reported because they are harder to publish and seem less interesting. This distorts the evidence base: if only successful trials of an intervention are visible, the intervention looks more effective than it is. Systematic reviews and models built on that evidence inherit the bias.
Corrections matter for the same reason. An error in a widely used dataset or model can propagate into many downstream analyses before it is found.
Sharing the materials needed to re-run an analysis, within privacy and licensing limits. Some data cannot be shared and must be described instead.
Recording software versions and environments, a practice developed with reproducible modelling. Long-term preservation remains hard.
Peer review of the protocol before results exist, so publication does not depend on the outcome.
Dated, visible corrections linked to the original result, rather than silent replacement.
A failed replication does not by itself show the original was wrong; differences in setting, sample or method must be examined before conclusions are drawn.
The current service level is R0: information and enquiries. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.