GCRI · Reports
Labs and foundry series
Labs and foundry reports record technical tests: prototypes, software, sensors, data pipelines and procedures exercised under stated conditions. Test evidence is easy to overstate, because a passed test under controlled conditions is read as readiness for use. This page sets out how test reports are structured, reviewed and archived, how readiness levels are evidenced rather than declared, and what such evidence can support for those deciding whether to adopt or fund a technology.
Labs and foundry series on the record.
Labs and foundry series: how the risk develops.
Technical test and readiness evidence. The dossier below sets out how the risk develops, what is measured, where the data come from, how it is modelled and who decides.
Test conditions narrow the claim
A test establishes behaviour under the conditions tested, with the inputs, environment and version used. Reports that omit those conditions invite readers to generalise beyond them.
Readiness declared not shown
Technology readiness levels are often self-assigned. Without the evidence each level requires, a readiness claim moves from test report to procurement decision unchecked.
Negative results go missing
Failed and inconclusive tests are seldom written up, so the published record overstates success rates and others repeat tests that have already failed.
Versions change underneath
Software and hardware change after testing. A result cited without the version tested is attached to a system that may no longer exist in that form.
Series production desk.
The screen a team would use to follow this subject: its measures, states and decisions. Names are examples and values are illustrative.
Series production desk
Labs and foundry series · Annual synthesis · Edition 7
Editions in production
| Annual synthesis | Global | Data check | 81 | W10 |
| Country profile 7 | Country | Peer review | 97 | W16 |
| Water outlook | Sectoral | Drafting | 32 | W38 |
| Methods note 4 | Methods | Released | 100 | done |
Data check, annual synthesis
Edition pipeline
- Outlineboard
- Draftauthors
- Data checkevery value
- Peer reviewtwo rounds
- Editorialdecision
- ReleaseDOI
- Correctionsversioned
Labs and Foundry editions report what was tested and built for each domain.
Editions carry verification evidence under NS-11 of the proposed Nexus Standards. A test result is not an approval.
- Negative results.
- Replications.
How labs and foundry series is modelled, measured and tested.
The model, its assumptions, its data and its uncertainty, in the form a scientist or engineer would check. Structures follow published methods. The values drawn are illustrative, not findings.
Model, equations and assumptions
The model structure used for labs and foundry series, the assumptions it makes and how each is tested.
Ri = f(Hi, Ei, Vi, Ci)hazard, exposure, vulnerability and coping capacityEAL = ∑k pk · Lkover events k with annual probability pkxt+1 = A xt + B ut + εtA carries the couplings, u the shock, ε the error| Assumption | Why it is made | How it is tested |
|---|---|---|
| Hazard and exposure are independent within an event | Keeps the loss integral tractable | Compare with a copula fit on joint records |
| Couplings are linear near the operating point | Lets shocks propagate through A | Perturb beyond the linear range and compare |
| Vulnerability curves transfer between places | Local curves are rarely available | Validate against observed losses where they exist |
- Inputs
- versioned datasets with provenance
- Outputs
- distributions, never single numbers
- Code
- released with the model card and tests
- Reproduction
- same data, code, environment and seed
What is measured, and how well.
Each indicator carries its unit, its cadence, the kind of source behind it and the evidence label a value would carry. No values are shown: a programme supplies them under its own authority.
| Indicator | Unit | Cadence | Source type | Label |
|---|---|---|---|---|
| Test protocols registered before testing | percent of reports | per report | Registry records | OBSERVED |
| Test reports with archived code, data and configuration | percent of reports | per report | Repository deposit records | OBSERVED |
| Readiness level claimed and evidence attached | level claimed; yes or no for evidence | per report | Report and evidence annex | DECLARED |
| Negative and inconclusive results published | count per period | annual | Series catalogue | REPORTED |
| Version of the tested system identified | percent of reports | per report | Report metadata | OBSERVED |
| Independent technical review completed | yes or no, with reviewer role | per report | Review record | DECLARED |
Where the evidence comes from.
Named public datasets and institutional records, each with its licence class. Sources by agreement are used only under their terms and never republished.
| Source | What it gives | Licence class |
|---|---|---|
| Zenodo | Archived test data, configurations and software releases with DOIs | Metadata CC0, licence per record |
| Software Heritage | Preserved source code at the exact revision tested | Code under its own licence |
| OSF Registries | Pre-registered test protocols and analysis plans | Metadata CC0, licence per registration |
| protocols.io | Published step-by-step methods with versions and DOIs | Licence per protocol, mostly CC BY |
| NIST National Vulnerability Database | Known vulnerabilities for software components under test | Public domain |
Who decides, and what the evidence serves.
Reports prepares evidence. The decisions below are taken by the authorities named, under their own rules, and never by the platform.
| Who decides | The question | Evidence this page serves |
|---|---|---|
| Procurement and adoption boards | Whether a tested technology is ready for a pilot or operational use | Readiness evidence annexes, test conditions and known limitations |
| Innovation funders and programme managers | Whether to fund the next stage of development | Readiness level with its evidence, negative results and open risks |
| Technical review panels | Whether a test report is complete and its conclusions follow from the evidence | Registered protocols, archived data and version identifiers |
| Standards and conformity assessment bodies | Whether test methods are suitable for a future standard | Documented test procedures and their reproducibility |
Outputs this topic produces
- Test reports stating conditions, versions and acceptance criteria
- Readiness evidence annexes mapping each level to its evidence
- Archived configurations and data for each test
- A register of negative and inconclusive results
What the series asks.
Technical claims about tools and systems need evidence that others can inspect. A system that is said to work under stress should show the test that proved it, the conditions of the test and what failed. Publishing that evidence, including negative results, guards against overstated readiness and helps institutions judge what a tool can and cannot do.
- What exactly was tested, under which protocol, with which data and in which environment?
- What passed, what failed and what was not tested?
- Does the evidence support the stated next step, and only that step?
- Which observations or signals warrant further analysis, and how reliable are they?
What the series covers.
Three series report evidence from building, testing and observing.
Experiments and testbeds
Results from completed protocols, including null and negative results.
Build and readiness
Evidence about Foundry systems against stated readiness criteria, never as certification.
Observation
Signals from monitoring, labelled by source and confidence, prompting analysis rather than action.
The series follows the logic of test and evaluation practice in engineering: a claim is only as good as the protocol that tested it, and readiness is always readiness for something specific. A tool that has passed a laboratory test is not ready for field use, and a system that worked in one exercise has not shown that it will work in every setting. Editions state these limits next to each result.
How editions are built, and their limits.
- 01
Protocols
Each result cites a protocol registered before the test, so outcomes cannot be selected after the fact.
- 02
Environments
Test conditions are described; results from a testbed may not transfer to live operation.
- 03
Readiness criteria
Readiness is assessed against criteria for a stated next step. It is not approval, bankability or certification.
- 04
Signal quality
Observatory signals state their source, coverage and false-alarm characteristics where known.
How it is studied, and what the methods cannot say.
- 01Data engineering, interoperability and stewardshipStructured publishing in JATS XML · Persistent identifiers across the recordApplied here
- 02Uncertainty quantification and model validationCalibrated language in assessments · Certainty of evidence ratingsApplied here
- 03Probability and statisticsMeta-analysis with heterogeneity stated · Small-study and publication biasApplied here
Where this page’s references lead.
The bodies and documents this page is read against. Status and editions change; each link goes to the issuing body.
Questions and answers.
What is a technology readiness level?
Why publish failed tests?
Does a passed test mean a system is safe?
Does a readiness report mean a system is approved?
Are failed tests published?
Can vendors publish results in the series?
Are observatory signals warnings?
Contribute evidence from this page.
Reports is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
- EmailGateway GW-GCRISend it from this pageEvidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen depositDeposit it and submit to the communityPublish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsGCRISee every way to contributeThe contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.
Bring a method, a dataset or a question.
Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.
Legal documents.
Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.