GCRI · Innovation and technological change
Technology assessment
Claims about technology are easy to make and hard to check. Technology assessment at The Global Centre for Risk and Innovation (GCRI) asks what a technology actually does, under which conditions, on what evidence and with what limits. Institutions can then tell a demonstration from a dependable capability before they adopt, procure or regulate. Part of innovation and technological change.
01Workspace
The workspace for technology assessment.
GCRI · GCRI · Innovation pipelineIllustrative
Innovation pipeline
GCRI · Stage gates · Example interface, not a record
Gate review in 8 dGCRI · GCRI
By readiness level
TRL 2 to 301
Methods review cycleconcept tested
TRL 4 to 502
Evidence label auditlab validation
National Model buildlab validation
TRL 6 to 701
Report edition planningpilot with owner
Adoption review01
Annual mission designowner decides
Readiness criteria
pilot vs gate
Pilot issues
- I-1Data licence for pilot siteTo check9 d
- I-2Observer protocol signedDone8 d
GCRI · Innovation pipeline. Innovation pipeline: example names and illustrative values, not a record.
02What is examined
Four questions behind every assessment.
A fourth question runs through all three: what would make the technology fail, and has that been looked for?
Readiness
How mature is it?
Evidence on maturity in the intended setting, not in the most favourable one.
Validation
Has it been tested?
Independent tests against stated objectives, with the test conditions reported.
Claims
What exactly is claimed?
Capability statements separated into measured, estimated and asserted.
03Readiness scales
Readiness levels, what they measure and what they leave out.
Technology readiness levels run from one, basic principles observed, to nine, a system proven in its operational environment. NASA introduced them; ISO 16290:2013 defines them for space systems, and many agencies use adapted versions. A readiness level describes maturity in a stated environment. It says nothing about cost, safety in a different setting, social acceptance or whether an organisation can operate the technology.
| Question | Evidence that answers it | Common gap |
|---|---|---|
| Does it work as claimed? | Independent tests against a pre-registered plan, on held-out data or in a representative environment | Vendor benchmarks on data the system was tuned to |
| Does it work here? | Trials in the target context, with local users and conditions | Transfer from a different climate, language or infrastructure |
| What does it depend on? | Bills of materials, supply and energy needs, skills required | Hidden dependence on a single supplier or service |
| How does it fail? | Adversarial and stress tests; incident records | Testing only the expected cases |
04Measurement practice
Reporting capability honestly.
- Report performance with confidence intervals and on the population it will serve, not only on aggregate test sets.
- Check for contamination: test data that leaked into training inflates scores, a known problem for AI benchmarks.
- For prediction models in health and similar settings, follow reporting guides such as TRIPOD+AI (2024).
- Separate a demonstration, a pilot and sustained operation; each is a different claim.
The OECD’s Framework for Anticipatory Governance of Emerging Technologies of 2024 and parliamentary technology assessment offices, networked through EPTA, provide models of assessment that look beyond performance to values and institutions.
05GCRI’s part
Assessment reports, not approvals.
An assessment records what was tested, how and with what result. It confers no status and is never a condition of procurement. Assessments are designed for the Labs Evidence series and, when they concern readiness for national deployment, the Foundry Build and Readiness series; evidence would follow NS-11 of the proposed Nexus Standards. Providers can submit test plans; independent researchers can submit replications.
06Measurement rules
How capability is reported.
- Measured capability is published with workload, data size, hardware, configuration, software version and interval
- Peak, sustained, available and allocated capacity are reported separately
- Device counts and specifications are descriptors, not application results
- Energy, cooling and water use are tracked, with measurements kept apart from estimates
- Failures and limitations found during testing are reported, not removed
07How tests run
From evaluation plan to recorded result.
- 01
Evaluation plan
Objectives, metrics, conditions and stop rules agreed before testing.
- 02
Controlled test
Experiments in Labs test environments or the annual Universe programme, with independent observation.
- 03
Recorded result
Results with their method, limits and evidence label, published where rights allow.
08Boundaries
Testing is not certification.
An assessment describes what was tested and found. It is not accreditation, certification, endorsement or approval for use. Evaluation never depends on a provider’s contribution. Providers start at technology and infrastructure providers. See also responsible technology and the Labs Evidence report series.
09Sources
Where this page’s references lead.
The bodies and documents this page is read against. Status and editions change; each link goes to the issuing body.
- Technology readiness levels (NASA)
- ISO 16290:2013 Space systems, definition of the technology readiness levels and their criteria of assessment (ISO)
- Collins and others, TRIPOD+AI statement, BMJ 385 (2024)
- OECD Framework for Anticipatory Governance of Emerging Technologies (OECD, 2024)
- European Parliamentary Technology Assessment network (EPTA)
- NIST AI Risk Management Framework 1.0 (NIST AI 100-1, 2023) and the Generative AI Profile (NIST AI 600-1, 2024)
- NS-11, Verification and monitoring, a proposed standard
10Questions
Questions and answers.
Can a vendor cite a GCRI assessment in marketing?
Only as a factual citation of the published result, with its scope and limits, and never as certification or endorsement. Misrepresentation can be reported to the integrity address.
Who pays for an assessment?
Terms are set out for each testbed call or agreement. Payment never buys a result, and evaluation never depends on a high contribution.
Are negative results published?
Yes, where rights allow. A technology that fails a test produces a result as valid as one that passes.
Does GCRI use readiness levels?
Readiness is described with evidence for the specific setting. Where a recognised readiness scale is referenced, it is named and the evidence for the level is shown.
Can an assessment be challenged?
Yes. Providers, researchers and the public can challenge a published assessment through the integrity route. An accepted correction creates a new recorded version with its history.
ReportsContribute
Contribute evidence from this page.
Reports is the Consortium’s quarterly publication. What you know about this subject can become evidence, a case or a correction in the next edition, credited to you and published under CC BY 4.0 where it is your own work.
GCRI
- EmailGateway GW-GCRISend it from this pageEvidence, a correction, a case, a method, a review, a translation or profile input. It arrives tagged with this page, so the editor who owns it receives it.
- ZenodoOpen depositDeposit it and submit to the communityPublish your own record on Zenodo with your ORCID iD, then submit it to the Reports community there. Curators review it; the DOI stays yours.
- ReportsGCRISee every way to contributeThe contribution types, how each is reviewed, what credit you receive, the licence and what is never accepted.
Never send personal data, sensitive infrastructure detail or anything you have no right to share. A contribution is reviewed before use; sending it confers no authorship or editorial rights, and sponsorship never buys a finding.
The Global Centre for Risk and Innovation
Bring a method, a dataset or a question.
Information and enquiries are open today. An enquiry receives a reference and names the next step; applications, payments, compute allocations and registrations open only when their team, terms and review capacity exist.
Legal
Legal documents.
Maps, data and figures on this site are shown for context and general information. They are not advice, forecasts or warnings; the limits of the information are set out in the Disclaimers and Data Notices.