Assessment, Quality & Accessibility

8.1 Assessment Types (QZ/LB/PRJ/OD/RT)

QZ — Quiz/Exam: proctored or supervised; fixed blueprint; randomized items; multiple forms (A/B/C).
LB — Lab: hands-on task (e.g., EO/GIS workflow, DSS dashboard, trigger design) with rubric; artifact required.
PRJ — Project/Capstone Component: multi-step deliverable with checkpoints, peer review (optional), and faculty grading.
OD — Oral Defense: structured viva or board-style defense; recorded; rubric-scored; panel notes retained.
RT — Red-Team Exercise: adversarial evaluation (cyber/model/ops); finding remediation evidenced; pass contingent on closed-loop fixes.
Rule: Every MC includes at least one LB and one graded component (QZ/OD/RT). Programs require a PRJ capstone.

Reliability: KR-20/α ≥ 0.75 (QZ); ≥ 0.80 for high-stakes capstone written exams.
Item discrimination: point-biserial ≥ 0.25 (median); flag < 0.15 for review.
Difficulty (p-value): target 0.35–0.85; maintain spread per blueprint.
DIF (fairness): Mantel–Haenszel |ΔMH| < 1.0 (moderate+ flagged); IRT-based DIF where item pools allow.
Form equivalence: score means within 0.25 SD after equating; bias checks by cohort/language.
Action: flagged items reviewed within one release; retroactive score adjustments documented.

Modified Angoff (default): panelists estimate % minimally competent candidates answering each item correctly; apply Hofstee guardrails; set cut score; document panel calibration.
Bookmark (alt for IRT pools): order items by difficulty; panel places bookmark at minimally acceptable ability; convert to raw cut.
Revalidation: at least biennial or when blueprint changes; publish cut-score rationale in course QA notes.

Bank: version-controlled items with metadata: objective, standard tag, difficulty, discrimination, language, accessibility notes, exposure count, last review date.
Security: randomization, time windows, IP/device checks (where permitted), item exposure cap, retired-item quarantine.
Forms: at least A/B (prefer A/B/C) with 20–30% anchor items for equating.
Packaging: export IMS QTI (2.2+) for portability; include media with captions/alt text; MathML for equations.
Change control: any edited item increments item rev; keep prior revs for audit.

Events: enroll, start, item response, lab step, submission, grade, feedback viewed, badge issued, verification.
Standards: emit xAPI (primary) and Caliper (optional) statements; store in LRS with 1-year hot / 5-year warm retention (PII minimized).
Dashboards: instructor views reliability and discrimination; learner sees progress, attempt analytics, renewal status.
Research use: de-identified aggregates; IRB/ethics review where required.

Mandatory for all content and tools.

Non-text alternatives: alt text for images, transcripts for audio, captions + transcripts for video; audio description where essential.
Operable: full keyboard navigation; visible focus; no keyboard traps; bypass blocks.
Understandable: consistent navigation; clear labels; error identification; reflow and zoom up to 200%.
Predictable & robust: no timeouts without warning & extension; ARIA roles correct; color contrast ≥ 4.5:1 (text).
New 2.2: focus not obscured; dragging alternatives; target size guidelines met.
Evidence: automated audit + manual checks logged at each release.

Mapping: WCAG 2.2 AA conformance plus ICT procurement clauses (captioning, documentation accessibility, platform AT compatibility).
Procurement: vendor tools (LTI, proctoring, GIS) must supply VPAT/EN 301 549 conformance statements; exceptions require mitigation plan.

Approved accommodations: extended time, alternative formats (HTML/PDF/EPUB), large print, color-safe palettes, screen-reader-friendly math, sign-language or CART for OD, alternative input for drag actions, low-bandwidth media.
Process: request at enrollment; decision ≤ 5 business days; confidential; applied across attempts.
Inclusive defaults: multilingual UI (EN/FR/ES/AR & RtL), plain-language summaries, captions on by default, downloadable resources.

Pre-publish CI/CD gates (must pass):

Catalog linter (codes, metadata, standards/competencies, i18n flags).
Accessibility audit (WCAG 2.2 AA automated + spot manual).
Psychometrics (for updated QZ): reliability ≥ 0.75 (or pilot plan on first release).
Provenance: data lineage & licenses (SPDX) recorded; artifact templates updated.
Security: item exposure limits; proctoring policy; content license checks.
Release notes: rev bump (MAJOR/MINOR/PATCH), change log, delta-exam impact.
External review (high-stakes): independent SME & accessibility reviewer sign-off.

Post-publish: telemetry watch (48–72h), rapid rollback option for critical defects.

Integrity: plagiarism checks (text/code), similarity reports, reference audits; proctoring and ID verification where justified and lawful.
AI/tool use: permitted only as declared in each course; must cite model/tool and version; no undisclosed generative content.
Data ethics: use only licensed/consented datasets; redact sensitive fields; dual-use review for cyber/AI/bio/EO labs.
Provenance: every artifact displays data sources, processing steps, model cards, and license; artifacts hashed; public ones may get DOIs.
Sanctions: warnings → zero on assignment → revocation (serious breaches). Appeals follow registrar policy.

Blueprint and rubrics published; at least one LB and one graded component per MC.
Reliability ≥ 0.75; discrimination ≥ 0.25 median; DIF checks completed.
Cut score set via Modified Angoff/Bookmark with recorded panel notes.
Items packaged in IMS QTI; forms A/B (or A/B/C) with anchors; exposure controls active.
xAPI/Caliper events flowing to LRS; dashboards render instructor/learner views.
WCAG 2.2 AA audit passed; VPAT/EN 301 549 statements for third-party tools on file.
Accommodations configured and documented for the run.
CI/CD gates passed; rev bumped; change log and (if MAJOR) delta exam posted.
Academic integrity and AI-use policy clearly stated; provenance recorded.
Post-publish telemetry monitored; rollback plan ready.