Assessment, Quality & Accessibility

8.1 Assessment Types (QZ/LB/PRJ/OD/RT)

• QZ — Quiz/Exam: proctored or supervised; fixed blueprint; randomized items; multiple forms (A/B/C).
• LB — Lab: hands-on task (e.g., EO/GIS workflow, DSS dashboard, trigger design) with rubric; artifact required.
• PRJ — Project/Capstone Component: multi-step deliverable with checkpoints, peer review (optional), and faculty grading.
• OD — Oral Defense: structured viva or board-style defense; recorded; rubric-scored; panel notes retained.
• RT — Red-Team Exercise: adversarial evaluation (cyber/model/ops); finding remediation evidenced; pass contingent on closed-loop fixes.
  Rule: Every MC includes at least one LB and one graded component (QZ/OD/RT). Programs require a PRJ capstone.

8.2 Psychometrics Targets (Alpha, Discrimination, DIF)

• Reliability: KR-20/α ≥ 0.75 (QZ); ≥ 0.80 for high-stakes capstone written exams.
• Item discrimination: point-biserial ≥ 0.25 (median); flag < 0.15 for review.
• Difficulty (p-value): target 0.35–0.85; maintain spread per blueprint.
• DIF (fairness): Mantel–Haenszel |ΔMH| < 1.0 (moderate+ flagged); IRT-based DIF where item pools allow.
• Form equivalence: score means within 0.25 SD after equating; bias checks by cohort/language.
• Action: flagged items reviewed within one release; retroactive score adjustments documented.

8.3 Standard-Setting (Modified Angoff/Bookmark)

• Modified Angoff (default): panelists estimate % minimally competent candidates answering each item correctly; apply Hofstee guardrails; set cut score; document panel calibration.
• Bookmark (alt for IRT pools): order items by difficulty; panel places bookmark at minimally acceptable ability; convert to raw cut.
• Revalidation: at least biennial or when blueprint changes; publish cut-score rationale in course QA notes.

8.4 Item Banking & IMS QTI Packaging

• Bank: version-controlled items with metadata: objective, standard tag, difficulty, discrimination, language, accessibility notes, exposure count, last review date.
• Security: randomization, time windows, IP/device checks (where permitted), item exposure cap, retired-item quarantine.
• Forms: at least A/B (prefer A/B/C) with 20–30% anchor items for equating.
• Packaging: export IMS QTI (2.2+) for portability; include media with captions/alt text; MathML for equations.
• Change control: any edited item increments item rev; keep prior revs for audit.

8.5 Learning Analytics (xAPI/Caliper) & LRS

• Events: enroll, start, item response, lab step, submission, grade, feedback viewed, badge issued, verification.
• Standards: emit xAPI (primary) and Caliper (optional) statements; store in LRS with 1-year hot / 5-year warm retention (PII minimized).
• Dashboards: instructor views reliability and discrimination; learner sees progress, attempt analytics, renewal status.
• Research use: de-identified aggregates; IRB/ethics review where required.

8.6 WCAG 2.2 AA Accessibility Requirements

Mandatory for all content and tools.

• Non-text alternatives: alt text for images, transcripts for audio, captions + transcripts for video; audio description where essential.
• Operable: full keyboard navigation; visible focus; no keyboard traps; bypass blocks.
• Understandable: consistent navigation; clear labels; error identification; reflow and zoom up to 200%.
• Predictable & robust: no timeouts without warning & extension; ARIA roles correct; color contrast ≥ 4.5:1 (text).
• New 2.2: focus not obscured; dragging alternatives; target size guidelines met.
• Evidence: automated audit + manual checks logged at each release.

8.7 EN 301 549 / Section 508 Compatibility

• Mapping: WCAG 2.2 AA conformance plus ICT procurement clauses (captioning, documentation accessibility, platform AT compatibility).
• Procurement: vendor tools (LTI, proctoring, GIS) must supply VPAT/EN 301 549 conformance statements; exceptions require mitigation plan.

8.8 Accommodations & Inclusive Design

• Approved accommodations: extended time, alternative formats (HTML/PDF/EPUB), large print, color-safe palettes, screen-reader-friendly math, sign-language or CART for OD, alternative input for drag actions, low-bandwidth media.
• Process: request at enrollment; decision ≤ 5 business days; confidential; applied across attempts.
• Inclusive defaults: multilingual UI (EN/FR/ES/AR & RtL), plain-language summaries, captions on by default, downloadable resources.

8.9 QA Gates & Release Workflow

Pre-publish CI/CD gates (must pass):

1. Catalog linter (codes, metadata, standards/competencies, i18n flags).
2. Accessibility audit (WCAG 2.2 AA automated + spot manual).
3. Psychometrics (for updated QZ): reliability ≥ 0.75 (or pilot plan on first release).
4. Provenance: data lineage & licenses (SPDX) recorded; artifact templates updated.
5. Security: item exposure limits; proctoring policy; content license checks.
6. Release notes: rev bump (MAJOR/MINOR/PATCH), change log, delta-exam impact.
7. External review (high-stakes): independent SME & accessibility reviewer sign-off.

Post-publish: telemetry watch (48–72h), rapid rollback option for critical defects.

8.10 Academic Integrity, AI Use & Provenance

• Integrity: plagiarism checks (text/code), similarity reports, reference audits; proctoring and ID verification where justified and lawful.
• AI/tool use: permitted only as declared in each course; must cite model/tool and version; no undisclosed generative content.
• Data ethics: use only licensed/consented datasets; redact sensitive fields; dual-use review for cyber/AI/bio/EO labs.
• Provenance: every artifact displays data sources, processing steps, model cards, and license; artifacts hashed; public ones may get DOIs.
• Sanctions: warnings → zero on assignment → revocation (serious breaches). Appeals follow registrar policy.

Acceptance Checklist (Assessment, Quality & Accessibility)

• Blueprint and rubrics published; at least one LB and one graded component per MC.
• Reliability ≥ 0.75; discrimination ≥ 0.25 median; DIF checks completed.
• Cut score set via Modified Angoff/Bookmark with recorded panel notes.
• Items packaged in IMS QTI; forms A/B (or A/B/C) with anchors; exposure controls active.
• xAPI/Caliper events flowing to LRS; dashboards render instructor/learner views.
• WCAG 2.2 AA audit passed; VPAT/EN 301 549 statements for third-party tools on file.
• Accommodations configured and documented for the run.
• CI/CD gates passed; rev bumped; change log and (if MAJOR) delta exam posted.
• Academic integrity and AI-use policy clearly stated; provenance recorded.
• Post-publish telemetry monitored; rollback plan ready.