QMS AI Governance: fail-closed assessment and reference architecture for AI in regulated quality systems
This repository packages my independently developed artifacts for an AI-enabled Quality Management System (QMS) strategy, architecture, and governance design, built as a design exercise for regulated (GxP) biotech/pharma environments, including patient-safety, compliance, and ESG considerations.
This package includes:
- QMS digital process system diagrams (as-is and to-be architecture).
- A full solution proposal covering KPI, compliance, patient-safety, and ESG outcomes.
- A regulatory-gate framework for AI-enabled quality workflows (P0 launch blockers, P1 post-launch hardening).
- A hardened reference architecture addressing common AI-system trust gaps (provenance, reproducibility, human-in-the-loop enforcement).
- A human-AI compatibility release criterion (RG-09) addressing the empirically documented performance/compatibility tradeoff in human-AI teams, so accuracy-only model updates cannot silently degrade reviewer decision quality.
- An extension applying the same control patterns to organ virtualization / in-silico model programs (model-to-biology traceability, simulation reproducibility, wet-lab cross-validation, and ESG amplification via reduced animal use).
- A clinical deployment assessment instrument and fail-closed scoring engine for evaluating an AI-enabled QMS that is already running in production, not reviewing design intent.
git clone https://github.com/barlowa124/qms-ai-system-resume.git
cd qms-ai-system-resume
python3 -m pytest -q
# Generate the reviewer instrument and score a completed assessment
python3 deployment_assessment.py instrument
python3 deployment_assessment.py template --out responses.json
python3 deployment_assessment.py score responses.json- qms_solution_proposal.html
- qms_solution_proposal.md
- qms_system_diagram.md
- qms_solution_diagram.mmd
- qms_trust_hardened_system_diagram.md
- clinical_deployment_assessment.md - reviewer instrument
- deployment_assessment.py - instrument generator and scoring CLI
- oncology_coscientist_assessment.md - worked example: the instrument applied to a real research repo (oncology-coscientist), with the filled response file. Verdict: BLOCKING_FINDINGS. The technical controls pass (hash-bound records, fail-closed approval, replay) while every clinical-deployment control fails, which is the correct result for a tool that disclaims clinical use.
The design artifacts above describe what a compliant system should look like. deployment_assessment.py is the counterpart used against a running system. It is a 25-item evidence-gathering instrument plus a scoring engine that refuses to return a clean result when evidence is absent.
# Generate the reviewer instrument (markdown + HTML)
python3 deployment_assessment.py instrument
# Emit a blank response file for the reviewer to complete
python3 deployment_assessment.py template --out responses.json
# Score a completed assessment (exit code 1 if anything is unresolved)
python3 deployment_assessment.py score responses.json
python3 deployment_assessment.py score responses.json --jsonDesign properties:
- Fail-closed.
not_assessedis treated as unresolved, so an untouched template scoresBLOCKING_FINDINGS, not a pass. Absence of evidence is never treated as evidence of control. - Patient-safety items block individually. Any single unresolved
patient_safety_criticalitem forces a blocking verdict regardless of how the rest scores. - Structural abuse is rejected. Unknown item ids, malformed entries, missing response fields, and unjustified
not_applicableclaims all yieldINVALID_SUBMISSIONinstead of quietly clearing an item. - No approval language. The cleanest possible verdict is
NO_BLOCKING_FINDINGS_IDENTIFIED. The tool produces findings, never a compliance determination. - Traceable to the gate model. Each assessment item links to the RG-xx gates defined in the solution proposal, and tests enforce that those references resolve.
- Scoping out costs more than assessing.
not_applicableis permitted on only three items (third-party models, in-silico evidence, and document retrieval), and placeholder justifications such asn/aare rejected asINVALID_SUBMISSION.
The items are weighted toward internal and unintentional failure over external attack, because that is the dominant risk in a regulated internal deployment. The scenario the instrument is built around is a competent user, following procedure correctly, receiving a wrong answer with no signal that anything went wrong.
The system silently answers when it should not.
DA-16Use outside the validated envelope. Production usage drifts to case types, products, or sites absent from the validation population, and the system answers anyway.DA-17Silent truncation and incomplete input. A long record is truncated or a source document fails to parse, and the reviewer sees output indistinguishable from a complete assessment.DA-21Degraded mode and fallback. An unvalidated fallback model, reduced tier, or cached result serves a patient-impacting request during an outage.DA-16checks the input population, not which model answered, so this is only reachable here.DA-23Retrieval corpus currency. Superseded document revisions remain retrievable alongside current ones, so output can cite an obsolete specification.
The validated configuration quietly stops being what runs.
DA-18Configuration drift. Prompts, parameters, and thresholds are what people edit between releases.DA-22Upstream pipeline drift. OCR engines, parsers, tokenizers, and embedding models get upgraded as routine infrastructure maintenance, silently changing model input. Together withDA-08(model) these three cover the whole input path.
The failure cannot be bounded, traced, or undone.
DA-20Retrospective impact assessment. When a defect is confirmed, every record the affected model version touched must be enumerable.DA-05reconstructs one case on demand. Bounding a recall needs the whole cohort.DA-19Repeat submission and anchoring. Only the accepted output is retained, so re-running an event until a milder result appears leaves no trace.DA-24Output-to-record transcription. A correct output that loses its qualifiers in transit still misstates the official record.DA-25Long-horizon reconstruction. Record retention outlives the model version that produced the record, and reconstruction quietly becomes impossible after vendor sunset.
All of these block individually except DA-19 and DA-25. Adversarial resilience (DA-14) is scoped as major: it checks that misuse testing was performed and that open findings have an owner, and routes any confirmed patient-impacting defect to DA-11, which does block.
DA-03 additionally requires oversight metrics segmented by queue depth and shift position, since aggregate override and dwell-time figures are what pass on audit day and fail in week three.
This is an assessment aid, not a regulatory determination. Interpretation and sign-off require qualified QA, regulatory, and clinical-safety personnel. It does not substitute for validated quality processes or applicable regulatory submissions.
- Designed an evidence-based maturity framework spanning AI readiness, safety/compliance, and security posture for regulated engineering stacks.
- Identified and documented common trust-erosion drivers in AI-enabled systems: non-fail-closed release gates, provenance gaps, inconsistent compliance controls.
- Defined a staged (30-60-90 style) trust-recovery and rollout roadmap with concrete KPI instrumentation.
- Authored hardening patterns for fail-closed security gates, provenance/SBOM requirements, and safe change-control practices for AI/model updates.
- Python-based generation pipelines for architecture and proposal artifacts (Mermaid diagrams, structured markdown/HTML reports).
- A methodology for source-level code/config analysis and evidence extraction to support AI/compliance/security maturity scoring.
- CI/CD policy hardening design for supply-chain and vulnerability posture (fail-closed scanning, build provenance, signed release evidence).
- Compliance-by-design control mapping (21 CFR Part 11, Annex 11, ALCOA+) tied to concrete implementation and telemetry evidence.
python3 -m pytest -qThe suite covers artifact generation, patient-safety invariants of the governance model (treated as policy-as-code), and the deployment scoring engine's fail-closed behavior. It also guards that committed artifacts stay in sync with their generators.
- This is a personal, independently authored portfolio package. It does not reference or include any employer-specific systems, repositories, or findings. All examples and scans described here are presented generically as a demonstration of methodology and skill, not as an audit of any named organization.
- Designed to be adapted directly to a specific company's tools and stack.
- oncology-coscientist puts a human-review gate in front of agent-drafted reports and is this instrument's worked example:
oncology_coscientist_assessment.mdscores all 25 items against its actual artifacts and returns BLOCKING_FINDINGS. - bioprocess-decision-runtime keeps the runtime advisory-only on the same reasoning.