Validation strategy

Consensus Engine is deterministic, which makes it testable. For a defined patient fact set, the expected output should be known in advance. The engine should not behave differently each time it runs, and it should not depend on a generative model’s interpretation of the case. It evaluates structured facts against reviewed rules and produces a reproducible result. Validation is the process that proves the engine is doing that safely.

​

Validation philosophy

The validation strategy is built around four layers: Clinical safety cannot depend on trust in the product team alone. It must be demonstrated through repeatable tests.

​

What validation must prove

Validation should prove more than “the software runs.” A clinical engine should not only be correct in common cases. It must also be safe in edge cases.

​

Golden test cases

Golden test cases are predefined patient scenarios with expected outputs. Each includes a test case ID, patient profile, biomarker inputs, context inputs, expected derived values, expected rule behavior, expected calibration behavior, expected suppression behavior, expected state, expected visibility, expected evidence trace, and a pass/fail result. Representative cases include a healthy baseline expected to produce OPTIMAL behavior, a severe hyperkalemia case expected to escalate to CRITICAL, and a TSH case affected by high-dose biotin expected to route to REQUIRES_CLINICAL_CORRELATION.

​

Test case categories

The golden test suite should cover the full clinical surface of the engine. The goal is to test not only expected positives, but also expected blocks. A rule that correctly does not fire is as important as a rule that does.

​

Example golden test structure

​

Calibration-specific validation

Calibrations require dedicated tests because they change interpretation based on context. Each calibration should have at least four types of tests: it applies correctly when required facts are present, does not over-apply when facts are absent, handles uncertainty by routing suggestive or conflicting data to review, and leaves a trace showing calibration ID, version, evidence, and rationale. For example, a Duffy-null/ACKR1 low neutrophil value should not be mislabeled without context; APOL1 kidney-risk calibration should require confirmed or appropriately governed genotype context; HbA1c interpretation should be limited or routed when reliability is uncertain; and vitamin D context calibration should apply only under approved context and evidence logic.

​

Suppression and hardening validation

Suppression and hardening rules are safety-critical. They prevent the system from converting unreliable or incomplete data into overconfident conclusions. Validation should confirm pregnancy suppression, acute illness suppression, recent exercise suppression, the fasting gate, diabetes hardening, CKD hardening, and assay confounding routing.

​

Boot-time validators

Boot-time validators run before the engine serves output, catching internal misconfiguration early. This prevents a technically valid but clinically unsafe engine state from reaching patients. Boot validators are not optional technical polish. They are part of the medical safety system.

​

Runtime output validation

Every evaluation should produce a complete runtime output record — the unit of auditability. Runtime output validation confirms a unique decision ID, patient and organization reference, input facts listed, rules fired listed, a calibration trace when applicable, an attached evidence source, captured lifecycle status, captured state, captured visibility, and stored version context. A decision without a trace should be treated as incomplete.

​

Medical Director review

Automated testing can confirm that logic behaves as expected. It cannot confirm that the logic is clinically appropriate — that requires medical review. The Medical Director reviews clinical correctness, evidence basis, preventive posture, safety posture, calibration logic, review routing, treatment boundaries, patient language, and release readiness.

​

Validation status reporting

Validation status should be reported carefully.

​

Release gating

Validation connects directly to release. A rule, calibration, pattern, or treatment flag should not become ACTIVE until the relevant review and validation gates are complete: evidence source attached, Medical Director review approved, golden tests passing, boot validators passing, runtime trace complete, visibility review complete, patient message approved, safety interactions reviewed, treatment boundary confirmed where applicable, version record created, rollback path available, and no open safety blockers. This is the safety guarantee: patient exposure is controlled by the system, not by informal judgment.

​

Outcomes monitoring

Validation does not end at release. Once live, outcomes monitoring evaluates whether rules and calibrations behave as expected in real-world use. Accumulated decision records, inputs, calibrations, and outcomes form a longitudinal dataset that can refine calibrations over time — but only under consent, privacy, and medical governance.

​

Regression testing

Every schema change should trigger regression testing, because changes can have indirect effects: a new unit conversion can affect multiple markers, a calibration can change a downstream pattern, a safety suppression can prevent a treatment flag, and a wording change can alter patient interpretation. Regression testing should run after a new biomarker, threshold update, calibration update, pattern update, safety rule update, formula update, runtime fact update, patient message update, lifecycle transition, or AI workflow update. A deterministic engine makes regression testing practical: the same input should produce the same expected output unless an approved change intentionally modifies behavior.

​

Failure handling

Validation should define what happens when something fails. A validation framework that only reports problems but does not block unsafe release is incomplete.

​

Documentation requirements

Each validation cycle should create a record including the schema version, test suite version, execution date, test environment, test results (pass, fail, skipped, or blocked), open issues, medical reviewer, engineering reviewer, release decision, and notes. This supports due diligence, internal governance, and later regulatory review.

​

Validation maturity model

The validation program can mature in phases. Not every layer has to be mature on day one, but the roadmap and current status must be clear.