Explainable AI vs Verifiable AI
Explainability tells you why the model reasoned. Verifiability proves what the system actually did. They are different problems with different solutions.
Explainable AI (XAI) — SHAP, LIME, attention visualization, counterfactual explanations, interpretable models — addresses the question "why did the model produce this output?" Verifiable AI addresses a different question: "can a third party prove what the system actually did, without trusting the system?" Both are needed for serious AI governance. Conflating them — assuming explainability provides what verification provides — leads to compliance gaps that surface in audits, regulatory reviews, and litigation.
What explainable AI provides
Explainable AI methods produce human-interpretable explanations of model behavior. Feature attribution methods (SHAP, LIME, integrated gradients) decompose a prediction into contributions from input features. Counterfactual explanations describe minimal input changes that would flip the prediction. Attention visualization for sequence models highlights which input tokens the model attended to most heavily. Interpretable models (decision trees, linear models, rule lists) are inherently explainable. Concept-based explanations describe model behavior in terms of higher-level concepts. Explainable AI supports several governance goals: helping users understand decisions, surfacing potential bias, debugging behavior, demonstrating reasoning, satisfying transparency requirements.
What explainable AI does not provide
Explainable AI does not provide cryptographic evidence that: the model that produced the decision was actually the model the entity claims to have used; the data inputs were the inputs the entity claims they were; the decision was made under the policy the entity claims was in force; the system has not been modified between the decision and the audit; the explanation itself is the explanation the system actually produced. These are verification questions, not explanation questions. The explanation describes the model's reasoning; the verification establishes whether the described reasoning is what actually happened. A model that produces a SHAP explanation is providing a derived view of its own behavior. The view is interpretable. It is not cryptographically bound to the actual processing.
What verifiable AI provides
Verifiable AI produces cryptographic evidence about what the system actually did. H33 evidence bundles document the model identity and version (ModelFingerprint), the policy in force (PolicyBind), the principal authorized (AuthorityBind), the execution pipeline (PipelineDag), the data corpus consulted (CorpusBind), the evidence rows that grounded the decision (EvidenceAttestation), the binding between output and citations (ResultCitationBind), and the system's confidence and abstention behavior (CalibratedAbstention). The bundle is signed with three independent post-quantum algorithm families. The bundle does not explain the model's reasoning. It documents what the system actually did.
Side-by-side
| Dimension | Explainable AI | Verifiable AI |
|---|---|---|
| Primary question | Why did the model produce this output? | What did the system actually do? |
| Output | Human-interpretable explanation | Cryptographic evidence bundle |
| Audience | Users, stakeholders, modelers | Regulators, auditors, courts |
| Verifiability | Explanation can be challenged | Evidence is cryptographically signed |
| Independent of producing system | Depends on the model behaving as described | Verifiable without trusting the producer |
| Regulatory acceptance | Satisfies transparency requirements | Satisfies verification requirements |
When both are needed
A lending decision. Explainable AI tells the applicant which factors contributed to the denial (applicant-facing). Verifiable AI tells the regulator that the model the bank claims was used was actually used, with the claimed policy and authority (regulator-facing). A healthcare AI recommendation. Explainable AI tells the clinician what factors the model weighed (clinical utility). Verifiable AI tells the malpractice investigator that the clinical decision support system actually consulted the patient record and produced the recommendation under the documented protocol (litigation-facing). A federal benefits determination. Explainable AI tells the applicant what factors affected the eligibility decision (applicant-facing). Verifiable AI tells the appeals reviewer that the AI used was the AI authorized for the determination type (appeals-facing). In each case, the two layers serve different audiences and answer different questions.
Common questions
Does H33 produce explanations?
H33 documents what the system did, not why the model decided. H33 captures whether explainability was used but does not itself generate explanations.
Can the explanation be cryptographically signed?
Yes. The explanation itself can be included in the H33 bundle, signed alongside the decision.
Is verifiability subject to the same challenge as explainability?
No. Explainability is a derived view; verifiability is direct cryptographic evidence.
Does H33 work with SHAP, LIME, or other XAI tools?
Yes. H33 is XAI-agnostic. The bundle can include the explanation method used.
What about the EU AI Act?
EU AI Act Article 13 supports explainability requirements. Articles 12 and 17 support verifiability requirements. Both are needed for full Act compliance for high-risk systems.
Related: Verifiable AI Decisions · Verifiable AI Actions · AI Decision Provenance · AI Compliance Evidence