The Short Answer: No, ChatGPT Is Not HIPAA Compliant by Default
If you are using ChatGPT Free, Plus, or Team — your organization is not HIPAA compliant. OpenAI's terms of service for these tiers explicitly state that user inputs may be used to train and improve the model. That alone is a HIPAA violation if any Protected Health Information (PHI) touches a prompt. There is no ambiguity here: sending a patient's name, diagnosis, medication list, or insurance ID into a consumer-tier LLM is a reportable breach under 45 CFR § 164.
ChatGPT Enterprise and the OpenAI API (with an Enterprise agreement) do offer a Business Associate Agreement (BAA). But a BAA does not mean what most healthcare leaders think it means. It does not mean the model never sees your data. It does not mean your PHI is encrypted during processing. It means OpenAI contractually agrees to be a "business associate" under HIPAA — accepting certain obligations around data handling, breach notification, and access controls. The data itself is still processed in plaintext.
A BAA is a contractual safeguard, not a technical one. Your PHI still exists unencrypted in OpenAI's RAM, GPU VRAM, inference caches, and potentially in log files during every API call. If OpenAI's infrastructure is breached, every prompt you ever sent is exposed — BAA or not.
What the OpenAI BAA Actually Covers
OpenAI's BAA is available exclusively for the Enterprise API tier. Here is what it covers — and what it does not:
| Tier | BAA Available | Data Used for Training | PHI Permitted |
|---|---|---|---|
| Free | No | Yes | No |
| Plus ($20/mo) | No | Yes (opt-out available) | No |
| Team | No | No (business data excluded) | No |
| Enterprise API | Yes | No | With BAA only |
Even with an Enterprise BAA in place, the agreement does not cover the ChatGPT web interface. If a clinician logs into chatgpt.com and types a patient question — even on an Enterprise account — that interaction may not be covered under the BAA. The BAA covers the API. The distinction matters because the web UI and the API have different data retention policies, different logging infrastructure, and different access control models.
The BAA also does not address data that may be cached in inference infrastructure, retained in ephemeral logs for debugging, or temporarily stored in GPU memory during processing. OpenAI commits to not training on Enterprise API data, but "not training" and "not retaining" are different promises with different risk profiles.
The Real Risk: Plaintext Inference
The fundamental architectural problem is not about contracts or policies. It is about physics. For a large language model to process your prompt, it must read your prompt. In plaintext. In memory. On a GPU. On a server you do not control.
When you send a prompt containing PHI to any LLM — ChatGPT, Claude, Gemini, Llama hosted on a cloud provider — the following happens:
- Your text is decrypted from TLS at the load balancer or API gateway
- The plaintext prompt is tokenized and loaded into GPU VRAM
- The model performs inference on unencrypted tokens
- The response is generated in plaintext before being encrypted for return
- The prompt may exist in memory, cache, swap, or logs for an indeterminate period
This is not a theoretical risk. In March 2023, a ChatGPT bug exposed chat history titles across users. In 2024, multiple AI companies disclosed data exposure incidents involving inference logs. Samsung banned ChatGPT internally after employees leaked proprietary source code through prompts. Every one of these incidents happened because the data existed in plaintext on infrastructure the user did not control.
A BAA means OpenAI will notify you of a breach. It does not prevent one. If an attacker gains access to OpenAI's inference infrastructure, every prompt containing PHI is immediately exposed. The BAA is a liability agreement. It is not a firewall.
What "HIPAA Compliant AI" Actually Requires
The HIPAA Security Rule (45 CFR § 164.312) mandates specific technical safeguards for electronic Protected Health Information (ePHI). Most AI vendors satisfy two of the three encryption requirements. The third is the one that matters most:
| Safeguard | Requirement | ChatGPT Enterprise | FHE-Based Inference |
|---|---|---|---|
| Encryption at rest | Data encrypted on disk | Yes (AES-256) | Yes |
| Encryption in transit | Data encrypted over network | Yes (TLS 1.3) | Yes |
| Encryption in use | Data encrypted during processing | No | Yes (FHE) |
| Access controls | Role-based, minimum necessary | Yes | Yes |
| Audit trails | Immutable access logs | Partial | Yes (Dilithium-signed) |
| Breach notification | 60-day notification window | Yes (via BAA) | N/A (no plaintext to breach) |
The gap is encryption in use. Every current LLM provider — OpenAI, Anthropic, Google, Meta — processes your data in plaintext during inference. They encrypt it before (in transit) and after (at rest). But during the actual computation, the moment the model reads your prompt and generates a response, the data is naked. That gap is the entire attack surface.
The HIPAA Security Rule's addressable specification for encryption (§ 164.312(a)(2)(iv) and § 164.312(e)(2)(ii)) does not explicitly mandate encryption in use — because when HIPAA was written, the concept did not exist. But the minimum necessary standard (§ 164.502(b)) and the general security requirement (§ 164.306(a)) together demand that covered entities implement safeguards "reasonably anticipated" to protect ePHI. In 2026, with FHE commercially available, arguing that plaintext inference is "reasonable" is increasingly difficult.
The FHE Solution: Inference on Encrypted Data
Fully Homomorphic Encryption (FHE) eliminates the plaintext gap entirely. With H33 AI Compliance, the workflow changes fundamentally:
- Client encrypts the prompt locally using an FHE public key. The plaintext never leaves the client.
- Server receives ciphertext — an encrypted blob that is mathematically indistinguishable from random noise.
- Model performs inference on ciphertext — the computation happens on encrypted data using homomorphic operations.
- Server returns encrypted result — still ciphertext.
- Client decrypts locally with the private key that never left their device.
At no point does the server — or the model, or the GPU, or the cache, or the logs — ever see plaintext PHI. This is not a policy promise or a contractual commitment. It is a mathematical impossibility. Without the private key (which only the client holds), the server cannot recover the plaintext, regardless of how many vulnerabilities exist in the infrastructure.
With FHE-encrypted inference, a breach at the AI provider exposes nothing. An attacker who compromises the entire inference stack — every GPU, every log file, every memory dump — obtains only ciphertext. The breach notification requirement under HIPAA's safe harbor provision (45 CFR § 164.402(2)) explicitly excludes data that was encrypted with a key not compromised in the breach. FHE makes AI inference breach-proof by architecture.
H33's FHE engines support encrypted inference across clinical NLP tasks including medical record summarization, ICD-10 code suggestion, prior authorization language processing, and patient communication drafting — all without exposing a single byte of PHI. Combined with post-quantum key exchange (Kyber-1024) and Dilithium-signed audit trails, the entire pipeline is both HIPAA compliant today and quantum-resistant for the future.
What to Do Today
If your organization is using ChatGPT or any large language model with PHI — or is evaluating doing so — here is the minimum action plan:
1. Upgrade to Enterprise with a BAA — minimum. If you are on Free, Plus, or Team and any employee has ever typed PHI into ChatGPT, you likely have a reportable breach. Upgrade immediately and execute a BAA. This is the floor, not the ceiling.
2. Never put PHI in the web UI. Even with Enterprise, route all PHI-containing requests through the API exclusively. The web interface has different data handling characteristics that may not be covered under your BAA.
3. Evaluate FHE-encrypted inference for production workloads. For any workflow where an LLM processes PHI at scale — clinical documentation, diagnostic support, patient messaging — plaintext inference is an architectural liability. FHE eliminates the liability entirely. See H33 AI Compliance and MedVault for healthcare-specific deployments.
4. Document your risk assessment. HIPAA requires covered entities to conduct and document a risk analysis (§ 164.308(a)(1)(ii)(A)). Your risk assessment must now account for the AI inference gap. If FHE is commercially available and you chose plaintext inference instead, your risk acceptance documentation needs to explain why.
The healthcare industry adopted cloud computing, then spent a decade retrofitting security. AI inference is following the same pattern — adoption first, security later. The difference is that FHE exists now. You do not have to wait for the breach to justify the architecture.
The Bottom Line
ChatGPT is not HIPAA compliant by default. The Enterprise tier with a BAA is the minimum viable configuration, but it still processes PHI in plaintext. A BAA is a legal document — it does not encrypt your data during inference. The only architecture that eliminates the plaintext exposure entirely is FHE-encrypted inference, where the model never sees your data and a breach exposes nothing.
The question is no longer whether AI will be used in healthcare. It will. The question is whether you build on an architecture where a single breach at your AI provider exposes every patient record you ever processed — or on one where that breach is mathematically irrelevant.