LLMs Are Plaintext Machines
Large language models do not understand encryption. They cannot. The fundamental operation of a transformer — self-attention — requires computing dot products between query, key, and value matrices derived from the input tokens. Those tokens must be numerical embeddings in plaintext. The model multiplies them, applies softmax, and produces output. Every intermediate state exists unencrypted in GPU VRAM.
When you send a prompt containing a customer's Social Security number, a patient's diagnosis, a financial transaction, or a privileged legal memorandum, that data occupies plaintext memory throughout the entire inference pass. The KV cache retains it for the duration of the conversation. Observability and logging tools may capture it. Crash dumps contain it. This is not a bug in any particular provider's implementation — it is the architectural reality of how transformer inference works.
Every token your LLM processes — input and output — exists in plaintext in GPU VRAM, attention caches, and model context. There is no encryption layer between your data and the inference infrastructure. Your prompt is the attack surface.
Five Real Privacy Risks with LLMs
The privacy exposure is not hypothetical. Each of these attack vectors has been demonstrated in published research or documented in real-world incidents.
Training Data Extraction
Adversarial prompts can extract memorized PII from training data. Research from Google Brain and DeepMind demonstrated that GPT-2 could be prompted to emit verbatim training examples — including names, phone numbers, email addresses, and IRC conversations. Carlini et al. (2021) showed that larger models memorize more training data, not less. If your data was used in fine-tuning, it can be extracted by anyone with API access.
Prompt Injection
Malicious inputs can cause the model to leak context window contents. An attacker embeds instructions in user-supplied content (a document, email, or web page the model is asked to summarize), and the model follows those instructions instead of the system prompt. The result: it outputs the contents of the system prompt, prior conversation turns, or retrieval-augmented generation (RAG) documents that the attacker should never see.
Cache and Log Exposure
The KV cache stores all prior conversation turns in plaintext for the duration of the session. Observability pipelines, API gateways, and logging infrastructure routinely capture full prompt and completion payloads. A breach of any logging system — the API gateway, the observability platform, the model provider's internal infrastructure — exposes every prompt ever sent through the pipeline.
Cross-Tenant GPU Memory Leakage
Multi-tenant inference platforms share GPU memory across customers. GPU VRAM is not zeroed between inference calls by default. Research has demonstrated that residual data from one tenant's inference pass can be read by a subsequent tenant on the same GPU. Shared infrastructure means shared risk — your data's security depends on every other tenant's threat model.
Model Output Containing PII
Even when the model behaves correctly, it may generate text that includes sensitive data from context. Ask a model to summarize a medical record, and the summary contains patient names. Ask it to draft a legal brief from case files, and the output includes privileged information. The model does not distinguish between sensitive and non-sensitive data — it reproduces whatever tokens maximize next-token probability.
“But We Use the Enterprise API”
Enterprise tiers from OpenAI, Anthropic, and Google offer Business Associate Agreements (BAAs), data processing agreements (DPAs), and contractual commitments that your data will not be used for training. These are real legal protections. They matter for compliance. They do absolutely nothing for technical security.
Your data is still processed in plaintext. The inference infrastructure still holds your prompt in GPU VRAM. The KV cache still stores your conversation history. API logs still capture request and response payloads. A contractual promise does not encrypt memory. It does not prevent a breach. It does not stop an insider threat. It does not protect against a zero-day in the infrastructure stack.
A BAA is a legal instrument. It defines liability after a breach occurs. It does not prevent the breach. Your data is still plaintext in GPU VRAM, still plaintext in logs, still plaintext in crash dumps. If the infrastructure is compromised — by an external attacker, an insider, or a misconfiguration — the contract is irrelevant. The data is exposed.
This is not a criticism of any specific provider. It is a structural limitation. No matter how strong the perimeter security, the inference engine requires plaintext access to your data in order to process it. The enterprise API changes who can see your data contractually. It does not change who can see your data technically. For AI companies building infrastructure that handles customer data, this gap is the central unsolved problem.
“But We Anonymize the Data First”
Pre-processing data to remove PII before sending it to the LLM is the most common mitigation. It is also self-defeating.
If you remove patient names from medical records, the model cannot correlate patient histories across visits. If you strip account numbers from financial transactions, the model cannot detect fraud patterns tied to specific accounts. If you redact names from legal documents, the model cannot identify conflicts of interest or track party relationships across filings. If you anonymize customer support tickets, the model cannot reference prior interactions or account context.
The data that makes the LLM useful is the same data that creates the privacy risk. Effective anonymization removes the information the model needs to produce valuable output. You are left with a choice: send useful data and accept the privacy exposure, or anonymize the data and accept degraded model performance.
Re-identification is the other failure mode. Research consistently demonstrates that anonymized datasets can be re-identified using auxiliary information. A medical record stripped of names but containing diagnosis codes, dates, and zip codes can be re-linked to individuals with surprisingly high accuracy. The LLM may perform this re-identification spontaneously, combining anonymized input with knowledge from its training data to infer identities you attempted to protect.
The FHE Solution: Encrypted Inference
Fully homomorphic encryption (FHE) eliminates the plaintext exposure entirely. With FHE, computation happens directly on encrypted data — without ever decrypting it. The inference infrastructure receives ciphertext tokens, performs attention and matrix operations on ciphertext, and returns ciphertext results. The plaintext never exists in the inference environment.
This is not tokenization. It is not masking. It is not a contractual promise. The mathematical structure of the encryption scheme (BFV or CKKS lattice-based FHE) permits specific computations — addition and multiplication of encrypted values — which are sufficient to implement transformer attention and feed-forward layers. The result is a model that produces identical outputs, but the infrastructure never observes the input, the intermediate states, or the output in plaintext.
GPU VRAM holds ciphertext. The KV cache stores ciphertext. Logs contain ciphertext. Crash dumps contain ciphertext. A breach of the inference infrastructure reveals encrypted noise — computationally indistinguishable from random data. The plaintext exists only on the client, never on the server.
H33 AI Compliance implements this as a single API wrapper around your existing AI pipeline. You encrypt prompts client-side using the H33 SDK, send encrypted payloads to your existing model endpoint (OpenAI, Anthropic, self-hosted, or any provider), and decrypt the response client-side. The model architecture does not change. The prompt structure does not change. The output quality does not change. What changes is that the infrastructure between your client and your client never sees plaintext.
What Changes, What Doesn’t
Without FHE
- Prompts in plaintext in GPU VRAM
- KV cache stores readable conversation history
- API logs capture full prompt + response text
- Crash dumps contain all session data
- Cross-tenant memory may leak between users
- Insider access exposes all customer data
- Breach = full plaintext exposure
With H33 FHE
- Prompts are ciphertext in GPU VRAM
- KV cache stores encrypted tokens only
- API logs capture encrypted payloads
- Crash dumps contain ciphertext noise
- Cross-tenant leakage reveals nothing useful
- Insider access yields encrypted data only
- Breach = computationally useless noise
Your model architecture stays the same. Your prompts stay the same. Your outputs are the same — decrypted client-side after the encrypted inference completes. The integration is a single API wrapper: encrypt before sending, decrypt after receiving. No model retraining. No prompt engineering changes. No infrastructure migration.
The encryption is lattice-based (BFV/CKKS), which means it is also post-quantum secure. A future quantum computer capable of breaking RSA and elliptic-curve cryptography cannot break the FHE encryption protecting your AI pipeline. You get privacy protection today and quantum resistance as a structural bonus.
Encrypt Your AI Pipeline
One API wrapper. Zero plaintext exposure. Same model, same prompts, same outputs — encrypted end to end.
Encrypt Your AI Pipeline →