How to Keep Enterprise AI Data Private: A Complete Guide

TL;DR — To keep enterprise AI data private: (1) self-host the AI platform so data never leaves your infrastructure, (2) encrypt secrets at the field level with Fernet, (3) enforce tenant isolation with per-tenant graph storage, (4) implement document-level ACLs so retrieval respects source permissions, and (5) redact PII before ingestion so sensitive data never reaches a searchable index.

68% of companies using AI tools have already experienced a data leak connected to those systems (Metomic 2025). Only 23% have written AI data security policies. 34.8% of corporate data flowing into AI tools is sensitive — up from 10.7% two years ago (Cyberhaven 2026). The gap between AI adoption and AI governance is widening, and the cost is measurable: breaches involving shadow AI average $4.63 million, roughly $670,000 more than traditional breaches (IBM 2025).

The problem is architectural. AI systems leak data at four points: LLM prompts sent to external APIs, RAG indexes that store unredacted content, API responses that surface sensitive context, and agent logs that record raw interactions. Each point needs its own control. This guide covers the five controls that close all four leak points.

Why SaaS AI Cannot Guarantee Data Privacy

When you send data to a SaaS AI provider — OpenAI, Anthropic, Google — you lose control at three layers:

1. Transmission: Your data leaves your network and transits to the provider's infrastructure. You rely on TLS for transport encryption, but the provider decrypts and processes the plaintext.
2. Storage: The provider may log your prompts, store them for model training (unless you opt out), and retain them in observability pipelines. OpenAI's training-data toggle, Microsoft Copilot's tenant-isolation guarantees, and Google's enterprise data terms have all changed in the last 18 months without direct notification to customers (Strac 2026).
3. Derived data: Your data becomes embeddings, vector indexes, KV caches, and fine-tuning artifacts. These derived assets contain semantic representations of your original data. Deleting the source document does not delete the embedding.

GDPR Article 25 (privacy by design) covers AI system architecture, including the choice of AI vendor and the configuration of redaction (Strac 2026). If your AI vendor changes their privacy policy, your compliance posture changes without your consent. Self-hosting eliminates this risk entirely — your data never leaves your infrastructure.

Control 1: Self-Host the AI Platform

Self-hosting means running the AI platform — the API server, the vector database, the knowledge graph, the embedding pipeline, and the LLM gateway — on infrastructure you control. The LLM itself can still be external (OpenAI, Anthropic) if you route calls through a self-hosted gateway like LiteLLM that enforces redaction and logging before the prompt leaves your network.

The key insight: self-hosting the platform is more important than self-hosting the model. If your platform is self-hosted, you control ingestion, chunking, embedding, indexing, retrieval, and logging. The LLM is a stateless inference engine — if you redact PII before the prompt reaches it, the LLM cannot leak what it never received.

For a production self-hosted AI stack, a proven combination is FastAPI (API layer), PostgreSQL with pgvector (vector search), FalkorDB (knowledge graph), and LiteLLM (LLM gateway). This stack runs in Docker containers on a single VPS for small teams or across a Kubernetes cluster for enterprise scale.

Control 2: Encrypt Secrets at the Field Level

Encrypting the database volume protects data at rest, but it does not protect against the most common attack vector: an application bug or SQL injection that reads plaintext from the database. Field-level encryption encrypts sensitive columns individually, so even if the database is compromised, the attacker gets ciphertext.

Fernet is a symmetric encryption scheme from the cryptography library that provides authenticated encryption with a random IV and a timestamp. It is the standard for encrypting API keys, OAuth tokens, and other secrets in multi-tenant AI platforms.

from cryptography.fernet import Fernet

# Generate a per-tenant key (store in your KMS, not in the database)
tenant_key = Fernet.generate_key()
cipher = Fernet(tenant_key)

# Encrypt an API key before storing it
encrypted_key = cipher.encrypt(b"sk-openai-xxxxxxxxxxxx")
# Store encrypted_key in the database

# Decrypt only when needed (in memory, never log)
decrypted_key = cipher.decrypt(encrypted_key)

The encryption key never touches the database. Store it in a KMS (AWS KMS, HashiCorp Vault, or a hardware security module). The database stores only ciphertext. If an attacker exfiltrates the database, they get encrypted blobs without the keys to decrypt them.

For multi-tenant platforms, use envelope encryption: generate a per-tenant data encryption key (DEK), encrypt it with a master key from the KMS, and store only the encrypted DEK. This lets you rotate the master key without re-encrypting all tenant data.

Control 3: Enforce Tenant Isolation

In a multi-tenant AI platform, the highest-risk data leak is cross-tenant contamination — where Tenant A's query retrieves Tenant B's documents from a shared vector index. This happens when the retrieval layer does not filter by tenant ID.

The OWASP GenAI Data Security guide identifies the core problem: the context window aggregates distinct trust domains — system prompts, RAG data, and user inputs — into a single flat namespace without internal access control (OWASP 2026). Tenant isolation must be enforced at the storage level, not just at the API level.

Three layers of tenant isolation:

1. Database-level: Each tenant gets a separate schema or a mandatory tenant_id column on every table. Row-level security policies in PostgreSQL enforce that queries can only access rows matching the current tenant context.
2. Vector index-level: pgvector queries must include a WHERE tenant_id = ? filter. Never rely on application-level filtering alone — enforce it at the database layer with row-level security.
3. Graph-level: In a knowledge graph (FalkorDB or Neo4j), store each tenant's entities and edges in a separate graph namespace. Graph traversal queries are scoped to the tenant's graph, making cross-tenant data access structurally impossible.

-- PostgreSQL row-level security for tenant isolation
ALTER TABLE documents ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON documents
  FOR ALL
  USING (tenant_id = current_setting('app.current_tenant_id')::uuid);

With this policy, even a SELECT * FROM documents without a WHERE clause returns only the current tenant's rows. The database enforces isolation, not the application.

Control 4: Implement Document-Level Access Control Lists

Tenant isolation prevents cross-tenant leaks. Document-level ACLs prevent intra-tenant leaks — where a junior employee queries the AI and retrieves a board-level confidential memo that was ingested from a shared Drive.

The problem: RAG retrieval is similarity-based. If a confidential document is semantically similar to the query, the retriever returns it regardless of whether the user should see it. Without document-level ACLs, the AI becomes a privilege escalation tool.

The solution: tag every ingested document with access control metadata at ingestion time, and filter retrieval results by the user's permissions before they reach the LLM.

# At ingestion: tag document with ACL metadata
document = {
    "content": chunk_text,
    "embedding": embedding_vector,
    "acl_groups": ["engineering", "leadership"],
    "acl_users": ["user-123", "user-456"],
    "sensitivity": "confidential",
    "source": "google_drive/board_memo.pdf"
}

# At retrieval: filter by user's groups before sending to LLM
results = pgvector.search(
    query_embedding,
    filter={
        "tenant_id": current_tenant_id,
        "acl_groups": {"$in": current_user_groups},
    },
    top_k=10
)

The retrieval query filters by both tenant ID and the user's group memberships. Documents the user cannot access are never retrieved, never sent to the LLM, and never appear in the response. This is the same pattern that Google Drive and SharePoint use — the AI respects the source system's permissions.

Control 5: Redact PII Before Ingestion

PII that enters a RAG index compounds with every retrieval cycle (Protecto 2026). An email address in a chunk becomes part of the embedding, part of the search index, and part of every response that retrieves that chunk. Redacting PII after ingestion means re-embedding the entire corpus. Redacting before ingestion is near-free.

The ingestion pipeline should run PII detection on every document before chunking:

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

def redact_pii(text: str) -> str:
    results = analyzer.analyze(
        text=text,
        entities=["EMAIL_ADDRESS", "PHONE_NUMBER", "US_SSN", "CREDIT_CARD"],
        language="en"
    )
    redacted = anonymizer.anonymize(
        text=text,
        analyzer_results=results
    )
    return redacted.text

# In the ingestion pipeline
clean_text = redact_pii(raw_document_text)
chunks = chunker.split(clean_text)
embeddings = embed_model.encode(chunks)

Microsoft Presidio is the open-source standard for PII detection and anonymization (explainx 2026). It detects emails, phone numbers, SSNs, credit cards, and custom entities, then masks, hashes, or replaces them before the text reaches the embedding model.

The redacted text retains its semantic meaning. "Contact john@acme.com for details" becomes "Contact [EMAIL] for details." The embedding captures the meaning of the sentence without the PII. The LLM can answer questions about the document's content without ever seeing the actual email address.

How the Five Controls Work Together

The pipeline closes all four leak points:

Compliance: GDPR, HIPAA, and the EU AI Act

These five controls map directly to regulatory requirements:

• GDPR Article 5 (purpose limitation, data minimization): PII redaction enforces data minimization at the pipeline level. Only non-PII text is embedded and stored.
• GDPR Article 25 (privacy by design): Self-hosting + field-level encryption + tenant isolation = privacy by design in the architecture, not in a policy document.
• GDPR Article 35 (DPIA): A self-hosted architecture with per-tenant isolation simplifies the Data Protection Impact Assessment because data flows are internal and auditable.
• HIPAA (Business Associate Agreements): Self-hosting means no BAA with an AI vendor is needed for the platform layer. Only the LLM provider (if external) requires a BAA, and redacted prompts contain no PHI.
• EU AI Act (high-risk systems, enforceable August 2, 2026): Document-level ACLs, audit trails, and human-in-the-loop approval gates address the Act's requirements for human oversight, technical documentation, and risk management.

FAQ

Can enterprise AI be truly private?

Yes. Self-host the AI platform so data never leaves your infrastructure, encrypt secrets at the field level, enforce tenant isolation with per-tenant graph storage, implement document-level ACLs, and redact PII before ingestion.

Is self-hosted AI more secure than SaaS AI?

Yes for regulated industries. Self-hosted AI keeps all data inside your infrastructure with no third-party access. You control encryption, access, retention, and data residency. SaaS AI sends your data to a provider you cannot fully audit.

What is PII redaction in AI?

PII redaction removes personally identifiable information (emails, phone numbers, SSNs, credit cards) from text before it is chunked, embedded, or stored. The AI never sees the raw sensitive data, so it cannot leak it in responses.

Does GDPR apply to AI prompts?

Yes. GDPR Article 5 (purpose limitation and data minimization) applies to every prompt. GDPR Article 25 (privacy by design) covers AI system architecture. GDPR Article 35 requires a Data Protection Impact Assessment for systematic AI processing.

What are the main ways AI leaks enterprise data?

AI leaks data at four points: LLM prompts sent to external APIs, RAG indexes that store unredacted content, API responses that include sensitive context, and agent logs that record raw interactions. Each point needs its own control.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →