Tenant Isolation in Multi-Tenant AI: Preventing Data Leaks
TL;DR — Tenant isolation in multi-tenant AI prevents cross-tenant data leaks across five layers: vector store, knowledge graph, conversation history, embeddings, and model outputs. An arXiv study found up to 95% of benign queries triggered cross-tenant leakage in shared RAG corpora (arXiv 2026). Two strategies: silo (separate namespace per tenant) for strong isolation, pool (shared store with metadata filtering) for cost efficiency. AI agents need tool-level isolation beyond API permissions. Per-tenant encryption keys provide defense-in-depth.
Multi-tenancy in a CRM is well-understood: filter every query by company_id, add row-level security policies, and move on. Multi-tenant AI adds three problems traditional SaaS doesn't have:
- RAG retrieval crosses tenant boundaries: A vector search for "customer churn" can return documents from another tenant if the vector store isn't properly partitioned
- AI agents act autonomously: Agents with tool access can search file repositories across tenants if the search tool isn't tenant-scoped
- Embeddings leak semantic content: Even without returning raw text, embedding similarity can expose information about another tenant's data
An arXiv preprint documented that in a multi-tenant RAG corpus with four tenants, up to 95% of benign queries triggered cross-tenant leakage — not from adversarial attacks, but from organic entity connections in the shared vector space (arXiv 2026). Security Magazine reports that 68% of organizations have had data exposed through AI agent tool access (Security Magazine 2026).
This post covers the five isolation layers, vector store strategies, AI agent isolation, and per-tenant encryption for multi-tenant AI platforms. For the broader privacy architecture, see how to keep enterprise AI data private.
The Five Isolation Layers
Multi-tenant AI requires isolation across five distinct data layers. Each layer has different isolation requirements and failure modes:
RAG document embeddings] L2[2. Knowledge Graph
Entity relationships] L3[3. Conversation History
Chat sessions] L4[4. Embedding Model
Shared vs per-tenant] L5[5. Model Outputs
Generated responses] end TenantA[Tenant A] --> L1 TenantB[Tenant B] --> L1 L1 -->|must not cross| Isolation[Isolation Boundary] Isolation --> L2 L2 --> L3 L3 --> L4 L4 --> L5
| Layer | What it stores | Isolation requirement |
|---|---|---|
| Vector store | Document embeddings for RAG retrieval | Tenant-scoped namespaces or metadata filtering |
| Knowledge graph | Entities, relationships, temporal data | Per-tenant graph or labeled partition |
| Conversation history | Chat sessions, context windows | Per-tenant storage with access controls |
| Embedding model | Shared model producing vectors | No isolation needed (model is stateless) |
| Model outputs | Generated text, citations | Tenant context enforced at generation time |
Vector Store Isolation Strategies
The vector store is the most critical isolation layer. Two main architectural approaches:
Strategy 1: Silo (separate namespace per tenant)
Each tenant gets a separate vector index or namespace. Queries are inherently tenant-scoped — a search in Tenant A's namespace cannot return Tenant B's documents.
| Approach | Isolation strength | Cost | Scalability |
|---|---|---|---|
| Separate database per tenant | Strongest | High | Poor (one DB per tenant) |
| Separate collection/namespace | Strong | Medium | Good |
| Separate partition within shared collection | Moderate | Low | Good |
When to use silo: Regulated industries (healthcare, finance), tenants with strict data residency requirements, when tenants bring their own encryption keys (BYOK).
Strategy 2: Pool (shared store with metadata filtering)
All tenants share one vector store. Every document has a tenant_id metadata field. Every query includes a tenant_id filter.
| Approach | Isolation strength | Cost | Risk |
|---|---|---|---|
| Metadata filter on every query | Moderate | Low | Filter omission = full leak |
| Row-level security (Postgres RLS) | Strong | Low | Requires database-level enforcement |
| Hybrid (pool + per-tenant partitions) | Strong | Medium | Best of both approaches |
When to use pool: Cost-sensitive deployments, tenants with similar security requirements, when the platform enforces metadata filtering at the infrastructure level (not application level).
The critical risk with pool: a single missing tenant_id filter in one code path exposes all tenants' data. The discipline check: search your codebase for client.search() or client.query() calls without tenant_id in the filter (Medium 2026).
AI Agent Isolation
AI agents introduce a new isolation challenge. Traditional web apps check permissions at the API layer. AI agents use tools to search file repositories, query databases, and call external APIs — often autonomously. If the search tool isn't limited to the tenant's workspace, the agent has admin access to everything.
| Isolation control | What it does |
|---|---|
| Tenant-scoped tool access | Every tool call includes tenant context; tools reject requests without it |
| File repository boundaries | Agent file search is limited to the tenant's workspace |
| API token scoping | Per-tenant API tokens with tenant-restricted scopes |
| Tool allowlist per tenant | Different tenants have different tool sets |
| Audit trail per action | Every agent action is logged with tenant context |
The key difference from traditional SaaS: agents act, they don't just read. A misconfigured database query leaks data passively. A misconfigured agent tool actively retrieves and processes data from other tenants (ScaleKit 2026).
Per-Tenant Encryption
Per-tenant encryption keys provide defense-in-depth. Even if a database breach occurs, one tenant's key cannot decrypt another tenant's data.
| Encryption approach | How it works | Isolation strength |
|---|---|---|
| Single key for all tenants | One encryption key for the entire database | None (breach = all tenants) |
| Per-tenant key (envelope encryption) | Each tenant has a unique data encryption key (DEK) | Strong (breach = one tenant) |
| Per-tenant key + BYOK | Tenant provides their own key | Strongest (platform cannot decrypt without tenant) |
For envelope encryption per tenant, the architecture uses a key encryption key (KEK) to wrap each tenant's DEK. The DEK encrypts the data. The KEK is stored in a KMS. This pattern is the enterprise standard for multi-tenant AI platforms.
Conversation History Isolation
Conversation history must be isolated per tenant. In a multi-tenant RAG system:
- Chat sessions are stored with tenant_id metadata
- Context windows only retrieve from the current tenant's history
- Session tokens are tenant-scoped
- Cross-tenant conversation sharing requires explicit authorization
The risk: if conversation history is stored in a shared table without tenant filtering, a user in Tenant A could see Tenant B's chat history through context retrieval.
Knowledge Graph Isolation
Knowledge graphs present a unique isolation challenge. Entities like "John Smith" or "Acme Corp" may exist in multiple tenants' graphs. The graph must enforce:
- Per-tenant graph partitions or labeled edges
- Entity resolution scoped to the tenant's data
- No cross-tenant entity merging without explicit configuration
- Query paths restricted to the tenant's subgraph
For knowledge graph security, the isolation must be at the storage level — not just the query level.
Multi-Tenant RAG Architecture
The shared LLM and embedding model are stateless — they don't store tenant data. The isolation boundary is at the vector store and conversation history layers. The model processes whatever context it receives; the platform's job is to ensure that context only comes from the requesting tenant.
Compliance and Tenant Isolation
| Regulation | Requirement | How tenant isolation helps |
|---|---|---|
| GDPR | Data protection by design (Art. 25) | Isolation is a design-level privacy control |
| HIPAA | PHI segregation | Per-tenant isolation prevents PHI cross-exposure |
| SOC 2 | Logical access controls | Tenant isolation demonstrates access control |
| EU AI Act | Risk management (Art. 9) | Isolation is a risk mitigation for data leakage |
| ISO 27001 | Information security | Tenant isolation is an access control measure |
For AI risk assessment, cross-tenant leakage is a critical risk that tenant isolation directly mitigates.
FAQ
What is tenant isolation in multi-tenant AI?
Tenant isolation in multi-tenant AI ensures that data from one tenant (organization) cannot be accessed by or leak to another tenant sharing the same AI infrastructure. It covers vector stores, knowledge graphs, conversation history, embeddings, and model outputs.
How common are cross-tenant data leaks in multi-tenant AI?
An arXiv study found that in a multi-tenant RAG corpus with four tenants, up to 95% of benign queries triggered cross-tenant leakage from organic entity connections (arXiv 2026). 68% of organizations have had data exposed through AI agent tool access (Security Magazine 2026).
What are the isolation strategies for vector databases in multi-tenant AI?
Two main strategies: silo (separate vector database or namespace per tenant) and pool (shared database with tenant_id metadata filtering). Silo provides stronger isolation but higher cost. Pool is cost-effective but requires strict metadata filtering enforcement at every query.
How do you isolate AI agents in a multi-tenant platform?
AI agents need tool-level isolation: restrict file search to the tenant's workspace, enforce tenant-scoped API tokens, and validate that every tool call includes a tenant context check. Traditional API-level permissions are insufficient because agents act autonomously.
Does tenant isolation require per-tenant encryption keys?
Not strictly required but strongly recommended. Per-tenant encryption keys (envelope encryption) ensure that even if a database breach occurs, one tenant's key cannot decrypt another tenant's data. This is the defense-in-depth approach for regulated industries.
Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →