Docker Compose AI Platform: Self-Hosted Stack in One File

TL;DR — A Docker Compose AI platform deploys the entire self-hosted AI stack with one command: docker compose up. The stack includes Ollama (LLM inference), PostgreSQL with pgvector (vector + SQL + ACLs), FalkorDB (knowledge graph), FastAPI (RAG API), Prometheus + Grafana (monitoring), and Nango (source connectors). GPU passthrough via NVIDIA Container Toolkit. Health checks ensure ordered startup. Named volumes for data persistence. Production-ready with restart policies, resource limits, and logging. Suitable for up to ~500 users on a single server. For larger deployments, migrate to Kubernetes. The minimum viable stack is Ollama + PostgreSQL + FastAPI — add services as needed. One file, one command, full self-hosted AI platform.

In 2026, self-hosted AI has gone mainstream. What used to require a GPU cluster and a PhD now runs on a single server with Docker Compose. One file defines the entire stack: LLM inference, vector database, knowledge graph, RAG API, monitoring, and source connectors.

Docker Compose solves three problems: reproducibility (same environment everywhere), orchestration (services start in order with health checks), and portability (move from laptop to server to cloud). This post provides the complete docker-compose.yml for a production-grade enterprise AI platform.

The Complete Docker Compose AI Stack

version: "3.9"

services:
  # ============================================
  # LLM Inference Layer
  # ============================================
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    restart: always
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
        limits:
          memory: 16G
    healthcheck:
      test: ["CMD-SHELL", "curl -sf http://localhost:11434/api/tags || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 3

  # ============================================
  # Data Layer: PostgreSQL + pgvector
  # ============================================
  postgres:
    image: pgvector/pgvector:pg16
    container_name: postgres
    ports:
      - "5432:5432"
    environment:
      POSTGRES_DB: aidb
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_INITDB_ARGS: "--encoding=UTF-8"
    volumes:
      - pg_data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    restart: always
    deploy:
      resources:
        limits:
          memory: 8G
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d aidb"]
      interval: 10s
      timeout: 5s
      retries: 5

  # ============================================
  # Knowledge Graph: FalkorDB
  # ============================================
  falkordb:
    image: falkordb/falkordb:latest
    container_name: falkordb
    ports:
      - "6379:6379"
    volumes:
      - falkor_data:/data
    restart: always
    command: ["--save", "60", "1", "--loglevel", "warning"]
    deploy:
      resources:
        limits:
          memory: 4G
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 3

  # ============================================
  # RAG API: FastAPI
  # ============================================
  api:
    build: .
    container_name: rag-api
    ports:
      - "8000:8000"
    environment:
      DATABASE_URL: postgresql+asyncpg://postgres:${DB_PASSWORD}@postgres/aidb
      FALKORDB_URL: redis://falkordb:6379
      OLLAMA_URL: http://ollama:11434
      EMBEDDING_MODEL: BAAI/bge-large-en-v1.5
      RERANKER_MODEL: BAAI/bge-reranker-v2-m3
      CACHE_THRESHOLD: "0.95"
      CONFIDENCE_THRESHOLD: "0.1"
    depends_on:
      postgres:
        condition: service_healthy
      falkordb:
        condition: service_healthy
      ollama:
        condition: service_healthy
    restart: always
    deploy:
      resources:
        limits:
          memory: 4G
    healthcheck:
      test: ["CMD", "curl", "-sf", "http://localhost:8000/health"]
      interval: 30s
      timeout: 5s
      retries: 3

  # ============================================
  # Source Connectors: Nango
  # ============================================
  nango:
    image: nangohq/nango:latest
    container_name: nango
    ports:
      - "3003:3003"
    environment:
      NANGO_ENCRYPTION_KEY: ${NANGO_ENCRYPTION_KEY}
      SERVER_PORT: "3003"
    volumes:
      - nango_data:/var/lib/nango
    restart: always

  # ============================================
  # Monitoring: Prometheus + Grafana
  # ============================================
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus_data:/prometheus
    restart: always

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    ports:
      - "3000:3000"
    environment:
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_PASSWORD}
    volumes:
      - grafana_data:/var/lib/grafana
    depends_on:
      - prometheus
    restart: always

  # ============================================
  # Reverse Proxy: Nginx (production TLS)
  # ============================================
  nginx:
    image: nginx:alpine
    container_name: nginx
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./certs:/etc/nginx/certs:ro
    depends_on:
      - api
    restart: always
    profiles: ["production"]

volumes:
  ollama_data:
  pg_data:
  falkor_data:
  nango_data:
  prometheus_data:
  grafana_data:

Service Architecture

Service Overview

Total resource footprint: ~35GB RAM + 1 GPU. Fits on a single server with 64GB RAM and 1 NVIDIA GPU.

Production Patterns

GPU Passthrough

# Requires NVIDIA Container Toolkit on host
# Install: distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
#          curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | sudo apt-key add -
#          sudo apt-get install -y nvidia-container-toolkit
#          sudo systemctl restart docker

ollama:
  deploy:
    resources:
      reservations:
        devices:
          - driver: nvidia
            count: 1              # or device_ids: ["0", "1"] for specific GPUs
            capabilities: [gpu]

Health Check Ordering

api:
  depends_on:
    postgres:
      condition: service_healthy   # Wait for PostgreSQL to be ready
    falkordb:
      condition: service_healthy   # Wait for FalkorDB to be ready
    ollama:
      condition: service_healthy   # Wait for Ollama to load model

Docker Compose Profiles

# Development profile (no Nginx, no TLS)
# Usage: docker compose up
# Nginx only starts with: docker compose --profile production up

nginx:
  profiles: ["production"]

# Monitoring only starts with: docker compose --profile monitoring up
prometheus:
  profiles: ["monitoring"]
grafana:
  profiles: ["monitoring"]

Resource Limits

# Prevent any single service from consuming all resources
deploy:
  resources:
    limits:
      memory: 16G
      cpus: '4'
    reservations:
      memory: 8G
      cpus: '2'

Dev vs Production Configuration

Scaling Beyond Docker Compose

Migration path: Docker Compose → Docker Swarm → Kubernetes. The docker-compose.yml converts to Docker Stack files (docker stack deploy) and Kubernetes manifests (using Kompose or Helm charts).

Docker Compose AI Platform Checklist

• [ ] Install Docker Engine + Docker Compose v2 on host
• [ ] Install NVIDIA driver + CUDA + NVIDIA Container Toolkit (for GPU)
• [ ] Create .env file with DB_PASSWORD, NANGO_ENCRYPTION_KEY, GRAFANA_PASSWORD
• [ ] Create init.sql for PostgreSQL (pgvector extension, tables, indexes, RLS)
• [ ] Create prometheus.yml with scrape targets for all services
• [ ] Create nginx.conf with TLS + rate limiting (production profile)
• [ ] Build FastAPI Dockerfile (Python 3.12, asyncpg, redis-py, sentence-transformers)
• [ ] Pull LLM model: docker exec ollama ollama pull llama3:70b
• [ ] Configure GPU passthrough in Ollama service
• [ ] Set health checks for all services with appropriate intervals
• [ ] Configure depends_on with condition: service_healthy for ordered startup
• [ ] Set resource limits (memory + CPU) per service
• [ ] Set restart: always for production
• [ ] Configure logging with max-size and max-file rotation
• [ ] Use named volumes for all persistent data
• [ ] Test with docker compose up (dev) and docker compose --profile production up
• [ ] Verify health checks pass: docker compose ps
• [ ] Set up hybrid retrieval in PostgreSQL
• [ ] Configure cross-encoder reranking in FastAPI
• [ ] Set up semantic answer caching (Redis or pgvector)
• [ ] Implement content hash deduplication in ingestion
• [ ] Configure document-level ACLs in PostgreSQL
• [ ] Set up Nango connectors for data sources
• [ ] Configure Prometheus alerts: latency p99, error rate, GPU utilization
• [ ] Create Grafana dashboards: query latency, cache hit rate, hallucination rate
• [ ] Set up automated backups: PostgreSQL PITR + FalkorDB RDB + volume snapshots
• [ ] Test disaster recovery: restore from backup
• [ ] Configure zero-trust security between services
• [ ] Monitor disk usage — LLM models and vectors consume significant space
• [ ] Consider self-hosted AI TCO for cost analysis
• [ ] Document the stack for AI company brain integration

FAQ

What is a Docker Compose AI platform?

A Docker Compose AI platform is a single docker-compose.yml file that defines and orchestrates all components of a self-hosted AI stack: LLM inference (Ollama or vLLM), vector database (PostgreSQL with pgvector), knowledge graph (FalkorDB), RAG API (FastAPI), monitoring (Prometheus + Grafana), and source connectors (Nango). Docker Compose handles networking, volume persistence, health checks, and service dependencies. One command (docker compose up) starts the entire platform. This simplifies deployment, ensures reproducibility, and makes the stack portable across environments.

How do you deploy an AI stack with Docker Compose?

Create a docker-compose.yml file defining each service (Ollama, PostgreSQL, FalkorDB, FastAPI, Prometheus), their ports, volumes, environment variables, and dependencies. Use health checks to ensure services start in order. Configure GPU passthrough with the NVIDIA Container Toolkit for LLM inference. Use named volumes for data persistence. Run 'docker compose up -d' to start all services. For production, add resource limits, restart policies, and logging configuration. The entire stack can be deployed on a single server or split across multiple machines using Docker Swarm or Kubernetes.

Can Docker Compose handle production AI workloads?

Docker Compose is suitable for small to medium production deployments (up to ~500 users). For larger deployments, migrate to Kubernetes. Docker Compose production considerations: use restart: always, set resource limits (memory, CPU), configure health checks, use named volumes for persistence, enable GPU passthrough for LLM inference, set up logging (ELK or Loki), and use Docker Compose profiles to separate dev and prod configurations. For high availability, use Docker Swarm (multi-node) or migrate to Kubernetes. Many teams run production AI on Docker Compose with a single powerful server.

How do you configure GPU passthrough in Docker Compose?

Install the NVIDIA Container Toolkit on the host, then specify GPU resources in docker-compose.yml under deploy.resources.reservations.devices. For a single GPU: 'driver: nvidia, count: 1, capabilities: [gpu]'. For specific GPUs: 'device_ids: [\"0\", \"1\"]'. Ollama and vLLM automatically detect and use available GPUs. Ensure the NVIDIA driver and CUDA are installed on the host. Test with 'docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi' before deploying the full stack.

What services should a Docker Compose AI platform include?

Core services: Ollama or vLLM (LLM inference), PostgreSQL with pgvector (vector + SQL + ACLs), FalkorDB (knowledge graph), FastAPI (RAG API). Supporting services: Prometheus + Grafana (monitoring), Nango (source connectors), BGE-Reranker (cross-encoder reranking). Optional: OpenSearch (if you need better full-text search than PostgreSQL tsvector), Redis (semantic cache), Nginx (reverse proxy + TLS). The minimum viable stack is Ollama + PostgreSQL + FastAPI. Add services as your use case demands.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →