Docker AI API Deployment: Multi-Stage Builds, GPU Compose, and Nginx for Production 2026

Q: How do you deploy a FastAPI AI API with Docker in production?

Use multi-stage Dockerfile, docker-compose with Ollama GPU, Nginx reverse proxy, and uvicorn workers. [Markaicode](https://markaicode.com/tutorial/how-to-deploy-fastapi-on-docker-production/): 'Deploy FastAPI with docker compose, gunicorn uvicorn nginx. Multi-stage build, .dockerignore, production deployment guide.' [AgileSoftLabs](https://www.agilesoftlabs.com/blog/2026/02/fastapi-docker-aws-ai-production): 'Handles 10K req/min on t3.medium ECS tasks with Gunicorn Uvicorn workers. Multi-stage: FROM python:3.11-slim, install deps, COPY app, EXPOSE 8000, CMD uvicorn. Final image <150MB vs 1GB+ naive builds. Includes .dockerignore for models.' [ZestMinds](https://www.zestminds.com/blog/fastapi-deployment-guide/): 'FastAPI deployment requires more than dev server. Stable deployment combines right Python runtime, ASGI server, reverse proxy, container orchestration.' Deploy: (1) Multi-stage Dockerfile: python:3.12-slim, <150MB. (2) docker-compose: FastAPI + Ollama + Nginx. (3) uvicorn --workers 4 --host 0.0.0.0. (4) Nginx: SSL, load balancing, rate limiting. (5) GPU: device reservation in compose. (6) Health checks. (7) Volume persistence.

Q: How do you create a multi-stage Dockerfile for AI APIs?

Use builder stage for dependencies and slim runtime stage for smaller image. [AgileSoftLabs](https://www.agilesoftlabs.com/blog/2026/02/fastapi-docker-aws-ai-production): 'Multi-stage: FROM python:3.11-slim, install deps, COPY app, EXPOSE 8000, CMD uvicorn. Final image <150MB vs 1GB+ naive builds. Includes .dockerignore for models.' [Markaicode](https://markaicode.com/tutorial/how-to-deploy-fastapi-on-docker-production/): 'Multi-stage build, .dockerignore, gunicorn uvicorn nginx production deployment.' Multi-stage: (1) Builder stage: install build deps, compile requirements. (2) Runtime stage: python:3.12-slim, copy installed packages. (3) .dockerignore: exclude __pycache__, .git, models, venv. (4) Non-root user for security. (5) EXPOSE 8000. (6) CMD uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4. (7) Final image <150MB. (8) Layer caching: COPY requirements.txt before app code.

Q: How do you configure docker-compose for FastAPI and Ollama with GPU?

Use docker-compose with FastAPI, Ollama GPU, Nginx, Redis, and persistent volumes. [Markaicode Ollama Docker](https://markaicode.com/integrate/docker-with-ollama/): 'For production you rarely run just Ollama — application container (FastAPI, Node.js) talks to it. Docker Compose keeps everything orchestrated. Host-mounted volume persists models. REST API exposed on port 11434.' [Markaicode Ollama Integration](https://markaicode.com/integrate/ollama-docker/): 'Persistent volumes avoid re-downloads and improve startup time. Ephemeral containers guarantee clean state but re-download models. For latency-sensitive workloads, GPU VRAM ceiling — not CPU or RAM — is almost always the first bottleneck.' Compose: (1) FastAPI: build from Dockerfile, port 8000, depends_on Ollama. (2) Ollama: image ollama/ollama, port 11434, GPU device reservation, volume for models. (3) Nginx: image nginx, port 80/443, proxy to FastAPI. (4) Redis: image redis, port 6379, volume for persistence. (5) Networks: bridge for isolation. (6) Volumes: ollama_data, redis_data. (7) Restart: unless-stopped. (8) Health checks for all services.

Q: How do you configure Nginx as a reverse proxy for FastAPI AI APIs?

Use Nginx to proxy to FastAPI, handle SSL, rate limiting, and SSE streaming. [Markaicode](https://markaicode.com/tutorial/how-to-deploy-fastapi-on-docker-production/): 'gunicorn uvicorn nginx production deployment. Nginx reverse proxy for SSL and load balancing.' [ZestMinds](https://www.zestminds.com/blog/fastapi-deployment-guide/): 'Stable deployment combines ASGI server, reverse proxy, container orchestration.' Nginx: (1) proxy_pass http://api:8000. (2) SSL termination with certbot. (3) Rate limiting: limit_req_zone. (4) SSE: proxy_buffering off, proxy_read_timeout 300s, proxy_http_version 1.1. (5) Load balancing: upstream with multiple FastAPI workers. (6) Static files: serve directly. (7) Gzip compression. (8) Client max body size for large prompts. (9) Health check endpoint passthrough.

Q: How do you optimize Docker images for AI API deployment?

Use multi-stage builds, slim base images, .dockerignore, and layer caching. [AgileSoftLabs](https://www.agilesoftlabs.com/blog/2026/02/fastapi-docker-aws-ai-production): 'Final image <150MB vs 1GB+ naive builds. Includes .dockerignore for models. Multi-stage FROM python:3.11-slim.' [Markaicode](https://markaicode.com/integrate/ollama-docker/): 'Persistent volumes avoid re-downloads and improve startup time.' Optimization: (1) Multi-stage: builder for deps, slim runtime. (2) python:3.12-slim base (not full). (3) .dockerignore: __pycache__, .git, models, venv, tests. (4) Layer caching: COPY requirements.txt before app code. (5) pip install --no-cache-dir. (6) Non-root user. (7) Final image <150MB. (8) Persistent volumes for models — avoid re-download. (9) EXPOSE only needed ports. (10) HEALTHCHECK for orchestration.

By Fadi Updated Apr 22, 2026 13 min read AI Backend

TL;DR — Docker AI API deployment: multi-stage, GPU compose, Nginx. Markaicode: "Deploy FastAPI with docker compose, gunicorn uvicorn nginx. Multi-stage build, .dockerignore, production guide." AgileSoftLabs: "10K req/min on t3.medium. Multi-stage <150MB vs 1GB+ naive. .dockerignore for models. Gunicorn Uvicorn workers." Markaicode Ollama Docker: "Application container talks to Ollama. Docker Compose orchestrates. Host-mounted volume persists models. Port 11434." Markaicode Ollama: "Persistent volumes avoid re-downloads. GPU VRAM ceiling is first bottleneck." ZestMinds: "Stable deployment combines ASGI server, reverse proxy, container orchestration." Learn more with FastAPI AI tutorial, Ollama Docker, async backend, and job queue.

Markaicode introduces the approach: "Deploy FastAPI with docker compose, gunicorn uvicorn nginx. Multi-stage build, .dockerignore, production deployment guide for Python developers."

AgileSoftLabs adds the production benchmark: "Handles 10K req/min on t3.medium ECS tasks with Gunicorn Uvicorn workers. Multi-stage: FROM python:3.11-slim, install deps, COPY app, EXPOSE 8000, CMD uvicorn. Final image <150MB vs 1GB+ naive builds."

Docker AI API Architecture

flowchart TD subgraph Build["Docker Build"] MultiStage["Multi-Stage Dockerfile
Stage 1: Builder
install build deps
compile requirements
Stage 2: Runtime
python:3.12-slim
copy installed packages
final image <150MB"] Dockerignore[".dockerignore
exclude __pycache__
exclude .git
exclude models
exclude venv
exclude tests"] LayerCache["Layer Caching
COPY requirements.txt first
pip install --no-cache-dir
COPY app code last
cache deps across builds"] end subgraph Compose["Docker Compose Services"] API["FastAPI Container
build from Dockerfile
port 8000
uvicorn --workers 4
depends_on Ollama
OLLAMA_URL=http://ollama:11434"] Ollama["Ollama Container
image ollama/ollama
port 11434
GPU device reservation
volume ollama_data
persistent models"] Nginx["Nginx Container
image nginx
port 80, 443
proxy_pass to api:8000
SSL termination
rate limiting
SSE proxy_buffering off"] Redis["Redis Container
image redis
port 6379
volume redis_data
job queue + cache"] end subgraph Config["Production Configuration"] Workers["uvicorn Workers
--workers 4
2-4 per core
each own event loop
each own connection pool"] Health["Health Checks
FastAPI: GET /health
Ollama: GET /api/tags
Redis: redis-cli ping
interval 30s
retries 5"] Volume["Volume Persistence
ollama_data: models
redis_data: queue state
named volumes
survive container restarts"] Security["Security
non-root user
EXPOSE only needed ports
read-only filesystem
resource limits
secrets via env"] end subgraph Network["Docker Network"] Bridge["Bridge Network
container isolation
internal communication
api ↔ ollama ↔ redis
nginx → api external"] end MultiStage --> API Dockerignore --> MultiStage LayerCache --> MultiStage API --> Ollama Nginx --> API API --> Redis Workers --> API Health --> API Health --> Ollama Health --> Redis Volume --> Ollama Volume --> Redis Security --> API Bridge --> API Bridge --> Ollama Bridge --> Redis Bridge --> Nginx style Build fill:#4169E1,color:#fff style Compose fill:#39FF14,color:#000 style Config fill:#2D1B69,color:#fff

Service Configuration

Service	Image	Port	Volume	Purpose
FastAPI	Custom build	8000	—	AI API endpoints
Ollama	ollama/ollama	11434	ollama_data	LLM inference
Nginx	nginx	80, 443	—	Reverse proxy, SSL
Redis	redis	6379	redis_data	Job queue, cache

Implementation

from dataclasses import dataclass
from typing import Optional
from enum import Enum

class DeployComponent(Enum):
    DOCKERFILE = "dockerfile"
    COMPOSE = "compose"
    NGINX = "nginx"
    OPTIMIZATION = "optimization"

@dataclass
class DockerAIApiDeployGuide:
    """Docker AI API deployment implementation guide."""

    def get_dockerfile(self) -> str:
        """Multi-stage Dockerfile for FastAPI AI API."""
        return (
            "# === MULTI-STAGE DOCKERFILE ===\n"
            "\n"
            "# Stage 1: Builder\n"
            "FROM python:3.12-slim AS builder\n"
            "WORKDIR /build\n"
            "\n"
            "# Install build dependencies\n"
            "RUN apt-get update && \\\n"
            "    apt-get install -y --no-install-\n"
            "      recommends gcc && \\\n"
            "    rm -rf /var/lib/apt/lists/*\n"
            "\n"
            "# Copy requirements and install\n"
            "COPY requirements.txt .\n"
            "RUN pip install --no-cache-dir\n"
            "  --prefix=/install\n"
            "  -r requirements.txt\n"
            "\n"
            "# Stage 2: Runtime\n"
            "FROM python:3.12-slim AS runtime\n"
            "WORKDIR /app\n"
            "\n"
            "# Copy installed packages\n"
            "COPY --from=builder\n"
            "  /install /usr/local\n"
            "\n"
            "# Copy application code\n"
            "COPY . .\n"
            "\n"
            "# Non-root user for security\n"
            "RUN useradd -m appuser && \\\n"
            "    chown -R appuser:appuser /app\n"
            "USER appuser\n"
            "\n"
            "# Expose port\n"
            "EXPOSE 8000\n"
            "\n"
            "# Health check\n"
            "HEALTHCHECK --interval=30s\n"
            "  --timeout=10s --retries=5\n"
            "  CMD curl -f http://localhost:8000\n"
            "    /health || exit 1\n"
            "\n"
            "# Run with uvicorn\n"
            "CMD ['uvicorn', 'main:app',\n"
            "  '--host', '0.0.0.0',\n"
            "  '--port', '8000',\n"
            "  '--workers', '4']\n"
            "\n"
            "# Final image: <150MB\n"
            "# (vs 1GB+ naive builds)"
        )

    def get_dockerignore(self) -> str:
        """.dockerignore file."""
        return (
            "# === .dockerignore ===\n"
            "\n"
            "# Python\n"
            "__pycache__/\n"
            "*.pyc\n"
            "*.pyo\n"
            ".pytest_cache/\n"
            ".mypy_cache/\n"
            ".ruff_cache/\n"
            "\n"
            "# Virtual environments\n"
            "venv/\n"
            ".venv/\n"
            "env/\n"
            "\n"
            "# Git\n"
            ".git/\n"
            ".gitignore\n"
            "\n"
            "# Models (large files)\n"
            "models/\n"
            "*.gguf\n"
            "*.bin\n"
            "\n"
            "# Tests\n"
            "tests/\n"
            "test_*.py\n"
            "\n"
            "# Docs\n"
            "docs/\n"
            "*.md\n"
            "\n"
            "# IDE\n"
            ".vscode/\n"
            ".idea/\n"
            "\n"
            # Docker\n"
            "Dockerfile\n"
            "docker-compose*.yml\n"
            ".dockerignore\n"
            "\n"
            "# Environment\n"
            ".env\n"
            ".env.*"
        )

    def get_docker_compose(self) -> str:
        """Docker Compose for FastAPI + Ollama + Nginx."""
        return (
            "# === DOCKER COMPOSE ===\n"
            "# docker-compose.yml\n"
            "\n"
            "version: '3.8'\n"
            "\n"
            "services:\n"
            "  api:\n"
            "    build: .\n"
            "    container_name: ai-api\n"
            "    ports:\n"
            "      - '8000:8000'\n"
            "    environment:\n"
            "      - OLLAMA_URL=http://ollama:11434\n"
            "      - REDIS_URL=redis://redis:6379\n"
            "    depends_on:\n"
            "      ollama:\n"
            "        condition: service_healthy\n"
            "      redis:\n"
            "        condition: service_healthy\n"
            "    restart: unless-stopped\n"
            "    healthcheck:\n"
            "      test: ['CMD', 'curl', '-f',\n"
            "        'http://localhost:8000/health']\n"
            "      interval: 30s\n"
            "      timeout: 10s\n"
            "      retries: 5\n"
            "      start_period: 30s\n"
            "    networks:\n"
            "      - ai-network\n"
            "    deploy:\n"
            "      resources:\n"
            "        limits:\n"
            "          memory: 1G\n"
            "\n"
            "  ollama:\n"
            "    image: ollama/ollama:latest\n"
            "    container_name: ollama\n"
            "    ports:\n"
            "      - '11434:11434'\n"
            "    volumes:\n"
            "      - ollama_data:/root/.ollama\n"
            "    restart: unless-stopped\n"
            "    healthcheck:\n"
            "      test: ['CMD', 'curl', '-f',\n"
            "        'http://localhost:11434/api/tags']\n"
            "      interval: 30s\n"
            "      timeout: 10s\n"
            "      retries: 5\n"
            "      start_period: 60s\n"
            "    networks:\n"
            "      - ai-network\n"
            "    deploy:\n"
            "      resources:\n"
            "        reservations:\n"
            "          devices:\n"
            "            - capabilities: [gpu]\n"
            "\n"
            "  nginx:\n"
            "    image: nginx:alpine\n"
            "    container_name: nginx\n"
            "    ports:\n"
            "      - '80:80'\n"
            "      - '443:443'\n"
            "    volumes:\n"
            "      - ./nginx.conf:/etc/nginx/\n"
            "        conf.d/default.conf\n"
            "      - ./certs:/etc/nginx/certs\n"
            "    depends_on:\n"
            "      - api\n"
            "    restart: unless-stopped\n"
            "    networks:\n"
            "      - ai-network\n"
            "\n"
            "  redis:\n"
            "    image: redis:7-alpine\n"
            "    container_name: redis\n"
            "    ports:\n"
            "      - '6379:6379'\n"
            "    volumes:\n"
            "      - redis_data:/data\n"
            "    restart: unless-stopped\n"
            "    healthcheck:\n"
            "      test: ['CMD',\n"
            "        'redis-cli', 'ping']\n"
            "      interval: 30s\n"
            "      timeout: 10s\n"
            "      retries: 5\n"
            "    networks:\n"
            "      - ai-network\n"
            "\n"
            "volumes:\n"
            "  ollama_data:\n"
            "  redis_data:\n"
            "\n"
            "networks:\n"
            "  ai-network:\n"
            "    driver: bridge"
        )

    def get_nginx_config(self) -> str:
        """Nginx reverse proxy configuration."""
        return (
            "# === NGINX CONFIG ===\n"
            "# nginx.conf\n"
            "\n"
            "upstream fastapi_backend {\n"
            "    server api:8000;\n"
            "    # Load balancing:\n"
            "    #   server api2:8000;\n"
            "    #   server api3:8000;\n"
            "}\n"
            "\n"
            "# Rate limiting\n"
            "limit_req_zone $binary_remote_addr\n"
            "  zone=api:10m rate=10r/s;\n"
            "\n"
            "server {\n"
            "    listen 80;\n"
            "    server_name _;\n"
            "    return 301 https://$host\n"
            "      $request_uri;\n"
            "}\n"
            "\n"
            "server {\n"
            "    listen 443 ssl;\n"
            "    server_name api.example.com;\n"
            "    \n"
            "    ssl_certificate\n"
            "      /etc/nginx/certs/cert.pem;\n"
            "    ssl_certificate_key\n"
            "      /etc/nginx/certs/key.pem;\n"
            "    \n"
            "    # Client body size for\n"
            "    #   large prompts\n"
            "    client_max_body_size 10M;\n"
            "    \n"
            "    # Gzip compression\n"
            "    gzip on;\n"
            "    gzip_types\n"
            "      application/json;\n"
            "    \n"
            "    # API endpoints\n"
            "    location / {\n"
            "        limit_req zone=api burst=20\n"
            "          nodelay;\n"
            "        \n"
            "        proxy_pass\n"
            "          http://fastapi_backend;\n"
            "        proxy_set_header Host $host;\n"
            "        proxy_set_header X-Real-IP\n"
            "          $remote_addr;\n"
            "        proxy_set_header X-Forwarded-For\n"
            "          $proxy_add_x_forwarded_for;\n"
            "        proxy_set_header\n"
            "          X-Forwarded-Proto $scheme;\n"
            "    }\n"
            "    \n"
            "    # SSE streaming endpoint\n"
            "    location /v1/chat/stream {\n"
            "        proxy_pass\n"
            "          http://fastapi_backend;\n"
            "        \n"
            "        # 6 SSE directives\n"
            "        proxy_buffering off;\n"
            "        proxy_cache off;\n"
            "        proxy_read_timeout 300s;\n"
            "        proxy_set_header\n"
            "          Connection '';\n"
            "        proxy_http_version 1.1;\n"
            "        chunked_transfer_encoding off;\n"
            "    }\n"
            "    \n"
            "    # Health check\n"
            "    location /health {\n"
            "        proxy_pass\n"
            "          http://fastapi_backend;\n"
            "        access_log off;\n"
            "    }\n"
            "}"
        )

    def get_requirements(self) -> str:
        """requirements.txt for production."""
        return (
            "# === requirements.txt ===\n"
            "fastapi>=0.115.0\n"
            "uvicorn[standard]>=0.34.0\n"
            "httpx>=0.28.0\n"
            "pydantic>=2.0\n"
            "redis>=5.0\n"
            "arq>=0.26.0\n"
            "sse-starlette>=2.0\n"
            "\n"
            "# Optional for production\n"
            "# gunicorn>=23.0\n"
            "#  (for gunicorn + uvicorn\n"
            "#   workers setup)"
        )

    def get_production_tips(self) -> dict:
        """Production deployment tips."""
        return {
            "multi_stage": "Multi-stage Dockerfile: builder for deps, slim runtime. Final image <150MB vs 1GB+ naive.",
            "dockerignore": ".dockerignore: exclude __pycache__, .git, models, venv, tests. Reduces build context.",
            "layer_cache": "COPY requirements.txt before app code. Cache deps across builds. pip install --no-cache-dir.",
            "workers": "uvicorn --workers 4 (2-4 per core). Each worker own event loop and connection pool.",
            "gpu": "GPU device reservation in docker-compose deploy.resources. Ollama with GPU support.",
            "volumes": "Persistent volumes for models (ollama_data) and queue (redis_data). Avoid re-downloads.",
            "health": "Health checks for all services: FastAPI /health, Ollama /api/tags, Redis ping. interval 30s, retries 5.",
            "nginx": "Nginx reverse proxy: SSL, rate limiting, SSE proxy_buffering off, load balancing.",
            "security": "Non-root user in Dockerfile. EXPOSE only needed ports. Resource limits. Secrets via env.",
            "scaling": "10K req/min on t3.medium with Gunicorn Uvicorn workers. Scale horizontally with multiple API containers.",
        }

Docker AI API Deployment Checklist

[ ] Multi-stage Dockerfile: builder stage for dependencies, slim runtime stage for smaller image
[ ] Multi-stage: final image <150MB vs 1GB+ naive builds
[ ] Base image: python:3.12-slim (not full python image)
[ ] .dockerignore: exclude pycache, *.pyc, .git, models, venv, tests, docs, IDE files
[ ] .dockerignore: exclude Dockerfile, docker-compose*.yml, .env files
[ ] Layer caching: COPY requirements.txt before app code — cache deps across builds
[ ] pip install --no-cache-dir to reduce image size
[ ] Non-root user for security: useradd appuser, USER appuser
[ ] EXPOSE only needed ports (8000 for FastAPI)
[ ] HEALTHCHECK in Dockerfile: curl -f http://localhost:8000/health
[ ] CMD: uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4
[ ] Docker Compose: FastAPI + Ollama + Nginx + Redis services
[ ] FastAPI service: build from Dockerfile, port 8000, depends_on Ollama and Redis
[ ] FastAPI: OLLAMA_URL=http://ollama:11434, REDIS_URL=redis://redis:6379 environment variables
[ ] FastAPI: health check, restart unless-stopped, resource limits (memory 1G)
[ ] Ollama service: image ollama/ollama, port 11434, GPU device reservation
[ ] Ollama: volume ollama_data:/root/.ollama for persistent models — avoid re-downloads
[ ] Ollama: health check GET /api/tags, start_period 60s (model loading time)
[ ] Ollama: GPU VRAM ceiling is first bottleneck for latency-sensitive workloads
[ ] Nginx service: image nginx:alpine, ports 80 and 443, depends_on api
[ ] Nginx: SSL termination with certbot certificates
[ ] Nginx: rate limiting with limit_req_zone (10r/s, burst 20)
[ ] Nginx: SSE streaming — proxy_buffering off, proxy_read_timeout 300s, proxy_http_version 1.1
[ ] Nginx: client_max_body_size 10M for large prompts
[ ] Nginx: gzip compression for JSON responses
[ ] Nginx: load balancing with upstream multiple FastAPI containers
[ ] Redis service: image redis:7-alpine, port 6379, volume redis_data
[ ] Redis: health check redis-cli ping, restart unless-stopped
[ ] Docker network: bridge network for container isolation and internal communication
[ ] Volume persistence: ollama_data for models, redis_data for queue state — survive restarts
[ ] Persistent volumes avoid re-downloads and improve startup time
[ ] Ephemeral containers guarantee clean state but re-download models on every run
[ ] depends_on with condition: service_healthy for startup ordering
[ ] restart: unless-stopped for all services
[ ] Health checks: FastAPI GET /health, Ollama GET /api/tags, Redis redis-cli ping
[ ] Health check: interval 30s, timeout 10s, retries 5, start_period 30-60s
[ ] uvicorn workers: 2-4 per core, each worker own event loop and connection pool
[ ] 10K req/min on t3.medium ECS tasks with Gunicorn Uvicorn workers
[ ] Resource limits: memory limits per container to prevent OOM
[ ] Security: non-root user, EXPOSE only needed ports, read-only filesystem where possible
[ ] Security: secrets via environment variables, not hardcoded in image
[ ] requirements.txt: fastapi, uvicorn[standard], httpx, pydantic, redis, arq, sse-starlette
[ ] Optional: gunicorn for gunicorn + uvicorn workers setup
[ ] Read FastAPI AI tutorial for API setup
[ ] Read Ollama Docker for Ollama deployment
[ ] Read async backend for async patterns
[ ] Read job queue for background tasks
[ ] Test: docker-compose up builds and starts all services
[ ] Test: health checks pass for all services
[ ] Test: FastAPI can reach Ollama at http://ollama:11434
[ ] Test: Nginx proxies requests to FastAPI correctly
[ ] Test: SSE streaming works through Nginx (not buffered)
[ ] Test: volumes persist data across container restarts
[ ] Test: image size is <150MB with multi-stage build
[ ] Test: non-root user runs the application
[ ] Document Dockerfile stages, compose services, Nginx config, volume strategy, health checks

FAQ

How do you deploy a FastAPI AI API with Docker in production?

Use multi-stage Dockerfile, docker-compose with Ollama GPU, Nginx reverse proxy, and uvicorn workers. Markaicode: "Deploy FastAPI with docker compose, gunicorn uvicorn nginx. Multi-stage build, .dockerignore, production deployment guide." AgileSoftLabs: "Handles 10K req/min on t3.medium ECS tasks with Gunicorn Uvicorn workers. Multi-stage: FROM python:3.11-slim, install deps, COPY app, EXPOSE 8000, CMD uvicorn. Final image <150MB vs 1GB+ naive builds. Includes .dockerignore for models." ZestMinds: "FastAPI deployment requires more than dev server. Stable deployment combines right Python runtime, ASGI server, reverse proxy, container orchestration." Deploy: (1) Multi-stage Dockerfile: python:3.12-slim, <150MB. (2) docker-compose: FastAPI + Ollama + Nginx. (3) uvicorn --workers 4 --host 0.0.0.0. (4) Nginx: SSL, load balancing, rate limiting. (5) GPU: device reservation in compose. (6) Health checks. (7) Volume persistence.

How do you create a multi-stage Dockerfile for AI APIs?

Use builder stage for dependencies and slim runtime stage for smaller image. AgileSoftLabs: "Multi-stage: FROM python:3.11-slim, install deps, COPY app, EXPOSE 8000, CMD uvicorn. Final image <150MB vs 1GB+ naive builds. Includes .dockerignore for models." Markaicode: "Multi-stage build, .dockerignore, gunicorn uvicorn nginx production deployment." Multi-stage: (1) Builder stage: install build deps, compile requirements. (2) Runtime stage: python:3.12-slim, copy installed packages. (3) .dockerignore: exclude pycache, .git, models, venv. (4) Non-root user for security. (5) EXPOSE 8000. (6) CMD uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4. (7) Final image <150MB. (8) Layer caching: COPY requirements.txt before app code.

How do you configure docker-compose for FastAPI and Ollama with GPU?

Use docker-compose with FastAPI, Ollama GPU, Nginx, Redis, and persistent volumes. Markaicode Ollama Docker: "For production you rarely run just Ollama — application container (FastAPI, Node.js) talks to it. Docker Compose keeps everything orchestrated. Host-mounted volume persists models. REST API exposed on port 11434." Markaicode Ollama Integration: "Persistent volumes avoid re-downloads and improve startup time. Ephemeral containers guarantee clean state but re-download models. For latency-sensitive workloads, GPU VRAM ceiling — not CPU or RAM — is almost always the first bottleneck." Compose: (1) FastAPI: build from Dockerfile, port 8000, depends_on Ollama. (2) Ollama: image ollama/ollama, port 11434, GPU device reservation, volume for models. (3) Nginx: image nginx, port 80/443, proxy to FastAPI. (4) Redis: image redis, port 6379, volume for persistence. (5) Networks: bridge for isolation. (6) Volumes: ollama_data, redis_data. (7) Restart: unless-stopped. (8) Health checks for all services.

How do you configure Nginx as a reverse proxy for FastAPI AI APIs?

Use Nginx to proxy to FastAPI, handle SSL, rate limiting, and SSE streaming. Markaicode: "gunicorn uvicorn nginx production deployment. Nginx reverse proxy for SSL and load balancing." ZestMinds: "Stable deployment combines ASGI server, reverse proxy, container orchestration." Nginx: (1) proxy_pass http://api:8000. (2) SSL termination with certbot. (3) Rate limiting: limit_req_zone. (4) SSE: proxy_buffering off, proxy_read_timeout 300s, proxy_http_version 1.1. (5) Load balancing: upstream with multiple FastAPI workers. (6) Static files: serve directly. (7) Gzip compression. (8) Client max body size for large prompts. (9) Health check endpoint passthrough.

How do you optimize Docker images for AI API deployment?

Use multi-stage builds, slim base images, .dockerignore, and layer caching. AgileSoftLabs: "Final image <150MB vs 1GB+ naive builds. Includes .dockerignore for models. Multi-stage FROM python:3.11-slim." Markaicode: "Persistent volumes avoid re-downloads and improve startup time." Optimization: (1) Multi-stage: builder for deps, slim runtime. (2) python:3.12-slim base (not full). (3) .dockerignore: pycache, .git, models, venv, tests. (4) Layer caching: COPY requirements.txt before app code. (5) pip install --no-cache-dir. (6) Non-root user. (7) Final image <150MB. (8) Persistent volumes for models — avoid re-download. (9) EXPOSE only needed ports. (10) HEALTHCHECK for orchestration.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →