Ollama VPS Setup: Deploy Local LLMs on Cloud Servers with GPU and Systemd

By Fadi Updated Apr 27, 2026 13 min read Local LLMs

TL;DR — Ollama VPS setup: deploy local LLMs on cloud servers. VPS.org: "One-line installer auto-detects architecture, installs binary and systemd service. Configure remote access with OLLAMA_HOST=0.0.0.0:11434. Set OLLAMA_ORIGINS for CORS control." MarkAICode: "Deploy on AWS EC2 with GPU, systemd, Nginx, TLS. Not just installing a binary — weaving into resilient, secure stack that survives reboots." RamNode: "CPU-optimized VPS deployment. Modern CPUs run smaller models with proper optimization." CodingMantra: "Ollama + Gemma 4 on VPS. Flagship AI on consumer hardware. Toolchain handles inference, GPU acceleration, API management." Medium: "Self-hosted Ollama on remote VM from local PC. Model-agnostic workflow for VS Code, Claude." Learn more with Ollama tutorial, no framework, debugging, and function calling.

MarkAICode frames the challenge: "Deploying Ollama in production is not just about installing a binary — it's about weaving it into a resilient, secure stack that survives reboots."

VPS.org describes the setup: "Install Ollama on Ubuntu VPS using the official installation script from ollama.ai. The one-line installer automatically detects system architecture and installs the binary and systemd service."

VPS Deployment Architecture

flowchart TD subgraph VPS["Cloud VPS"] Systemd["Systemd Service
auto-configured
runs as ollama user
survives reboots"] Ollama["Ollama Server
localhost:11434
or 0.0.0.0:11434"] Models["Models
/root/.ollama
or /home/ollama/.ollama"] Env["Environment
OLLAMA_HOST
OLLAMA_ORIGINS
OLLAMA_MAX_LOADED_MODELS"] end subgraph Security["Security Layer"] Firewall["UFW Firewall
allow 11434 from known IPs
or restrict entirely"] SSH["SSH Tunnel
ssh -L 11434:localhost:11434
no port exposure"] Nginx["Nginx Reverse Proxy
terminate TLS
add auth headers
rate limiting"] TLS["TLS Certificate
Let's Encrypt
or self-signed"] Fail2ban["Fail2ban
protect SSH
brute force defense"] end subgraph Hardware["VPS Hardware"] CPU["CPU VPS
8GB: 3B models
16GB: 7B models
32GB: 14B models"] GPU["GPU VPS
NVIDIA T4 16GB
NVIDIA A10G 24GB
CUDA acceleration"] Disk["Disk
20GB+ for models
SSD recommended
fast downloads"] end subgraph Access["Client Access"] Direct["Direct (dev only)
http://vps-ip:11434
not recommended"] Tunnel["SSH Tunnel (secure)
localhost:11434
via SSH forwarding"] Proxy["Nginx + TLS (prod)
https://llm.example.com
authenticated"] end subgraph Providers["VPS Providers"] AWS["AWS EC2
GPU instances
g4dn, g5 series"] DO["DigitalOcean
CPU droplets
GPU droplets"] Linode["Linode/Akamai
CPU and GPU
NVIDIA RTX"] RamNode["RamNode
CPU-optimized
budget option"] end Systemd --> Ollama Ollama --> Models Systemd --> Env Ollama --> Security Firewall --> Ollama SSH --> Ollama Nginx --> Ollama TLS --> Nginx Fail2ban --> Security Ollama --> Hardware CPU --> Ollama GPU --> Ollama Disk --> Models Security --> Access Direct --> Ollama Tunnel --> SSH Proxy --> Nginx Providers --> Hardware style VPS fill:#4169E1,color:#fff style Security fill:#FF6B6B,color:#fff style Hardware fill:#39FF14,color:#000

VPS Provider Comparison

Provider	GPU Available	Min Price	Best For
AWS EC2	Yes (g4dn, g5)	~$0.50/hr GPU	Production, scalable
DigitalOcean	Yes (GPU droplets)	~$0.40/hr GPU	Simple, developer-friendly
Linode/Akamai	Yes (NVIDIA RTX)	~$0.50/hr GPU	Mid-range, good value
RamNode	No (CPU only)	~$10/mo	Budget, CPU models

Implementation

from dataclasses import dataclass
from typing import Optional
from enum import Enum

class VPSType(Enum):
    CPU = "cpu"
    GPU = "gpu"

@dataclass
class OllamaVPSSetupGuide:
    """Ollama VPS setup implementation guide."""

    def get_install_steps(self) -> str:
        """Install Ollama on VPS."""
        return (
            "# === SSH INTO VPS ===\n"
            "$ ssh root@your-vps-ip\n"
            "\n"
            "# === UPDATE SYSTEM ===\n"
            "$ apt update && apt upgrade -y\n"
            "\n"
            "# === INSTALL OLLAMA ===\n"
            "# One-line installer:\n"
            "# auto-detects architecture\n"
            "# installs binary + systemd\n"
            "$ curl -fsSL \\\n"
            "    https://ollama.com/\n"
            "    install.sh | sh\n"
            "\n"
            "# === VERIFY INSTALL ===\n"
            "$ ollama --version\n"
            "$ systemctl status ollama\n"
            "# Active: active (running)\n"
            "\n"
            "# === PULL MODELS ===\n"
            "$ ollama pull llama3.2\n"
            "$ ollama pull qwen2.5-coder\n"
            "\n"
            "# === TEST LOCALLY ===\n"
            "$ curl localhost:11434/api/chat \\\n"
            "    -d '{\"model\":\"llama3.2\",\n"
            "    \"messages\":[{\"role\":\n"
            "    \"user\",\"content\":\"Hi\"}]}'"
        )

    def get_systemd_config(self) -> str:
        """Systemd service configuration."""
        return (
            "# === DEFAULT SERVICE ===\n"
            "# Auto-created at:\n"
            "# /etc/systemd/system/\n"
            "#   ollama.service\n"
            "# Runs as user: ollama\n"
            "\n"
            "# === CUSTOM OVERRIDE ===\n"
            "$ sudo systemctl edit ollama\n"
            "\n"
            "# Add environment vars:\n"
            "[Service]\n"
            "Environment=\n"
            "    OLLAMA_HOST=0.0.0.0:11434\n"
            "Environment=\n"
            "    OLLAMA_ORIGINS=*\n"
            "    .example.com,https://\n"
            "    app.example.com\n"
            "Environment=\n"
            "    OLLAMA_MAX_LOADED_MODELS=2\n"
            "Environment=\n"
            "    OLLAMA_KEEP_ALIVE=10m\n"
            "\n"
            "# === RELOAD AND RESTART ===\n"
            "$ sudo systemctl daemon-reload\n"
            "$ sudo systemctl restart ollama\n"
            "\n"
            "# === ENABLE ON BOOT ===\n"
            "$ sudo systemctl enable ollama\n"
            "\n"
            "# === CHECK LOGS ===\n"
            "$ journalctl -u ollama -f\n"
            "\n"
            "# === ENVIRONMENT VARS ===\n"
            "# OLLAMA_HOST: listen addr\n"
            "#   0.0.0.0:11434 = all\n"
            "#   127.0.0.1:11434 = local\n"
            "# OLLAMA_ORIGINS: CORS\n"
            "#   comma-separated origins\n"
            "# OLLAMA_MAX_LOADED_MODELS:\n"
            "#   concurrent models in mem\n"
            "# OLLAMA_KEEP_ALIVE:\n"
            "#   how long to keep model\n"
            "#   loaded after last use"
        )

    def get_firewall_security(self) -> str:
        """Firewall and security setup."""
        return (
            "# === UFW FIREWALL ===\n"
            "$ ufw default deny incoming\n"
            "$ ufw default allow outgoing\n"
            "$ ufw allow ssh\n"
            "\n"
            "# Option A: Restrict to\n"
            "# known IPs only\n"
            "$ ufw allow from \\\n"
            "    203.0.113.50 to any \\\n"
            "    port 11434\n"
            "\n"
            "# Option B: Deny 11434\n"
            "# entirely (use SSH tunnel)\n"
            "$ ufw deny 11434\n"
            "\n"
            "# Option C: Allow all\n"
            "# (NOT recommended for prod)\n"
            "$ ufw allow 11434\n"
            "\n"
            "$ ufw enable\n"
            "$ ufw status verbose\n"
            "\n"
            "# === SSH TUNNEL ===\n"
            "# Secure access without\n"
            "# exposing port 11434\n"
            "# On your LOCAL machine:\n"
            "$ ssh -L 11434:localhost:\n"
            "    11434 user@vps-ip\n"
            "\n"
            "# Now localhost:11434 on\n"
            "# your machine routes to\n"
            "# VPS Ollama server\n"
            "# Use OpenAI SDK with\n"
            "# base_url=\n"
            "#   http://localhost:11434/v1\n"
            "\n"
            "# === FAIL2BAN ===\n"
            "$ apt install fail2ban\n"
            "$ systemctl enable fail2ban\n"
            "$ systemctl start fail2ban\n"
            "# Protects against SSH\n"
            "# brute force attacks\n"
            "\n"
            "# === OLLAMA_ORIGINS ===\n"
            "# Restrict CORS to your\n"
            "# domains only\n"
            "# In systemd override:\n"
            "# Environment=OLLAMA_ORIGINS=\n"
            "#   https://app.example.com"
        )

    def get_nginx_reverse_proxy(self) -> str:
        """Nginx reverse proxy with TLS."""
        return (
            "# === INSTALL NGINX ===\n"
            "$ apt install nginx\n"
            "\n"
            "# === NGINX CONFIG ===\n"
            "# /etc/nginx/sites-available/\n"
            "#   ollama\n"
            "server {\n"
            "    listen 80;\n"
            "    server_name llm.\n"
            "        example.com;\n"
            "    return 301 https://\n"
            "        $host$request_uri;\n"
            "}\n"
            "\n"
            "server {\n"
            "    listen 443 ssl;\n"
            "    server_name llm.\n"
            "        example.com;\n"
            "\n"
            "    ssl_certificate /etc/\n"
            "        letsencrypt/live/\n"
            "        llm.example.com/\n"
            "        fullchain.pem;\n"
            "    ssl_certificate_key \\\n"
            "        /etc/letsencrypt/\n"
            "        live/llm.example.com/\n"
            "        privkey.pem;\n"
            "\n"
            "    location / {\n"
            "        proxy_pass http://\n"
            "            localhost:11434;\n"
            "        proxy_set_header \\\n"
            "            Host $host;\n"
            "        proxy_set_header \\\n"
            "            X-Real-IP \\\n"
            "            $remote_addr;\n"
            "\n"
            "        # Streaming support\n"
            "        proxy_buffering off;\n"
            "        proxy_read_timeout \\\n"
            "            300s;\n"
            "\n"
            "        # Rate limiting\n"
            "        limit_req zone=ollama\n"
            "            burst=10 nodelay;\n"
            "    }\n"
            "}\n"
            "\n"
            "# === ENABLE SITE ===\n"
            "$ ln -s /etc/nginx/sites-\n"
            "    available/ollama \\\n"
            "    /etc/nginx/sites-enabled/\n"
            "$ nginx -t\n"
            "$ systemctl reload nginx\n"
            "\n"
            "# === TLS WITH LETSENCRYPT ===\n"
            "$ apt install certbot \\\n"
            "    python3-certbot-nginx\n"
            "$ certbot --nginx -d \\\n"
            "    llm.example.com\n"
            "\n"
            "# === RATE LIMITING ===\n"
            "# In nginx.conf http block:\n"
            "# limit_req_zone $binary_\n"
            "#   remote_addr zone=ollama:\n"
            "#   10m rate=10r/s\n"
            "\n"
            "# Now access via:\n"
            "# https://llm.example.com/v1\n"
            "# OpenAI SDK:\n"
            "# base_url=https://llm.\n"
            "#   example.com/v1"
        )

    def get_gpu_setup(self) -> str:
        """GPU VPS setup."""
        return (
            "# === NVIDIA DRIVER ===\n"
            "$ apt install nvidia-driver-\n"
            "    535\n"
            "$ reboot\n"
            "\n"
            "# === VERIFY GPU ===\n"
            "$ nvidia-smi\n"
            "# Should show GPU info\n"
            "\n"
            "# === CUDA TOOLKIT ===\n"
            "$ apt install nvidia-cuda-\n"
            "    toolkit\n"
            "\n"
            "# === OLLAMA AUTO-DETECTS GPU ===\n"
            "# No additional config\n"
            "# Ollama detects NVIDIA\n"
            "# GPU and uses CUDA\n"
            "# automatically\n"
            "\n"
            "# === VERIFY GPU USAGE ===\n"
            "$ ollama ps\n"
            "# Shows model loaded on GPU\n"
            "\n"
            "# === GPU VPS PROVIDERS ===\n"
            "# AWS EC2: g4dn.xlarge\n"
            "#   (NVIDIA T4 16GB)\n"
            "# AWS EC2: g5.xlarge\n"
            "#   (NVIDIA A10G 24GB)\n"
            "# DigitalOcean: GPU droplet\n"
            "# Linode: NVIDIA RTX 4000\n"
            "\n"
            "# === DOCKER WITH GPU ===\n"
            "docker run -d \\\n"
            "  --gpus all \\\n"
            "  -v ollama:/root/.ollama \\\n"
            "  -p 11434:11434 \\\n"
            "  ollama/ollama"
        )

    def get_production_tips(self) -> dict:
        """Production deployment tips."""
        return {
            "systemd": (
                "Auto-configured by "
                "installer. Enable on "
                "boot: systemctl enable "
                "ollama. Survives reboots."
            ),
            "logging": (
                "journalctl -u ollama -f "
                "for real-time logs. "
                "Configure log rotation "
                "for long-running servers."
            ),
            "monitoring": (
                "Monitor GPU usage with "
                "nvidia-smi. Monitor "
                "memory, CPU, disk. "
                "Set up alerts for "
                "downtime."
            ),
            "backups": (
                "Backup models directory "
                "(/root/.ollama or "
                "/home/ollama/.ollama). "
                "Avoid re-downloading "
                "multi-GB models."
            ),
            "updates": (
                "Update Ollama: curl -fsSL "
                "https://ollama.com/install.sh "
                "| sh. Check version: "
                "ollama --version."
            ),
            "concurrent": (
                "OLLAMA_MAX_LOADED_MODELS "
                "controls concurrent models "
                "in memory. Set based on "
                "available RAM/VRAM."
            ),
            "keep_alive": (
                "OLLAMA_KEEP_ALIVE: how "
                "long model stays loaded "
                "after last use. Default "
                "5m. Increase for frequent "
                "use, decrease to save "
                "memory."
            ),
        }

Ollama VPS Setup Checklist

[ ] Install Ollama on VPS: curl -fsSL https://ollama.com/install.sh | sh
[ ] One-line installer auto-detects architecture, installs binary and systemd service
[ ] Systemd service auto-configured at /etc/systemd/system/ollama.service
[ ] Systemd runs as dedicated ollama user (not root)
[ ] Enable on boot: sudo systemctl enable ollama
[ ] Start: sudo systemctl start ollama
[ ] Status: sudo systemctl status ollama
[ ] Logs: journalctl -u ollama -f
[ ] Override: sudo systemctl edit ollama for custom environment variables
[ ] OLLAMA_HOST=0.0.0.0:11434 — listen on all interfaces for remote access
[ ] OLLAMA_HOST=127.0.0.1:11434 — local only (default, more secure)
[ ] OLLAMA_ORIGINS — control CORS, restrict to your domains
[ ] OLLAMA_MAX_LOADED_MODELS — concurrent models in memory
[ ] OLLAMA_KEEP_ALIVE — how long model stays loaded after last use (default 5m)
[ ] Pull models: ollama pull llama3.2, ollama pull qwen2.5-coder
[ ] Test locally: curl localhost:11434/api/chat with JSON body
[ ] Firewall: ufw default deny incoming, ufw allow ssh
[ ] Firewall option A: ufw allow from YOUR_IP to any port 11434 (restrict to known IPs)
[ ] Firewall option B: ufw deny 11434 (use SSH tunnel instead)
[ ] Firewall option C: ufw allow 11434 (NOT recommended for production)
[ ] SSH tunnel: ssh -L 11434:localhost:11434 user@vps — secure access without port exposure
[ ] SSH tunnel: localhost:11434 on your machine routes to VPS Ollama server
[ ] Nginx reverse proxy: terminate TLS, add authentication headers, rate limiting
[ ] Nginx: proxy_pass http://localhost:11434, proxy_buffering off for streaming
[ ] Nginx: proxy_read_timeout 300s for long-running inference
[ ] TLS: Let's Encrypt with certbot --nginx -d llm.example.com
[ ] Rate limiting: limit_req_zone in nginx.conf, burst=10 nodelay in location
[ ] Fail2ban: protect against SSH brute force attacks
[ ] Never expose port 11434 directly to internet without authentication
[ ] GPU VPS: install nvidia-driver-535, verify with nvidia-smi
[ ] GPU: Ollama auto-detects NVIDIA GPU and uses CUDA — no additional config
[ ] GPU: verify with ollama ps showing model loaded on GPU
[ ] GPU Docker: docker run --gpus all for GPU acceleration in containers
[ ] GPU VPS providers: AWS EC2 (g4dn T4, g5 A10G), DigitalOcean GPU, Linode RTX
[ ] CPU VPS: 8GB RAM for 3B models, 16GB for 7B, 32GB for 14B
[ ] CPU VPS providers: RamNode (budget CPU), DigitalOcean, Linode
[ ] Disk: 20GB+ for models, SSD recommended for faster I/O
[ ] Network: 1Gbps+ for fast model downloads (models are 2-10GB)
[ ] Backup models directory to avoid re-downloading multi-GB models
[ ] Update Ollama: re-run install script, check ollama --version
[ ] Monitor: nvidia-smi for GPU, standard monitoring for CPU/memory/disk
[ ] Production: Nginx + TLS + firewall + fail2ban + systemd = resilient stack
[ ] Read Ollama tutorial for local setup
[ ] Read no framework for agent integration
[ ] Read debugging for tracing on VPS
[ ] Read function calling for tool use
[ ] Test: Ollama installs and starts via systemd on VPS
[ ] Test: OLLAMA_HOST=0.0.0.0 allows remote access from known IP
[ ] Test: SSH tunnel routes localhost:11434 to VPS Ollama
[ ] Test: Nginx reverse proxy serves HTTPS with valid TLS certificate
[ ] Test: GPU detected and used by Ollama (nvidia-smi shows activity)
[ ] Test: firewall blocks unauthorized access to port 11434
[ ] Test: systemd service survives reboot (auto-starts on boot)
[ ] Test: fail2ban blocks repeated failed SSH attempts
[ ] Document VPS provider, specs, OLLAMA_HOST, firewall rules, Nginx config, TLS setup

FAQ

How do you install Ollama on a VPS?

Use the one-line installer, configure systemd service, and set OLLAMA_HOST for remote access. VPS.org: "Install Ollama on Ubuntu VPS using the official installation script from ollama.ai. The one-line installer automatically detects system architecture and installs the binary and systemd service. Configure Ollama for remote access by setting OLLAMA_HOST environment variable to 0.0.0.0:11434." MarkAICode: "Deploy Ollama production server on AWS EC2 with GPU, systemd, Nginx, and TLS. Secure, scalable local LLM inference with step-by-step code and benchmarks. Deploying Ollama in production is not just about installing a binary — it is about weaving it into a resilient, secure stack that survives reboots." RamNode: "Deploying Ollama on RamNode VPS infrastructure, optimized for CPU performance. Modern CPUs can effectively run smaller language models with proper optimization." Install: (1) curl -fsSL https://ollama.com/install.sh | sh. (2) Systemd service auto-configured. (3) Set OLLAMA_HOST=0.0.0.0:11434 for remote access. (4) sudo systemctl enable ollama. (5) sudo systemctl start ollama.

How do you configure Ollama for remote access?

Set OLLAMA_HOST=0.0.0.0:11434 and configure firewall to allow port 11434. VPS.org: "Configure Ollama for remote access by setting OLLAMA_HOST environment variable to 0.0.0.0:11434. Set OLLAMA_ORIGINS to control which origins can access the API." Medium: "Self-hosted Ollama model on a remote VM from your local PC. Model-agnostic workflow for using Ollama models hosted on any VM, VPS, or private server from VS Code chat, Claude." MarkAICode: "Secure, scalable local LLM inference with Nginx and TLS." Remote access: (1) OLLAMA_HOST=0.0.0.0:11434 — listen on all interfaces. (2) OLLAMA_ORIGINS — control allowed origins for CORS. (3) Firewall: ufw allow 11434 or restrict to specific IPs. (4) SSH tunnel: ssh -L 11434:localhost:11434 user@vps for secure access without exposing port. (5) Nginx reverse proxy with TLS for production. (6) Never expose port 11434 directly to internet without auth.

What VPS specs do you need for Ollama?

CPU VPS: 8GB+ RAM for 7B models, 16GB+ for 14B. GPU VPS: NVIDIA T4/A10G for acceleration. RamNode: "Deploying Ollama on VPS infrastructure, optimized for CPU performance. Modern CPUs can effectively run smaller language models with proper optimization." MarkAICode: "Deploy Ollama production server on AWS EC2 with GPU, systemd, Nginx, and TLS. Secure, scalable local LLM inference." CodingMantra: "Ollama + Gemma 4 on VPS. Google March 2026 release brought flagship-level AI to models small enough to run on consumer hardware. Ollama toolchain handles model inference, GPU acceleration, and API management." Specs: (1) CPU VPS 8GB RAM: 3B models (llama3.2:3b). (2) CPU VPS 16GB RAM: 7-8B models (llama3.2, mistral, qwen2.5). (3) CPU VPS 32GB RAM: 14B models (phi4). (4) GPU VPS (NVIDIA T4 16GB): 7-13B models with acceleration. (5) GPU VPS (NVIDIA A10G 24GB): 14-33B models. (6) Disk: 20GB+ for models. (7) Network: 1Gbps+ for fast model downloads.

How do you secure Ollama on a VPS?

Use firewall, SSH tunnel, Nginx reverse proxy with TLS, and restrict origins. MarkAICode: "Deploy Ollama production server with systemd, Nginx, and TLS. Secure, scalable local LLM inference. Weaving it into a resilient, secure stack that survives reboots." VPS.org: "Set OLLAMA_ORIGINS to control which origins can access the API." Security: (1) Firewall: ufw allow from YOUR_IP to any port 11434 — restrict to known IPs. (2) SSH tunnel: ssh -L 11434:localhost:11434 user@vps — no port exposure. (3) Nginx reverse proxy: terminate TLS, add authentication headers. (4) OLLAMA_ORIGINS: restrict CORS to your domains. (5) Never expose port 11434 directly to internet. (6) Systemd: run as dedicated ollama user, not root. (7) Fail2ban: protect against brute force on SSH. (8) Updates: keep Ollama updated with ollama --version check.

How do you set up Ollama with systemd on a VPS?

The installer auto-configures systemd; customize with override for environment variables. VPS.org: "The one-line installer automatically detects system architecture and installs the binary and systemd service." RamNode: "Create service override directory for custom configuration." MarkAICode: "Systemd service that survives reboots. Resilient, secure stack." Systemd: (1) Auto-configured by installer: /etc/systemd/system/ollama.service. (2) Override: sudo systemctl edit ollama for custom env vars. (3) Environment: OLLAMA_HOST=0.0.0.0:11434, OLLAMA_ORIGINS=*.example.com. (4) Enable: sudo systemctl enable ollama (starts on boot). (5) Start: sudo systemctl start ollama. (6) Status: sudo systemctl status ollama. (7) Logs: journalctl -u ollama -f. (8) Restart: sudo systemctl restart ollama. (9) Runs as ollama user by default.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →