Local LLM Model Selection: Qwen3, DeepSeek-R1, Llama 3.3, and Phi-4 Benchmark Guide

Q: Which local LLM should you choose in 2026?

Depends on use case and hardware. [AI Learning Hub](https://ailearnings.in/blog/open-source-llm-comparison/): 'Five model families matter most: Llama 3.3, Mistral, Qwen 2.5, Phi-4, Gemma 3. Use case table: General chat = Llama 3.3 70B, Code generation = Qwen2.5 Coder 32B, Math/reasoning = Phi-4 14B, Multilingual = Qwen2.5 72B, Budget 8GB = Phi-4 14B.' [LLM Hardware](https://llmhardware.io/guides/best-llms-to-run-locally): 'Best local LLMs 2026: Qwen3 8B for 8GB GPUs, Qwen3 14B for 12GB, Qwen3 32B for 24GB, Llama 3.3 70B for 48GB+. DeepSeek-R1 distill best for reasoning at every tier.' [ToolHalla](https://toolhalla.ai/blog/deepseek-vs-llama-vs-qwen-2026): 'Qwen 3.5 leads benchmarks with Apache 2.0, DeepSeek R1 dominates math (AIME 97.3), Llama 3.3 70B is safe generalist. Best single-GPU: Qwen 3-30B-A3B (16GB VRAM, ~40 tok/s).' Choose: Qwen3 for coding/multilingual, DeepSeek-R1 for reasoning/math, Llama 3.3 for general/creative, Phi-4 for best reasoning per VRAM dollar.

Q: Which LLM is best for coding, reasoning, or general use?

Coding: Qwen3. Reasoning: DeepSeek-R1. General: Llama 3.3. [AI Learning Hub](https://ailearnings.in/blog/open-source-llm-comparison/): 'Use case table: General production chat = Llama 3.3 70B. Code generation = Qwen2.5 Coder 32B. RAG pipeline = Llama 3.3 70B. Math/reasoning = Phi-4 14B. Multilingual = Qwen2.5 72B. Budget 8GB = Phi-4 14B. Instruction following = Llama 3.3 70B. Structured output JSON = Qwen2.5 32B.' [LLM Hardware](https://llmhardware.io/guides/best-llms-to-run-locally): 'Qwen3 models lead for coding in 2026 at every VRAM tier. DeepSeek-R1 distill series top choice for reasoning and math at every tier — chain-of-thought thinking dramatically improves accuracy on multi-step math, logic puzzles, analytical tasks.' [LLM Hardware Qwen3 vs DeepSeek](https://llmhardware.io/guides/qwen3-vs-deepseek-vs-llama): 'Coding: Qwen3 Excellent. Creative Writing: Llama 3.3 Best. Math/Reasoning: DeepSeek-R1 Best. Multilingual: Qwen3 Best. General Chat: Llama 3.3 Best.' Best: Qwen3 for coding, DeepSeek-R1 for reasoning, Llama 3.3 for general/creative, Phi-4 for budget reasoning.

By Fadi Updated Apr 24, 2026 13 min read Local LLMs

TL;DR — Local LLM model selection 2026. AI Learning Hub: "Five families matter: Llama 3.3, Mistral, Qwen 2.5, Phi-4, Gemma 3. General = Llama 3.3 70B, Code = Qwen2.5 Coder 32B, Math = Phi-4 14B, Multilingual = Qwen2.5 72B, Budget = Phi-4 14B. Q4_K_M quantization appropriate for production." LLM Hardware: "Best per VRAM: 4GB=Qwen3 4B, 8GB=Qwen3 8B, 12GB=Qwen3 14B, 16GB=Qwen 3-30B-A3B, 24GB=Qwen3 32B, 48GB=Llama 3.3 70B. DeepSeek-R1 best reasoning at every tier." ToolHalla: "Qwen 3.5 leads benchmarks Apache 2.0, DeepSeek R1 dominates math AIME 97.3, Llama 3.3 70B safe generalist. Best single-GPU: Qwen 3-30B-A3B 16GB ~40 tok/s." Ionio: "Q5_K_M and GPTQ-INT8 optimal trade-offs. AWQ and lower-bit GGUF use cautiously." Learn more with Ollama tutorial, VPS setup, function calling, and no framework.

AI Learning Hub frames the landscape: "I will cover the five model families that matter most for production use in early 2026: Llama 3.3, Mistral, Qwen 2.5, Phi-4, and Gemma 3. For each, I will give benchmark context, practical strengths and weaknesses, hardware requirements, and the Ollama commands to get started immediately."

LLM Hardware provides the tier guide: "The best local LLMs in 2026 are Qwen3 8B for 8 GB GPUs, Qwen3 14B for 12 GB, Qwen3 32B for 24 GB, and Llama 3.3 70B for 48 GB and above. For reasoning and math, the DeepSeek-R1 distill series is best-in-class at every tier."

Model Selection Architecture

flowchart TD subgraph UseCase["Use Case Decision"] Coding["Coding
Qwen3 / Qwen2.5 Coder
best at every VRAM tier"] Reasoning["Reasoning / Math
DeepSeek-R1 Distill
chain-of-thought
AIME 97.3"] General["General Chat
Llama 3.3 70B
creative writing
instruction following"] Budget["Budget / 8GB
Phi-4 14B
best reasoning per VRAM"] Multilingual["Multilingual
Qwen3 / Qwen2.5
best multilingual support"] end subgraph VRAM["VRAM Tiers (Q4_K_M)"] T4["4GB VRAM
Qwen3 4B
15-25 tok/s"] T8["8GB VRAM
Qwen3 8B
25-40 tok/s
top choice 2026"] T12["12GB VRAM
Qwen3 14B
18-30 tok/s"] T16["16GB VRAM
Qwen 3-30B-A3B MoE
~40 tok/s
best single-GPU"] T24["24GB VRAM
Qwen3 32B
30-45 tok/s
flagship consumer"] T48["48GB VRAM
Llama 3.3 70B
14-20 tok/s
best open-weights 70B"] end subgraph Benchmarks["Benchmark Leaders"] MMLU["MMLU
Qwen2.5 72B: 87.2
Llama 3.3 70B: 86.0
Phi-4 14B: 84.8"] HumanEval["HumanEval
Qwen3-Coder: 94.1
DeepSeek R1: 90.2
Llama 3.3: 88.4"] MATH["MATH / AIME
DeepSeek R1: 97.3
Qwen 3-30B: 95.2
Llama 3.3: 77.0"] MTBench["MT-Bench
Qwen2.5 72B: 9.2
Llama 3.3 70B: 9.1
Phi-4 14B: 8.9"] end subgraph Quant["Quantization"] Q4["Q4_K_M
4-bit, ~40% size
marginal quality loss
production-ready"] Q5["Q5_K_M
5-bit, ~50% size
better quality
optimal trade-off"] Q8["Q8_0
8-bit, ~70% size
near-lossless
best quality"] GPTQ["GPTQ-INT8
optimal trade-off
most domains
production recommended"] end Coding --> T8 Reasoning --> T8 General --> T48 Budget --> T12 Multilingual --> T16 T4 --> Q4 T8 --> Q4 T12 --> Q5 T16 --> Q4 T24 --> Q4 T48 --> Q5 style UseCase fill:#4169E1,color:#fff style VRAM fill:#39FF14,color:#000 style Benchmarks fill:#2D1B69,color:#fff style Quant fill:#FF6B6B,color:#fff

Benchmark Comparison

Model	Params	MMLU	HumanEval	MATH	AIME 2025	MT-Bench
Qwen2.5 72B	72B	87.2	86.8	83.1	—	9.2
Llama 3.3 70B	70B	86.0	88.4	77.0	77.0	9.1
Phi-4	14B	84.8	82.6	80.4	—	8.9
Qwen2.5 32B	32B	83.5	79.1	74.0	—	8.8
Mistral Small 3	24B	81.0	78.2	65.0	—	8.6
Gemma 3 27B	27B	79.5	72.0	62.0	—	8.4
DeepSeek R1	—	—	90.2	—	97.3	—
Qwen 3-30B-A3B	30B MoE	—	—	76.7	95.2	—

VRAM Tier Guide

VRAM	Best Model	Quant	Speed	Runner-up
4GB	Qwen3 4B	Q4_K_M	15-25 tok/s	DS-R1-Distill-7B
8GB	Qwen3 8B	Q4_K_M	25-40 tok/s	DS-R1-Distill-7B
12GB	Qwen3 14B	Q4_K_M	18-30 tok/s	DS-R1-Distill-14B
16GB	Qwen 3-30B-A3B	Q4_K_M	~40 tok/s	Phi-4 Q8
24GB	Qwen3 32B	Q4_K_M	30-45 tok/s	DS-R1-Distill-32B
48GB	Llama 3.3 70B	Q4_K_M	14-20 tok/s	DS-R1-Distill-70B
64GB+	Llama 3.3 70B	Q8	12-18 tok/s	Qwen3 32B Q8

Use Case Decision Table

Use Case	Recommended Model	Runner-up	VRAM (Q4)
General chat	Llama 3.3 70B	Mistral Small 24B	43GB / 14GB
Code generation	Qwen2.5 Coder 32B	Llama 3.3 70B	20GB / 43GB
RAG pipeline	Llama 3.3 70B	Qwen2.5 32B	43GB / 20GB
Math / reasoning	Phi-4 14B	Qwen2.5 72B	10GB / 43GB
Multilingual	Qwen2.5 72B	Mistral Small 24B	43GB / 14GB
Budget (8GB)	Phi-4 14B	Llama 3.1 8B	10GB / 6GB
Instruction following	Llama 3.3 70B	Qwen2.5 72B	43GB / 43GB
Structured output (JSON)	Qwen2.5 32B	Llama 3.3 70B	20GB / 43GB
Apache 2.0 license	Mistral 7B	Qwen2.5	5GB / varies

Implementation

from dataclasses import dataclass
from typing import Optional
from enum import Enum

class ModelFamily(Enum):
    QWEN = "qwen"
    DEEPSEEK = "deepseek"
    LLAMA = "llama"
    PHI = "phi"
    GEMMA = "gemma"
    MISTRAL = "mistral"

@dataclass
class LocalLLMModelSelectionGuide:
    """Local LLM model selection implementation guide."""

    def get_ollama_commands(self) -> dict:
        """Ollama pull commands per model."""
        return {
            "qwen3_8b": "ollama pull qwen3:8b",
            "qwen3_14b": "ollama pull qwen3:14b",
            "qwen3_32b": "ollama pull qwen3:32b",
            "qwen2.5_coder": "ollama pull qwen2.5-coder:32b",
            "deepseek_r1_7b": "ollama pull deepseek-r1:7b",
            "deepseek_r1_14b": "ollama pull deepseek-r1:14b",
            "deepseek_r1_32b": "ollama pull deepseek-r1:32b",
            "llama3.3_70b": "ollama pull llama3.3:70b",
            "phi4": "ollama pull phi4",
            "gemma3_27b": "ollama pull gemma3:27b",
            "mistral_small": "ollama pull mistral-small",
        }

    def get_vram_table(self) -> dict:
        """VRAM requirements per model at Q4_K_M."""
        return {
            "qwen3_4b": {"vram": "~3GB", "gpu": "RTX 3060 6GB", "speed": "15-25 tok/s"},
            "qwen3_8b": {"vram": "~6GB", "gpu": "RTX 4060 8GB", "speed": "25-40 tok/s"},
            "qwen3_14b": {"vram": "~9GB", "gpu": "RTX 4070 12GB", "speed": "18-30 tok/s"},
            "qwen_3_30b_a3b": {"vram": "~16GB", "gpu": "RTX 4080 16GB", "speed": "~40 tok/s"},
            "qwen3_32b": {"vram": "~18GB", "gpu": "RTX 4090 24GB", "speed": "30-45 tok/s"},
            "deepseek_r1_7b": {"vram": "~5.5GB", "gpu": "RTX 4060 8GB", "speed": "~55 tok/s"},
            "deepseek_r1_14b": {"vram": "~9GB", "gpu": "RTX 4070 12GB", "speed": "~35 tok/s"},
            "deepseek_r1_32b": {"vram": "~18GB", "gpu": "RTX 4090 24GB", "speed": "~20 tok/s"},
            "phi4_14b": {"vram": "~10GB", "gpu": "RTX 3080 12GB", "speed": "~30 tok/s"},
            "llama3.3_70b": {"vram": "~43GB", "gpu": "Mac M4 Pro 48GB", "speed": "14-20 tok/s"},
            "gemma3_27b": {"vram": "~14GB", "gpu": "RTX 4090 24GB", "speed": "~25 tok/s"},
            "mistral_small_24b": {"vram": "~14GB", "gpu": "RTX 4090 24GB", "speed": "~25 tok/s"},
        }

    def get_quantization_guide(self) -> dict:
        """Quantization format guide."""
        return {
            "q4_k_m": {
                "bits": 4,
                "size_ratio": "~40% of full",
                "quality": "Marginal loss",
                "recommendation": "Production-ready for most tasks",
            },
            "q5_k_m": {
                "bits": 5,
                "size_ratio": "~50% of full",
                "quality": "Better than Q4",
                "recommendation": "Optimal trade-off for most domains",
            },
            "q8_0": {
                "bits": 8,
                "size_ratio": "~70% of full",
                "quality": "Near-lossless",
                "recommendation": "Best quality, needs more VRAM",
            },
            "gptq_int8": {
                "bits": 8,
                "size_ratio": "~50% of full",
                "quality": "Good retention",
                "recommendation": "Optimal for most domains",
            },
            "awq": {
                "bits": 4,
                "size_ratio": "~40% of full",
                "quality": "May degrade",
                "recommendation": "Use cautiously",
            },
            "gguf": {
                "bits": "variable",
                "size_ratio": "varies",
                "quality": "varies",
                "recommendation": "Optimized for llama.cpp, CPU/GPU",
            },
        }

    def get_decision_logic(self) -> str:
        """Python decision logic for model selection."""
        return (
            "def select_model(\n"
            "    vram_gb: int,\n"
            "    use_case: str,\n"
            "    license_req: str = None\n"
            ") -> str:\n"
            "    '''Select best local\n"
            "    LLM for constraints.'''\n"
            "    \n"
            "    # VRAM tier mapping\n"
            "    if vram_gb < 4:\n"
            "        return 'qwen3:4b'\n"
            "    elif vram_gb < 8:\n"
            "        return 'deepseek-r1:7b'\n"
            "    elif vram_gb < 12:\n"
            "        if use_case == 'reasoning':\n"
            "            return 'deepseek-r1:7b'\n"
            "        return 'qwen3:8b'\n"
            "    elif vram_gb < 16:\n"
            "        if use_case == 'reasoning':\n"
            "            return 'deepseek-r1:14b'\n"
            "        if use_case == 'coding':\n"
            "            return 'qwen3:14b'\n"
            "        return 'phi4'\n"
            "    elif vram_gb < 24:\n"
            "        if use_case == 'reasoning':\n"
            "            return 'deepseek-r1:32b'\n"
            "        if use_case == 'coding':\n"
            "            return 'qwen3:32b'\n"
            "        return 'qwen3:32b'\n"
            "    elif vram_gb < 48:\n"
            "        return 'qwen3:32b'\n"
            "    else:\n"
            "        if use_case == 'coding':\n"
            "            return 'qwen2.5-coder:32b'\n"
            "        if use_case == 'reasoning':\n"
            "            return 'deepseek-r1:32b'\n"
            "        return 'llama3.3:70b'\n"
            "\n"
            "# Usage\n"
            "model = select_model(\n"
            "    vram_gb=24,\n"
            "    use_case='coding'\n"
            ")\n"
            "# Returns: 'qwen3:32b'\n"
            "print(f'Best model: {model}')\n"
            "print(f'Pull: ollama pull {model}')"
        )

    def get_family_comparison(self) -> dict:
        """Model family comparison."""
        return {
            "qwen": {
                "strength": "Coding, multilingual, structured output",
                "license": "Apache 2.0 (most open)",
                "best_for": "Coding agents, multilingual apps, JSON output",
                "top_model": "Qwen3 32B / Qwen 3-30B-A3B MoE",
                "weakness": "Smaller ecosystem than Llama",
            },
            "deepseek": {
                "strength": "Reasoning, math, chain-of-thought",
                "license": "MIT",
                "best_for": "Math, logic, analytical tasks, research",
                "top_model": "DeepSeek-R1-Distill-32B",
                "weakness": "Slower due to chain-of-thought, verbose output",
            },
            "llama": {
                "strength": "General chat, creative writing, ecosystem",
                "license": "Llama License (restricted)",
                "best_for": "General purpose, RAG, instruction following",
                "top_model": "Llama 3.3 70B",
                "weakness": "Requires 43GB+ VRAM, restricted license",
            },
            "phi": {
                "strength": "Best reasoning per VRAM dollar",
                "license": "MIT",
                "best_for": "Budget hardware, STEM tasks, math",
                "top_model": "Phi-4 14B",
                "weakness": "Smaller model, less versatile than 70B",
            },
            "gemma": {
                "strength": "Multimodal, Google ecosystem",
                "license": "Gemma License",
                "best_for": "Vision tasks, Google integration",
                "top_model": "Gemma 3 27B",
                "weakness": "Lower benchmarks than Qwen/Llama",
            },
            "mistral": {
                "strength": "Apache 2.0, instruction following",
                "license": "Apache 2.0",
                "best_for": "Apache 2.0 requirement, instruction tasks",
                "top_model": "Mistral Small 3 24B",
                "weakness": "Lower benchmarks than Qwen/Llama",
            },
        }

Local LLM Model Selection Checklist

[ ] Five model families matter most in 2026: Llama 3.3, Mistral, Qwen 2.5/3, Phi-4, Gemma 3
[ ] No single winner across all dimensions — match model to use case
[ ] Qwen3: best for coding at every VRAM tier, multilingual, structured output (JSON)
[ ] Qwen 3-30B-A3B MoE: best single-GPU pick (16GB VRAM, ~40 tok/s, Chatbot Arena 1384)
[ ] DeepSeek-R1: best for reasoning and math at every tier (AIME 97.3, chain-of-thought)
[ ] Llama 3.3 70B: best general purpose, creative writing, instruction following
[ ] Phi-4 14B: best reasoning per VRAM dollar (10GB VRAM, MATH 80.4)
[ ] Gemma 3 27B: multimodal, Google ecosystem
[ ] Mistral Small 3 24B: Apache 2.0 license, instruction following
[ ] Benchmarks: MMLU — Qwen2.5 72B 87.2, Llama 3.3 86.0, Phi-4 84.8
[ ] Benchmarks: HumanEval — Qwen3-Coder 94.1, DeepSeek R1 90.2, Llama 3.3 88.4
[ ] Benchmarks: AIME 2025 — DeepSeek R1 97.3, Qwen 3-30B 95.2, Llama 3.3 77.0
[ ] Benchmarks: MT-Bench — Qwen2.5 72B 9.2, Llama 3.3 9.1, Phi-4 8.9
[ ] VRAM 4GB: Qwen3 4B Q4 (15-25 tok/s)
[ ] VRAM 8GB: Qwen3 8B Q4 (25-40 tok/s, top choice 2026)
[ ] VRAM 12GB: Qwen3 14B Q4 (18-30 tok/s)
[ ] VRAM 16GB: Qwen 3-30B-A3B MoE Q4 (~40 tok/s, best single-GPU)
[ ] VRAM 24GB: Qwen3 32B Q4 (30-45 tok/s, flagship consumer)
[ ] VRAM 48GB: Llama 3.3 70B Q4 (14-20 tok/s, best open-weights 70B)
[ ] VRAM 64GB+: Llama 3.3 70B Q8 (12-18 tok/s, near-lossless)
[ ] Use case: general chat → Llama 3.3 70B, code → Qwen2.5 Coder 32B, math → Phi-4 14B
[ ] Use case: RAG → Llama 3.3 70B, multilingual → Qwen2.5 72B, budget 8GB → Phi-4 14B
[ ] Use case: instruction following → Llama 3.3 70B, JSON output → Qwen2.5 32B
[ ] Use case: Apache 2.0 required → Mistral 7B or Qwen2.5
[ ] Quantization: Q4_K_M — 4-bit, ~40% size, marginal quality loss, production-ready
[ ] Quantization: Q5_K_M — 5-bit, ~50% size, optimal trade-off for most domains
[ ] Quantization: Q8_0 — 8-bit, ~70% size, near-lossless, best quality
[ ] Quantization: GPTQ-INT8 — optimal trade-off for most domains
[ ] Quantization: AWQ — use cautiously, may degrade quality
[ ] Quantization: GGUF — optimized for llama.cpp, CPU/GPU inference, edge devices
[ ] Ionio: Q5_K_M and GPTQ-INT8 offer optimal trade-offs, AWQ use cautiously
[ ] AI Learning Hub: Q4_K_M or Q5_K_M appropriate for production, marginal quality difference
[ ] Llama 3.3 70B Q4: ~43GB VRAM — dual RTX 3090, RTX 6000 Ada, or Mac M3/M4 Max/Ultra 64GB+
[ ] All models have GGUF versions, run via Ollama with one command
[ ] All models support 128K context length
[ ] Ollama pull commands: ollama pull qwen3:8b, ollama pull deepseek-r1:7b, ollama pull llama3.3:70b
[ ] License: Qwen Apache 2.0 (most open), DeepSeek MIT, Llama restricted, Phi MIT, Mistral Apache 2.0
[ ] Read Ollama tutorial for setup
[ ] Read VPS setup for cloud deployment
[ ] Read function calling for tool use
[ ] Read no framework for agent integration
[ ] Test: selected model runs at expected speed on your hardware
[ ] Test: Q4_K_M quantization produces acceptable quality for your use case
[ ] Test: model handles your context length requirements
[ ] Test: coding model produces correct code for your language
[ ] Test: reasoning model shows chain-of-thought for math problems
[ ] Test: JSON structured output works with your pipeline
[ ] Document model choice, VRAM, quantization, use case, benchmark scores

FAQ

Which local LLM should you choose in 2026?

Depends on use case and hardware. AI Learning Hub: "Five model families matter most: Llama 3.3, Mistral, Qwen 2.5, Phi-4, Gemma 3. Use case table: General chat = Llama 3.3 70B, Code generation = Qwen2.5 Coder 32B, Math/reasoning = Phi-4 14B, Multilingual = Qwen2.5 72B, Budget 8GB = Phi-4 14B." LLM Hardware: "Best local LLMs 2026: Qwen3 8B for 8GB GPUs, Qwen3 14B for 12GB, Qwen3 32B for 24GB, Llama 3.3 70B for 48GB+. DeepSeek-R1 distill best for reasoning at every tier." ToolHalla: "Qwen 3.5 leads benchmarks with Apache 2.0, DeepSeek R1 dominates math (AIME 97.3), Llama 3.3 70B is safe generalist. Best single-GPU: Qwen 3-30B-A3B (16GB VRAM, ~40 tok/s)." Choose: Qwen3 for coding/multilingual, DeepSeek-R1 for reasoning/math, Llama 3.3 for general/creative, Phi-4 for best reasoning per VRAM dollar.

How do benchmarks compare across local LLMs?

Qwen2.5 72B leads MMLU and MATH, Phi-4 punches above weight, Llama 3.3 strong on HumanEval. AI Learning Hub: "MMLU: Qwen2.5 72B 87.2, Llama 3.3 70B 86.0, Phi-4 14B 84.8, Mistral Small 24B 81.0. HumanEval: Llama 3.3 70B 88.4, Qwen2.5 72B 86.8, Phi-4 14B 82.6. MATH: Qwen2.5 72B 83.1, Phi-4 14B 80.4, Llama 3.3 70B 77.0. MT-Bench: Qwen2.5 72B 9.2, Llama 3.3 70B 9.1, Phi-4 14B 8.9." LLM Hardware: "Math/Reasoning: DeepSeek-R1-Distill Best, Qwen3 Excellent. Coding: Qwen3 Excellent, DeepSeek Very Good. Creative Writing: Llama 3.3 Best. Multilingual: Qwen3 Best. All 128K context." ToolHalla: "AIME 2025: DeepSeek R1 97.3, Qwen 3-30B-A3B 95.2, Llama 3.3 70B 77.0. HumanEval: Qwen3-Coder-Next 94.1, DeepSeek R1 90.2, Llama 3.3 88.4." Benchmarks: Qwen leads coding and multilingual, DeepSeek dominates math, Llama 3.3 best creative/general, Phi-4 best reasoning per VRAM.

What VRAM do you need for each local LLM?

4GB for 4B models, 8GB for 8B, 16GB for 30B MoE, 24GB for 32B, 48GB for 70B. LLM Hardware: "4GB: Qwen3 4B Q4 (15-25 tok/s). 8GB: Qwen3 8B Q4 (25-40 tok/s, top choice 2026). 12GB: Qwen3 14B Q4 (18-30 tok/s). 16GB: Qwen 3-30B-A3B MoE (~40 tok/s, best single-GPU). 24GB: Qwen3 32B Q4 (30-45 tok/s, flagship consumer). 48GB: Llama 3.3 70B Q4 (14-20 tok/s). 64GB+: Llama 3.3 70B Q8 (12-18 tok/s, near-lossless)." LLM Hardware Qwen3 vs DeepSeek: "Qwen3 8B Q4: ~6GB, RTX 4060. Qwen3 14B Q4: ~9GB, RTX 4070. Qwen3 32B Q4: ~18GB, RTX 4090. Llama 3.3 70B Q4: ~43GB, Mac mini M4 Pro 48GB. All Q4_K_M quantization." ToolHalla: "Qwen 3-30B-A3B: 16GB VRAM, ~40 tok/s. Llama 3.3 70B: 38GB min, 48GB recommended. DS-R1-Distill-7B: 4GB min, 6GB recommended."

Should you use quantized models in production?

Yes — Q4_K_M or Q5_K_M are appropriate for most production workloads with marginal quality loss. AI Learning Hub: "Q4_K_M or Q5_K_M quantized models are appropriate for most production workloads. The quality difference from full precision is marginal for the majority of tasks and the memory savings are significant." Ionio: "Benchmarked Qwen2.5, DeepSeek, Mistral, LLaMA 3.3 across MMLU, GSM8K, BBH, C-Eval, IFEval. Q5_K_M and GPTQ-INT8 offer optimal trade-offs for most domains. AWQ and lower-bit GGUF formats should be used cautiously. GGUF Q4_K_M, Q5_K_M optimized for CPU/GPU inference using llama.cpp — ideal for edge or low-power scenarios." Quantization: (1) Q4_K_M: 4-bit, ~40% of full size, marginal quality loss. (2) Q5_K_M: 5-bit, ~50% of full size, better quality. (3) Q8_0: 8-bit, ~70% of full size, near-lossless. (4) GPTQ-INT8: optimal trade-off for most domains. (5) AWQ: use cautiously, may degrade quality. (6) GGUF: optimized for llama.cpp, CPU/GPU inference.

Which LLM is best for coding, reasoning, or general use?

Coding: Qwen3. Reasoning: DeepSeek-R1. General: Llama 3.3. AI Learning Hub: "Use case table: General production chat = Llama 3.3 70B. Code generation = Qwen2.5 Coder 32B. RAG pipeline = Llama 3.3 70B. Math/reasoning = Phi-4 14B. Multilingual = Qwen2.5 72B. Budget 8GB = Phi-4 14B. Instruction following = Llama 3.3 70B. Structured output JSON = Qwen2.5 32B." LLM Hardware: "Qwen3 models lead for coding in 2026 at every VRAM tier. DeepSeek-R1 distill series top choice for reasoning and math at every tier — chain-of-thought thinking dramatically improves accuracy on multi-step math, logic puzzles, analytical tasks." LLM Hardware Qwen3 vs DeepSeek: "Coding: Qwen3 Excellent. Creative Writing: Llama 3.3 Best. Math/Reasoning: DeepSeek-R1 Best. Multilingual: Qwen3 Best. General Chat: Llama 3.3 Best." Best: Qwen3 for coding, DeepSeek-R1 for reasoning, Llama 3.3 for general/creative, Phi-4 for budget reasoning.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →