When should I use LoRA vs QLoRA for fine-tuning?

The choice between LoRA and QLoRA is primarily a hardware decision. Both methods produce the same quality adapters — the difference is how much VRAM you need to train them. Use LoRA when: (1) You have 24GB+ VRAM — A100 40GB, RTX 6000, RTX 3090 (24GB). LoRA needs 14-16 GB for a 7B model. (2) You want maximum training speed — LoRA is 2-3x faster than QLoRA. No dequantization overhead. ~100 steps/min vs ~60 steps/min on A100. (3) You're fine-tuning models up to 30B parameters — LoRA handles 13-30B on a single high-end GPU. (4) You need deterministic, reproducible training — no quantization artifacts or paged optimizer quirks. (5) You're running production pipelines — standard transformers library, no bitsandbytes dependency. (6) You're doing research or benchmarking — maximum training throughput, no quantization variables. (7) You're already running full-precision models — simplest adapter setup with standard optimizers. (8) You can justify the hardware cost — A100 40GB rental is more expensive than consumer GPUs. Use QLoRA when: (1) You're on a consumer GPU — RTX 4090 (24GB), RTX 4080 (16GB), RTX 3090 (24GB), M2 Pro. QLoRA needs 6-8 GB for a 7B model. (2) You have less than 24GB VRAM — QLoRA is the only viable fine-tuning method. (3) You're fine-tuning large models (70B+) — QLoRA fits a 70B model on a single 80GB A100. LoRA would require multi-GPU. (4) You're in a resource-constrained environment — laptops, edge devices, cheap cloud instances. (5) You're cost-sensitive — $5 for 500 examples on rented consumer GPU vs $50+ on A100. (6) You want to fine-tune models that won't fit in full precision — 4-bit quantization enables fine-tuning models that would otherwise require multi-GPU setups. (7) You're a hobbyist, student, or small team — QLoRA democratized LLM fine-tuning. Anyone with a consumer GPU can fine-tune. (8) You're prototyping — start with QLoRA on a cheap GPU. If quality is insufficient, upgrade to LoRA on a better GPU. The decision framework: (1) Do you have 24GB+ VRAM? → Yes: use LoRA (faster, simpler). No: use QLoRA (only option). (2) Are you fine-tuning a 70B+ model? → Use QLoRA (fits on single GPU). (3) Is training speed critical? → Use LoRA (2-3x faster). (4) Is cost the primary constraint? → Use QLoRA (cheaper hardware). (5) Are you in production with deterministic requirements? → Use LoRA (no quantization artifacts). (6) Are you prototyping or learning? → Use QLoRA (accessible, cheap). What about quality? Both LoRA and QLoRA produce adapters within 0.1-1% of full fine-tuning quality. The quality difference between LoRA and QLoRA is unmeasurable (<0.1%) when using identical rank/alpha. The QLoRA paper demonstrated that a 65B model fine-tuned with QLoRA matched full fine-tuning quality on instruction-following benchmarks. For most tasks, quality should not be the deciding factor. Choose based on hardware, speed, and cost. What about rank? Rank affects both LoRA and QLoRA equally. Higher rank = more adapter capacity = more VRAM. A 7B model with LoRA at rank 8 uses ~16GB. At rank 64, it uses ~32GB — pushing users toward QLoRA unexpectedly. If you need high rank and have limited VRAM, QLoRA is the solution. The key: 'Use LoRA for production fine-tuning when you have adequate GPU memory: it trains 2-3x faster than QLoRA with identical final model quality. Use QLoRA if you're constrained by hardware or need to fine-tune 70B models on a single A100.' The choice is about training hardware, not deployed model quality (theneuralbase 2026, mljourney 2026, metricgate 2026).

What is NF4 quantization and how does it work in QLoRA?

NF4 (NormalFloat4) is a 4-bit quantization format designed specifically for quantizing pretrained neural network weights. It is one of the three key innovations that make QLoRA work. The problem NF4 solves: standard 4-bit quantization (like INT4) maps values uniformly across the range. But pretrained weights are not uniformly distributed — they follow a roughly Gaussian (normal) distribution, with most values clustered around zero and fewer in the tails. Uniform quantization wastes representational capacity on the tails (where few values fall) and has coarse resolution around zero (where most values fall). How NF4 works: (1) Normal distribution matching — NF4 maps its 16 representable 4-bit values to the quantiles of a standard normal distribution. This means the 16 values are spaced according to where weight values actually fall. (2) More resolution around zero — because most weights are near zero, NF4 allocates more of its 16 values around zero and fewer in the tails. This minimizes discretization error exactly where most weight values fall. (3) Minimized error — by matching the quantization grid to the empirical distribution of weights, NF4 minimizes the average quantization error compared to uniform INT4 quantization. (4) 16 representable values — 4 bits can represent 16 distinct values. NF4 chooses those 16 values to be the optimal quantile points of a normal distribution. The math: for a standard normal distribution N(0,1), the 16 quantile points divide the distribution into 16 equal-probability regions. Each 4-bit value maps to the center of its region. When a weight is quantized, it is mapped to the nearest quantile point. The result: NF4 achieves lower quantization error than INT4 for normally distributed weights, which is the case for virtually all pretrained LLM weights. Double quantization: QLoRA adds a second innovation on top of NF4. The quantization process requires per-block scale and offset constants (stored in fp32). These constants themselves take memory. Double quantization quantizes these constants from fp32 to fp8, saving ~0.37 bits per parameter. The total memory per parameter: 4 bits (NF4 weight) + 8/64 bits (fp8 scale) + 32/(64x256) bits (double quant scale) = ~4.127 bits/param. For a 7B model: 4.127 bits x 7B = ~3.6 GB. Compare to fp16: 16 bits x 7B = 14 GB. That's a 3.9x reduction. Paged optimizers: the third QLoRA innovation. During training, peak memory spikes can cause OOM errors — especially with long sequences. Paged optimizers use NVIDIA unified memory to automatically offload optimizer states to CPU RAM when GPU memory is full, then page them back when needed. This prevents OOM crashes during memory spikes. It's a safety net, not a performance feature — paging to CPU is slow, but it prevents crashes. All three are enabled by default in bitsandbytes and are transparent to the user. The training process: (1) Load model in 4-bit NF4 — BitsAndBytesConfig with load_in_4bit=True, bnb_4bit_quant_type='nf4', bnb_4bit_compute_dtype=torch.bfloat16. (2) During forward pass — 4-bit weights are dequantized to bf16 on the fly. Computation happens in bf16. Dequantized weights are discarded after computation. (3) During backward pass — gradients are computed only for the LoRA adapters (A and B matrices). The frozen 4-bit weights never receive gradients. (4) Adapters train in bf16 — the LoRA adapter matrices stay in bfloat16 for stable training. The 4-bit quantization only applies to the frozen base. (5) Paged optimizers — if GPU memory fills up, optimizer states are paged to CPU RAM automatically. The key: 'NF4 allocates more of its 16 representable values around zero and fewer in the tails, minimizing discretization error exactly where most weight values fall.' NF4 + double quantization + paged optimizers = 4-bit fine-tuning with minimal quality loss. The QLoRA paper proved this works at scale: a 65B model fine-tuned with QLoRA on a single 48GB GPU matched full fine-tuning quality (metricgate 2026, mljourney 2026, teqvolt 2026, dev 2026).

How do I choose the right LoRA rank and alpha?

Rank (r) and alpha are the two most important hyperparameters for both LoRA and QLoRA. Getting them right is essential for good fine-tuning results. Rank (r): controls adapter capacity. It's the width of the bottleneck — how much the adapter is allowed to learn. Higher rank captures more complex patterns but uses more memory and raises overfitting risk on small datasets. The math: for a weight matrix W of size d x d, LoRA adds two matrices A (r x d) and B (d x r). Instead of updating d^2 parameters, you update 2dr. For d=4096 (common hidden dimension) and r=8: 65,536 parameters instead of 16,777,216 — a 256x reduction. Rank guidelines: (1) r=4 — simple tasks: format adaptation, style transfer, small datasets. Minimal capacity. (2) r=8 — solid default for instruction-tuning. Covers most of the benefit. Good starting point. (3) r=16 — good balance for most tasks. Recommended starting point in 2026. Handles domain adaptation well. (4) r=32 — complex tasks requiring more capacity. Real-world domain fine-tunes. (5) r=64 — when r=32 is still underfitting. Rare. Approaches cost of updating more parameters directly. (6) r=128+ — complex, data-rich tasks that need broad new behavior. Rarely needed. (7) r=256 — will not automatically improve results. Higher rank means more trainable parameters, more memory, and higher overfitting risk. Diminishing returns. The strategy: start at r=8 or r=16. Train, evaluate. If quality is insufficient, increase to r=32, then r=64. Only go beyond r=64 if you see clear under-fitting. Rank rarely needs to exceed 128. Memory impact of rank: rank dramatically affects VRAM. A 7B model with LoRA at rank 8 uses ~16GB. At rank 64, it uses ~32GB — pushing users toward QLoRA unexpectedly. If you need high rank and have limited VRAM, use QLoRA. Alpha: a scaling factor. The adapter's output is multiplied by alpha/r before being added to the frozen weights. Alpha controls how loudly the adapter speaks relative to the base model. Two conventions: (1) Microsoft's convention — alpha = 2 x rank. The most common recommendation. The effective scale is alpha/r = 2.0, regardless of rank. This makes learning rate behave predictably across rank sweeps. (2) QLoRA paper convention — alpha at 50% or 25% of rank. Reported strong results. The practical read: because the effective scale is alpha/r, what actually matters is the ratio, not the raw numbers. Pick a convention, hold it steady, and tune rank first — then adjust alpha only if the adapter is under- or over-shooting. Why alpha = 2 x rank: the adapter's contribution is scaled by alpha divided by rank during training. When alpha is double the rank, that ratio is always 2.0, regardless of whether you choose rank 8 or rank 64. Your learning rate behaves predictably. This is why every tutorial says 'set alpha to 2 x rank.' Target modules: which weight matrices to apply LoRA to. (1) Minimum — q_proj and v_proj (attention query and value projections). The default in HuggingFace PEFT. (2) Better — all four attention projections: q_proj, k_proj, v_proj, o_proj. (3) Best — all linear layers: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj. Use target_modules='all-linear' in PEFT. Empirically, applying LoRA to all linear layers consistently outperforms applying it only to attention projections. Architecture-specific gotchas: (1) Llama — q_proj, v_proj works. (2) Mistral — needs q_proj, k_proj, v_proj, o_proj. Using only q_proj, v_proj fails silently — no error raised, training just happens on wrong layers. (3) Qwen — different naming convention. (4) Non-transformer architectures — may need different target modules. Always use target_modules='all-linear' for supported architectures to avoid these issues. Learning rate: (1) LoRA — 1e-4 to 3e-4 (5-10x higher than full fine-tuning). Adapter weights start at zero, so they need bigger steps. The frozen base model cannot be harmed. (2) QLoRA — slightly lower than LoRA. 1e-4 to 2e-4. (3) Full fine-tuning — 1e-5 to 2e-5. Much lower because all parameters are being updated. The key: 'Start at rank 8 for instruction-tuning, 16-32 for domain adaptation, 64+ only if you see clear under-fitting. Set alpha = 2 x rank. Use target_modules=all-linear. Tune rank first, then adjust alpha only if needed.' The recommended starting configuration: r=16, alpha=32, target_modules='all-linear', learning_rate=2e-4, epochs=3 (metricgate 2026, mljourney 2026, teqvolt 2026, dev 2026, theneuralbase 2026).

Do LoRA and QLoRA have inference latency?

No — neither LoRA nor QLoRA adds inference latency. This is the quiet killer feature that makes them production-viable. How it works: (1) During training — LoRA adds adapter matrices (A and B) alongside the frozen base model. During training, the forward pass computes W + BA, where W is the frozen base and BA is the adapter contribution. This adds a small amount of compute during training. (2) During inference — the adapter can be merged into the base model. The adapter matrices A and B are multiplied together and added directly to the frozen weights: W_merged = W + BA. The merged model has the same architecture as the original — no additional matrices, no additional computation. (3) No inference overhead — because the adapter is merged into the weights, inference speed is identical to the base model. There is zero latency penalty. The model doesn't know it was fine-tuned with LoRA — it's just a standard model with slightly different weights. (4) QLoRA inference — QLoRA's 4-bit quantization is a training-time optimization. For inference, you can: (a) Merge the adapter into the base model and dequantize to fp16/bf16 — standard inference, no overhead. (b) Keep the base in 4-bit and apply the adapter — actually faster inference (4-bit is faster than fp16) with slightly lower quality. (c) Use vLLM with adapter hot-swap — load the base model once, swap adapters per request. Slight overhead for adapter loading but negligible. (5) Adapter hot-swap — vLLM supports serving multiple LoRA adapters from the same base model. Different users or tasks can use different adapters without loading separate models. The adapter swap takes milliseconds — negligible for production. This is a major advantage of LoRA over full fine-tuning: one base model + N adapters instead of N full models. The math: for a weight matrix W of size d x d, the LoRA adapter adds A (r x d) and B (d x r). During training, the forward pass computes Wx + BAx, adding 2dr operations per layer. For d=4096 and r=8: 65,536 additional operations per layer — negligible compared to the 16,777,216 operations for the base matrix. During inference (after merging): W_merged = W + BA. The merged matrix is the same size as the original. Zero additional operations. Comparison with full fine-tuning: (1) Full fine-tuning — produces a full model copy (14 GB for 7B). One model per task. No adapter overhead but massive storage and memory cost. (2) LoRA/QLoRA — produces a 50 MB adapter. One base model + N adapters. Zero inference overhead. Negligible storage. (3) Multi-task serving — LoRA: one base model + 10 adapters = 14 GB + 500 MB. Full fine-tuning: 10 full models = 140 GB. LoRA is 10x more efficient for multi-task serving. The key: 'LoRA can reduce the number of trainable parameters by 10,000x and the GPU memory requirement by 3x versus full fine-tuning — while matching or beating full fine-tuning quality — with no added inference latency.' The no-inference-latency property is what makes LoRA/QLoRA the default for production fine-tuning in 2026. You get the quality benefits of fine-tuning without any deployment penalty (teqvolt 2026, theneuralbase 2026, mljourney 2026, dev 2026).

Can I use QLoRA to fine-tune a 70B model on a single GPU?

Yes — QLoRA was specifically designed to enable fine-tuning large models on limited hardware. The QLoRA paper (Dettmers et al., 2023) demonstrated fine-tuning a 65B parameter model on a single 48GB GPU, matching full fine-tuning quality on instruction-following benchmarks. This was previously impossible. How it works: (1) A 70B model in fp16 requires ~140 GB of VRAM just to load the weights. Full fine-tuning requires ~600+ GB (weights + gradients + optimizer states). This needs 8+ high-end GPUs. (2) With QLoRA's 4-bit NF4 quantization, the 70B base model shrinks to ~35 GB (4.127 bits/param x 70B). This fits on a single 48GB GPU (A100 48GB, A6000 48GB). (3) The LoRA adapters train in bf16 on top of the 4-bit base. Adapter memory is negligible (~50-100 MB for rank 16). (4) Paged optimizers handle memory spikes by offloading optimizer states to CPU RAM. (5) Total VRAM for 70B QLoRA: ~35-48 GB. Fits on a single 48GB GPU. GPU requirements by model size (QLoRA): (1) 7B model — 6-8 GB VRAM. RTX 4090, RTX 3090, Colab T4. (2) 13B model — 10-14 GB VRAM. RTX 4090, A4000. (3) 30B model — 18-24 GB VRAM. RTX 4090, A5000. (4) 70B model — 35-48 GB VRAM. A100 48GB, A6000 48GB. (5) 70B model (with paged optimizers) — fits on 48GB GPU with optimizer states paged to CPU. What you need: (1) A 48GB+ GPU — A100 48GB, A6000 48GB, or L40S 48GB. (2) bitsandbytes >= 0.41.0 — for NF4 quantization and double quantization support. (3) PEFT library — for LoRA adapter configuration. (4) TRL or Unsloth — for training. (5) ~16-32 GB system RAM — for paged optimizer offloading. (6) Training time — 70B QLoRA is slow. Expect 10-20x longer than 7B QLoRA. Budget hours to days depending on dataset size. Cost: (1) Rent an A100 80GB on cloud — ~$2-4/hour. Fine-tune 70B with QLoRA for $20-100 depending on dataset size. (2) Compare to full fine-tuning 70B — requires 8x A100 80GB = ~$16-32/hour. $160-1600 for the same training run. (3) QLoRA is 8-10x cheaper for 70B fine-tuning. Quality: the QLoRA paper demonstrated that a 65B model fine-tuned with QLoRA matched full fine-tuning quality on instruction-following benchmarks. The quality gap is <1% on most tasks. For most production use cases, QLoRA quality is sufficient. Limitations: (1) Training speed — 70B QLoRA is slow. Dequantization overhead on every forward pass. Expect 30-50% of LoRA throughput. (2) Batch size — with 48GB VRAM, batch size is limited to 1-2. Use gradient accumulation (8-16) for effective batch 8-32. (3) Sequence length — long sequences increase activation memory. Use gradient checkpointing to trade compute for memory. (4) Rank — higher rank increases adapter memory. r=8 or r=16 is recommended for 70B to stay within VRAM. (5) Paged optimizer latency — when optimizer states are paged to CPU, training slows down. This is a safety net, not a performance feature. The key: 'QLoRA makes it feasible to fine-tune a 70B model on a single 80GB A100. The original paper demonstrated that a 65B model fine-tuned with QLoRA on a single 48GB GPU matched the quality of full fine-tuning on instruction following benchmarks.' QLoRA democratized large-model fine-tuning. What used to require a GPU cluster can now be done on a single GPU (mljourney 2026, theneuralbase 2026, metricgate 2026, teqvolt 2026).

July 13, 2026

TL;DR — LoRA vs QLoRA in 2026: LoRA trains 0.1% of params, 14-16GB VRAM for 7B, 2-3x faster than QLoRA. QLoRA: 4-bit NF4 quantization, 6-8GB VRAM for 7B, fits on RTX 4090, 70B on single A100. QLoRA 30-50% slower due to dequantization overhead. Quality gap: <0.1-1%. LoRA: 256x param reduction per matrix (d=4096, r=8). QLoRA: 4-bit NF4 + double quantization (4.127 bits/param) + paged optimizers. Rank: r=8 simple, r=16 default, r=32-64 complex. Alpha = 2x rank. Target modules: all-linear. Cost: QLoRA $5 for 500 examples. When to use LoRA: 24GB+ VRAM, max speed. When to use QLoRA: consumer GPU, budget, large models. Both produce 50MB adapters, hot-swappable, no inference latency.

LoRA vs QLoRA in 2026: Memory, Speed, Quality, and When to Use Each

Fine-tuning used to mean owning a GPU cluster. LoRA made it cheap. QLoRA made it possible on a consumer GPU. Together they turned model customization from an infrastructure problem into a weekend project. In 2026, LoRA and QLoRA are the default for 95% of production fine-tunes.

Key Statistics

Metric	LoRA	QLoRA	Source
Trainable params (7B, r=8)	~4M (0.1%)	~4M (0.1%)	mljourney 2026
Base model memory (7B)	14 GB (fp16)	3.5-4 GB (NF4)	dev 2026
Total VRAM (7B)	14-16 GB	6-8 GB	theneuralbase 2026
Training speed (A100)	~100 steps/min	~60 steps/min	theneuralbase 2026
Speed difference	2-3x faster	30-50% slower	mljourney 2026
Quality vs full fine-tune	<0.1% gap	<0.1-1% gap	theneuralbase 2026
Adapter size	~50 MB	~50 MB	metricgate 2026
Param reduction per matrix	256x (d=4096, r=8)	256x (d=4096, r=8)	dev 2026
Bits per param (base)	16 (fp16)	4.127 (NF4+DQ)	metricgate 2026
Max model (single GPU)	13-30B	70B+	theneuralbase 2026
QLoRA cost (500 examples)	—	~$5	medium 2026
Inference latency	None	None	teqvolt 2026

LoRA vs QLoRA Comparison

Dimension	LoRA	QLoRA	Winner
Memory (7B)	~16GB VRAM	~6-8GB VRAM	QLoRA
Training speed	~100 steps/min	~60 steps/min	LoRA
Max model (single GPU)	13-30B	70B+	QLoRA
Quantization	No (FP16/BF16)	Yes (4-bit NF4)	LoRA
Final quality	Identical to full FT	Within 0.1-1%	Tie
Implementation	Standard transformers	bitsandbytes + peft	LoRA
Inference speed	Same as base	Same as base	Tie
Hardware flexibility	24GB+ required	Consumer GPUs	QLoRA
Adapter portability	50 MB adapter	50 MB adapter	Tie

Source: theneuralbase (2026), mljourney (2026), metricgate (2026).

VRAM Requirements by Model Size

Model Size	Full Fine-Tuning	LoRA	QLoRA	GPU (QLoRA)
7B	60-100+ GB	14-16 GB	6-8 GB	RTX 4090, Colab T4
13B	120-200+ GB	24-32 GB	10-14 GB	RTX 4090
30B	300+ GB	48-64 GB	18-24 GB	RTX 4090, A5000
70B	600+ GB	80-120 GB	35-48 GB	A100 48GB, A6000

Sources: youngju (2026), theneuralbase (2026), spheron (2026).

QLoRA Architecture

flowchart TD Base["Base Model\n7B params, fp16\n14 GB VRAM"] --> Quant["1. NF4 Quantization\n4-bit NormalFloat\nMatches normal distribution\n16 values at quantile points\nMore resolution around zero\n14 GB → 3.5 GB"] Quant --> DQ["2. Double Quantization\nQuantize the quantization constants\nfp32 → fp8\nSaves ~0.37 bits/param\nTotal: 4.127 bits/param\n3.5 GB → 3.6 GB (with constants)"] DQ --> Adapters["3. LoRA Adapters (bf16)\nA: r x d, B: d x r\nr=16: ~50 MB\nTrain in bfloat16\nFrozen base stays 4-bit"] Adapters --> Train["4. Training\nForward: dequantize 4-bit → bf16\nCompute W + BA\nBackward: gradients only for A, B\nBase weights never updated"] Train --> Paged["5. Paged Optimizers\nNVIDIA unified memory\nOffload optimizer states to CPU\nwhen GPU memory is full\nPrevents OOM crashes\nSafety net, not performance"] Paged --> Output["Output\n50 MB adapter file\nMerge into base for inference\nZero inference latency\nHot-swappable"]

Source: metricgate (2026), mljourney (2026), dev (2026).

Decision Framework

flowchart TD Start["Fine-tuning a model"] --> VRAM{"Available VRAM?"} VRAM -->|"< 16 GB"| QLoRA1["QLoRA\nOnly viable option\n6-8 GB for 7B\nRTX 4090, Colab T4"] VRAM -->|"16-24 GB"| Model1{"Model size?"} VRAM -->|"24+ GB"| Model2{"Model size?"} Model1 -->|"7B-13B"| QLoRA2["QLoRA\nMore headroom\nFaster than at 8GB"] Model1 -->|"30B+"| QLoRA3["QLoRA\nRequired for large models\non limited hardware"] Model2 -->|"7B-30B"| LoRA1["LoRA\n2-3x faster\nNo quantization overhead\nMaximum quality"] Model2 -->|"70B+"| QLoRA4["QLoRA\n70B on single A100 80GB\nLoRA needs multi-GPU"] QLoRA1 --> Config["Configure\nr=16, alpha=32\nNF4, double quant\npaged_adamw_32bit\ngradient_checkpointing"] QLoRA2 --> Config QLoRA3 --> Config QLoRA4 --> Config LoRA1 --> Config2["Configure\nr=16, alpha=32\nall-linear target modules\nadamw_8bit\ncosine LR scheduler"] Config --> Train["Train\n2-4 epochs\nlr=2e-4\n30-90 min for 1K examples"] Config2 --> Train Train --> Eval["Evaluate\nValidation loss\nTask-specific metrics\nCompare vs base model"] Eval --> Deploy["Deploy\nMerge adapter into base\nServe with vLLM\nZero inference latency"]

Source: theneuralbase (2026), mljourney (2026).

Configuration Guide

Parameter	LoRA	QLoRA	Description
Rank (r)	16	16	Start at 16. r=8 simple, r=32-64 complex
Alpha	32 (2x rank)	32 (2x rank)	Scaling factor. alpha/r = 2.0
Target modules	all-linear	all-linear	q,k,v,o + gate,up,down
Learning rate	2e-4	2e-4	5-10x higher than full FT
Epochs	2-4	2-4	More risks overfitting
Batch size	4 (grad accum 4)	2 (grad accum 8)	Effective batch 16
Optimizer	adamw_8bit	paged_adamw_32bit	Paged for QLoRA OOM safety
Precision	bf16	bf16 (compute)	QLoRA base is 4-bit NF4
Gradient checkpointing	True	True	Saves 30% VRAM
Quantization	None	NF4 + double quant	QLoRA only

Sources: youngju (2026), aiworkflowlab (2026), theneuralbase (2026).

Implementation Guide

Phase	What to Do	Timeline
1. Check VRAM	Determine available GPU memory	1 hour
2. Choose method	24GB+ → LoRA. <24GB → QLoRA. 70B → QLoRA	1 hour
3. Install dependencies	pip install unsloth trl peft bitsandbytes	1 hour
4. Load base model	LoRA: fp16. QLoRA: 4-bit NF4 with BitsAndBytesConfig	1 hour
5. Configure LoRA	r=16, alpha=32, target_modules='all-linear'	1 hour
6. Prepare dataset	500-1,000 high-quality examples, official chat template	1-2 weeks
7. Configure training	epochs=3, lr=2e-4, cosine scheduler, gradient checkpointing	1 hour
8. Train	30-90 min for 1K examples (7B). Hours for 70B	1-3 hours
9. Evaluate	Validation loss, task metrics, compare vs base	1-2 days
10. Merge adapter	Merge A,B into base weights for inference	1 hour
11. Deploy	Serve with vLLM or Ollama. Hot-swap adapters	1-2 days
12. Monitor	Track quality, latency, cost in production	Ongoing

Best Practices

Start with QLoRA on a 7B model — the quality gap between 7B and 70B fine-tuned models is often smaller than expected, and the iteration speed difference is enormous. Validate your dataset and training config on 7B before scaling up (ailearnings 2026).
Use target_modules='all-linear' — applying LoRA to all linear layers consistently outperforms applying it only to attention projections. Avoids architecture-specific gotchas (Mistral needs all 4 attention projections, Qwen has different naming) (mljourney 2026).
Set alpha = 2 x rank — the effective scale alpha/r = 2.0 makes learning rate behave predictably across rank sweeps. This is the convention used by Microsoft and most tutorials (teqvolt 2026, medium 2026).
Start at rank 16 — r=16 is the 2026 recommended starting point. If quality is insufficient, increase to 32, then 64. Only go beyond 64 if you see clear under-fitting (metricgate 2026).
Use paged_adamw_32bit for QLoRA — prevents OOM crashes during memory spikes. Requires tuning gradient_accumulation_steps and max_grad_norm to avoid silent OOM errors mid-epoch (theneuralbase 2026).
Enable double quantization — saves ~0.37 bits/param. Enabled by default in bitsandbytes. Requires bitsandbytes >= 0.41.0 (dev 2026).
Monitor rank's VRAM impact — rank 8 and rank 64 have completely different VRAM footprints. A 7B model goes from ~16GB (r=8) to ~32GB (r=64) with LoRA. If you need high rank on limited VRAM, use QLoRA (theneuralbase 2026).
Merge adapter for inference — merging eliminates any inference overhead. The merged model is identical in architecture to the base. Use vLLM for multi-adapter serving if you need hot-swap (teqvolt 2026).

For related topics, see our how to fine-tune LLM, AI model distillation, small language models, training data preparation, and evaluate fine-tuned LLM guides.

FAQ

What is the quality difference between LoRA, QLoRA, and full fine-tuning?

The quality difference between LoRA, QLoRA, and full fine-tuning is minimal for most tasks. Both LoRA and QLoRA produce adapters within 0.1-1% of full fine-tuning quality. Full fine-tuning: (1) Quality — baseline. The gold standard. Updates every parameter. (2) Best for — extreme domain shifts (highly specialized medical/legal text), continual pretraining on new text corpora, when LoRA at rank 64 still has unacceptable quality gap. (3) Cost — 60-100+ GB VRAM for 7B. 600+ GB for 70B. Requires multi-GPU infrastructure. (4) When needed — rarely. Only when LoRA/QLoRA quality is insufficient after extensive tuning. LoRA: (1) Quality — identical to full fine-tune baseline. On-par or better than full fine-tuning on RoBERTa, DeBERTa, GPT-2, and GPT-3 (original LoRA paper). (2) Quality gap — <0.1% vs full fine-tuning. Unmeasurable on most benchmarks. (3) Best for — 95% of production fine-tunes. Default starting point. (4) Cost — 14-16 GB VRAM for 7B. 85% reduction vs full fine-tuning. QLoRA: (1) Quality — within 0.1-1% of LoRA/full fine-tuning on most benchmarks. (2) Quality gap — <1% vs LoRA. The QLoRA paper demonstrated a 65B model fine-tuned with QLoRA on a single 48GB GPU matched full fine-tuning quality on instruction-following benchmarks. (3) Best for — consumer GPUs, budget-constrained teams, 70B models on single GPU. (4) Cost — 6-8 GB VRAM for 7B. 75% reduction vs LoRA. When quality matters: (1) For most tasks — LoRA and QLoRA quality is indistinguishable from full fine-tuning. Choose based on hardware, not quality. (2) For extreme domain shifts — full fine-tuning may be needed. If the domain is very different from pretraining distribution (highly specialized medical/legal text), LoRA adapters may not have enough capacity. Try LoRA at rank 64 first. (3) For competition-level tasks — full fine-tuning may have a slight edge. But for production tasks, the difference is negligible. (4) For continual pretraining — full fine-tuning is the right choice. You're adding new knowledge, not just adapting behavior. The key: 'Both methods preserve the full fine-tuning capability when using identical rank/alpha. Quality difference is unmeasurable (<0.1%).' For 95% of production use cases, LoRA or QLoRA is sufficient. Reserve full fine-tuning for extreme domain shifts where LoRA at high rank still falls short (theneuralbase 2026, mljourney 2026, teqvolt 2026, dev 2026)."
- question: "What are the production gotchas with LoRA and QLoRA?"
answer: "LoRA and QLoRA are powerful but have production gotchas that will trip developers who aren't aware of them. Gotcha 1: Rank dramatically affects memory. A 7B model with LoRA at rank 8 uses ~16GB. At rank 64, it uses ~32GB. This pushes users toward QLoRA unexpectedly. If you increase rank to improve quality and suddenly OOM, the solution is QLoRA, not a bigger GPU. Always check VRAM requirements for your chosen rank before training. Gotcha 2: QLoRA's paged_adamw_8bit optimizer is not a magic solution. It requires tuning gradient_accumulation_steps and max_grad_norm to avoid silent OOM errors mid-epoch. The default values in most tutorials (batch_size=4) fail on most consumer GPUs. Start with batch_size=2 and gradient_accumulation_steps=8. Monitor for silent OOM — training continues but quality degrades. Gotcha 3: target_modules architecture differences. target_modules=[q_proj,v_proj] works for Llama but fails silently on Mistral (needs q_proj,k_proj,v_proj,o_proj), Qwen (different naming), and non-transformer architectures. No error is raised — training just happens on wrong layers, producing poor results. Always use target_modules='all-linear' for supported architectures. Gotcha 4: BitsAndBytesConfig version requirements. BitsAndBytesConfig requires bitsandbytes >= 0.41.0 (released Nov 2023). Older versions throw AttributeError on bnb_4bit_use_double_quant. Always check your bitsandbytes version before starting QLoRA training. Gotcha 5: Chat template mismatch. Every instruction-tuned model has a specific template format (Llama 3 uses <|begin_of_text|>, <|start_header_id|>, etc.). Applying the wrong template or a custom one degrades quality significantly. Always use tokenizer.apply_chat_template(). Never use a custom template. Gotcha 6: Forgetting to hold out evaluation data. Separate training and evaluation data before writing a single line of training code. Hold out 10-15% of examples. If you only discover you need an eval set after training, the data has already been contaminated. Gotcha 7: Monitoring training loss instead of validation loss. Training loss always decreases — that's what gradient descent does. Validation loss tells you if the model is actually learning or just memorizing. If validation loss increases while training loss decreases, stop training — you're overfitting. Gotcha 8: Not comparing against the base model. Always compare your fine-tuned model against the base model on the same test set. If the fine-tuned model doesn't outperform the base model, the fine-tuning didn't help. Also compare against the base model with good prompts (few-shot, CoT) — if prompting achieves similar results, fine-tuning wasn't necessary. Gotcha 9: Using raw conversation transcripts. Real conversations are messy: agents say 'um,' make typos, give inconsistent answers. Clean your data. Take your best conversations, clean them up, and make sure every response demonstrates exactly the behavior you want. Think of each example as a specification, not a transcript. Gotcha 10: Not merging the adapter for inference. During training, the adapter is separate from the base. For inference, merge the adapter into the base weights: W_merged = W + BA. This eliminates any inference overhead. If you don't merge, you're carrying unnecessary adapter computation overhead. Gotcha 11: Ignoring learning rate for different methods. LoRA: 1e-4 to 3e-4 (5-10x higher than full FT). QLoRA: slightly lower, 1e-4 to 2e-4. Full fine-tuning: 1e-5 to 2e-5. Using the wrong learning rate for your method leads to poor results. Gotcha 12: Not using gradient checkpointing. Gradient checkpointing saves 30% VRAM at the cost of ~20% slower training. For QLoRA on consumer GPUs, this is often the difference between fitting and OOM. Always enable it. Gotcha 13: Forgetting to set max_grad_norm. For QLoRA with paged optimizers, set max_grad_norm=0.3 to prevent gradient spikes from causing instability. The default may not work for your specific model and dataset. The key: 'Three production gotchas that will trip developers: rank dramatically affects memory, paged optimizer is not a magic solution, and target_modules architecture differences cause silent failures.' Be aware of these gotchas before starting your fine-tuning project (theneuralbase 2026, ailearnings 2026, mljourney 2026)."