Efficient Fine-tuning

LoRA / QLoRA

Fine-tune billion-parameter models on a single consumer GPU. Train adapters instead of full weights.

What is LoRA?

LoRA (Low-Rank Adaptation) freezes the original model weights and injects small trainable "adapter" matrices. Instead of training 70 billion parameters, you train 10-100 million.

The key insight: weight updates during fine-tuning have low intrinsic rank. We can approximate ΔW with two smaller matrices: ΔW = A × B, where A and B are much smaller.

# Standard fine-tuning: Update W directly (70B params)
W_new = W_original + ΔW

# LoRA: Learn low-rank decomposition
# A: (d × r), B: (r × d) where r << d (typically r = 8 to 64)
W_new = W_original + A @ B  # Only train A and B (~1% of params)

LoRA vs QLoRA

LoRA

• Base model in FP16
• 70B requires ~140GB VRAM
• Faster training
• Slightly better quality

QLoRA

• Base model in 4-bit (NF4)
• 70B fits in ~40GB VRAM
• Slower (dequantize on fly)
• ~99% of LoRA quality

💡 Recommendation: Use QLoRA for 70B+ models on consumer GPUs. Use LoRA for smaller models or when you have ample VRAM.

VRAM for Training

Model	Full Fine-tune	LoRA	QLoRA
Llama 7B	~120 GB	~18 GB	~6 GB
Llama 13B	~200 GB	~32 GB	~10 GB
Llama 70B	~1 TB	~160 GB	~48 GB

Quick Start with Unsloth

from unsloth import FastLanguageModel
from trl import SFTTrainer

# Load model with QLoRA
model, tokenizer = FastLanguageModel.from_pretrained(
    "unsloth/llama-3-8b-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True
)

# Add LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16,  # LoRA rank
    lora_alpha=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_dropout=0,
)

# Train
trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=2048
)
trainer.train()

# Save adapter (small, ~100MB)
model.save_pretrained("my-lora-adapter")

Key Parameters

rank (r)

Size of adapter matrices. Higher = more capacity but more VRAM. Typical: 8-64.

lora_alpha

Scaling factor. Usually set equal to r. Affects learning rate effectively.

target_modules

Which layers to adapt. Usually attention layers: ["q_proj", "k_proj", "v_proj", "o_proj"]