Meta Llama Models
The industry standard for open foundation models. From lightweight 1B/3B models for edge devices to the massive 405B frontier-class model. Available for self-hosting, fine-tuning, and cloud deployment.
Llama 3.2 & 3.1 Family
Llama 3.1 405B
Frontier ClassThe world's largest and most capable open foundation model. Rivals GPT-4o and Claude 3.5 Sonnet in general knowledge, math, and coding. Ideal for synthetic data generation and distillation.
Llama 3.1 70B
WorkhorseExcellent balance of performance and efficiency. Great for RAG, reasoning, and complex instruction following. Can be run on dual consumer GPUs (e.g., 2x RTX 3090/4090).
Llama 3.1 8B
EfficientIncredibly powerful for its size. Outperforms many larger legacy models. runs comfortably on most consumer laptops and GPUs. Perfect for classification, summarization, and local agents.
Llama 3.2 1B & 3B
Edge & MobileLightweight models designed for on-device use (mobile phones, IoT, Raspberry Pi). Supports text and image input (Multimodal 11B/90B also available, but 1B/3B are text-only optimized).
How to Run Llama
Option 1: Run Locally with Ollama (Fastest Way)
The easiest way to get up and running on Mac, Linux, or Windows. No Python required.
Option 2: Using Hugging Face Transformers
import torch
from transformers import pipeline
model_id = "meta-llama/Llama-3.1-8B-Instruct"
pipe = pipeline(
"text-generation",
model=model_id,
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto",
)
messages = [
{"role": "system", "content": "You are a helpful pirate assistant."},
{"role": "user", "content": "Who goes there?"},
]
outputs = pipe(
messages,
max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])
Option 3: Via Cloud API (Groq Example)
Use Llama 3 models via Groq for ultra-low latency (500+ tokens/sec).
from groq import Groq
client = Groq()
completion = client.chat.completions.create(
model="llama-3.1-70b-versatile",
messages=[
{"role": "system", "content": "You are a coding assistant."},
{"role": "user", "content": "Write a binary search in Python."}
],
temperature=0.5,
max_tokens=1024,
stream=True,
stop=None,
)
for chunk in completion:
print(chunk.choices[0].delta.content or "", end="")Prompting Llama 3
Chat Format
Llama 3 uses a specific system for special tokens. It is crucial to use the correct chat template.
<|begin_of_text|><|start_header_id|>system<|end_header_id|> You are a helpful assistant.<|eot_id|><|start_header_id|>user<|end_header_id|> Hello!<|eot_id|><|start_header_id|>assistant<|end_header_id|>
Best Practices
- Be Explicit: Llama follows instructions very literally.
- System Prompts: Strong system prompts significantly improve persona adherence.
- JSON Mode: Works great with simple "Output JSON" instructions, no strict mode needed often.