Mistral & Mixtral Models

About Mistral AI

Mistral AI is a French AI company founded in 2023 by former Meta and Google DeepMind researchers. In just one year, they became Europe's most valuable AI startup, known for:

Efficiency: Models that punch above their weight class
Open Weights: Most models released with Apache 2.0 license (fully commercial use)
Innovation: Pioneers of Sliding Window Attention and popularized MoE for LLMs
Multilingual: Strong performance in European languages (French, German, Spanish, Italian)

Model Family

Large

Mistral Large 2

Flagship

Mistral's most capable model. Competitive with GPT-4o and Claude 3.5 Sonnet. Excellent for complex reasoning, coding, and multilingual tasks. The go-to choice for enterprise deployments requiring top-tier performance.

Context Window 128K tokens

Parameters 123 Billion

License Commercial (API)

Strengths Code, Reasoning

8x22B

Mixtral 8x22B

MoE

Sparse Mixture-of-Experts model. 176B total parameters but only uses ~44B per token (activates 2 of 8 experts). Offers near-Large performance at much lower inference cost. Open weights with Apache 2.0 license.

Context Window 64K tokens

Total Params 176B (44B active)

License Apache 2.0

VRAM (4-bit) ~100 GB

8x7B

Mixtral 8x7B

Mistral 7B

Efficient

Mistral's original model that started it all. Outperformed Llama 2 13B on all benchmarks with half the parameters. Features Sliding Window Attention for efficient long-context. Runs on consumer hardware.

Context Window 32K tokens

Parameters 7.3 Billion

License Apache 2.0

VRAM (4-bit) ~5 GB

Codestral

Coding

Specialized coding model trained on 80+ programming languages. Competitive with GPT-4 for code generation, debugging, and refactoring. Available via API and for local use with specific non-commercial restrictions.

Context Window 32K tokens

Parameters 22 Billion

License MNPL (non-commercial)

Languages 80+

What is Mixture of Experts (MoE)?

Mixture of Experts (MoE) is an architecture where the model has multiple "expert" sub-networks, but only activates a subset (typically 2) for each input token. This enables:

Advantages

• Massive total knowledge capacity
• Fast inference (uses fraction of params)
• Better cost/performance ratio
• Enables specialization per task

Trade-offs

• Higher VRAM (all experts in memory)
• More complex quantization
• Routing overhead
• Harder to fine-tune


# MoE Architecture (simplified)
# Mixtral 8x7B: 8 experts, 2 active per token

Input Token → Router (gating network)
                  ↓
        Selects top-2 experts based on token
                  ↓
    Expert 3 (weight 0.6) + Expert 7 (weight 0.4)
                  ↓
        Weighted combination of outputs
                  ↓
            Final Output

# Total params: 8 × 7B = 46.7B
# Active params per token: 2 × 6.5B ≈ 12.9B
# Inference speed: Similar to a 13B dense model

How to Run Mistral Models

Option 1: Run Locally with Ollama

# 1. Install Ollama from ollama.com

# 2. Run Mistral 7B

ollama run mistral

# 3. Or run Mixtral 8x7B (requires ~26GB VRAM)

ollama run mixtral

# 4. For coding tasks

ollama run codestral

Option 2: Mistral API (Official)

from mistralai.client import MistralClient
from mistralai.models.chat_completion import ChatMessage

client = MistralClient(api_key="your-api-key")

messages = [
    ChatMessage(role="user", content="Explain MoE architecture in simple terms")
]

response = client.chat(
    model="mistral-large-latest",  # or "mistral-medium", "mistral-small"
    messages=messages,
    temperature=0.7,
    max_tokens=1024
)

print(response.choices[0].message.content)

Option 3: Hugging Face Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "mistralai/Mixtral-8x7B-Instruct-v0.1"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    load_in_4bit=True  # Quantize for lower VRAM
)

messages = [
    {"role": "user", "content": "Write a Python function for binary search"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Mistral vs Llama vs GPT

Model	Params	Context	License	Best For
Mixtral 8x7B	46.7B (12.9B active)	32K	Apache 2.0	Best open MoE, fast inference
Llama 3.1 70B	70B (all active)	128K	Llama Community	Dense reasoning, longer context
Mistral Large 2	123B	128K	Commercial API	Enterprise, multilingual
GPT-4o	~200B (estimated)	128K	API Only	General excellence, multimodal
Mistral 7B	7.3B	32K	Apache 2.0	Edge, low resources
Llama 3.1 8B	8B	128K	Llama Community	Longer context, newer