>
L
Model Details
Domain:
Task:
Model Access:
Open weights (restricted use)
Introduction
We’re sharing the first models in the Llama 4 herd, which will enable people to build more personalized multimodal experiences. Llama 4 Scout, a 17 billion active parameter model with 16 experts, is the best multimodal model in the world in its class and is more powerful than all previous generation Llama models, while fitting in a single NVIDIA H100 GPU. Additionally, Llama 4 Scout offers an industry-leading context window of 10M and delivers better results than Gemma 3, Gemini 2.0 Flash-Lite, and Mistral 3.1 across a broad range of widely reported benchmarks. Llama 4 Maverick, a 17 billion active parameter model with 128 experts, is the best multimodal model in its class, beating GPT-4o and Gemini 2.0 Flash across a broad range of widely reported benchmarks, while achieving comparable results to the new DeepSeek v3 on reasoning and coding—at less than half the active parameters. Llama 4 Maverick offers a best-in-class performance to cost ratio with an experimental chat version scoring ELO of 1417 on LMArena. These models are our best yet thanks to distillation from Llama 4 Behemoth, a 288 billion active parameter model with 16 experts that is our most powerful yet and among the world’s smartest LLMs. Llama 4 Behemoth outperforms GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on several STEM benchmarks. Llama 4 Behemoth is still training, and we’re excited to share more details about it even while it’s still in flight. Download the Llama 4 Scout and Llama 4 Maverick models today on llama.com and Hugging Face. Try Meta AI built with Llama 4 in WhatsApp, Messenger, Instagram Direct, and on the web.
Benchmarking
FLOPs2.24e+24
Notes: 22T training tokens per model card: https://github.com/meta-llama/llama-models/blob/main/models/llama4/MODEL_CARD.md Maverick was trained using co-distillation from Llama 4 Behemoth. It isn't 100% clear that all 22T tokens used distillation, but we assume this for the time being. Estimating training compute from parameters and tokens: Compute = 6 FLOP per token per parameter * 17B active parameters * 22T tokens = 2.244e24 FLOP (Implying mean throughput was 262 TFLOPS/GPU, or 13.2% MFU in FP8) The model card also states that Llama 4 Maverick used 2.38M H100-hours. The blog post gives a figure of 390 TFLOPS/GPU, but this may have been the utilization rate for Behemoth, or all of the models together. Using this utilization, we have: Compute = 390 TFLOP/s * 2.38 million hours = 3.342e24 FLOP (This value is higher than the compute implied by parameters and tokens, and suggests utilization may have been lower for Maverick than for Behemoth.)
Training
Training Code AccessibilityLlama 4 license (branding requirements, size cap 700M MAU) https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Original no training code here https://github.com/meta-llama/llama-models/tree/main/models/llama4
Size Notes: "The overall data mixture for training consisted of more than 30 trillion tokens, which is more than double the Llama 3 pre-training mixture and includes diverse text, image, and video datasets."
Parameters
Parameters400000000000
Notes: "Llama 4 Maverick models have 17B active parameters and 400B total parameters." https://ai.meta.com/blog/llama-4-multimodal-intelligence/
