>
L
Model Details
Domain:
Task:
Model Access:
Open weights (restricted use)
Introduction
We’re sharing the first models in the Llama 4 herd, which will enable people to build more personalized multimodal experiences. Llama 4 Scout, a 17 billion active parameter model with 16 experts, is the best multimodal model in the world in its class and is more powerful than all previous generation Llama models, while fitting in a single NVIDIA H100 GPU. Additionally, Llama 4 Scout offers an industry-leading context window of 10M and delivers better results than Gemma 3, Gemini 2.0 Flash-Lite, and Mistral 3.1 across a broad range of widely reported benchmarks. Llama 4 Maverick, a 17 billion active parameter model with 128 experts, is the best multimodal model in its class, beating GPT-4o and Gemini 2.0 Flash across a broad range of widely reported benchmarks, while achieving comparable results to the new DeepSeek v3 on reasoning and coding—at less than half the active parameters. Llama 4 Maverick offers a best-in-class performance to cost ratio with an experimental chat version scoring ELO of 1417 on LMArena. These models are our best yet thanks to distillation from Llama 4 Behemoth, a 288 billion active parameter model with 16 experts that is our most powerful yet and among the world’s smartest LLMs. Llama 4 Behemoth outperforms GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on several STEM benchmarks. Llama 4 Behemoth is still training, and we’re excited to share more details about it even while it’s still in flight. Download the Llama 4 Scout and Llama 4 Maverick models today on llama.com and Hugging Face. Try Meta AI built with Llama 4 in WhatsApp, Messenger, Instagram Direct, and on the web.
Benchmarking
FLOPs
4.08e+24Notes: 40T training tokens per model card: https://github.com/meta-llama/llama-models/blob/main/models/llama4/MODEL_CARD.md Estimating training compute from parameters and tokens: 6 FLOP per token per parameter * 17B active parameters * 40T tokens = 4.08e24 FLOP (Implying mean throughput was 227 TFLOPS/GPU, or 11.5% MFU in FP8) The model card also states that Llama 4 Scout used 5.0M H100-hours. The blog post gives a figure of 390 TFLOPS/GPU, but this may have been the utilization rate for Behemoth, or all of the models together. Using this utilization, we have: Compute = 390 TFLOP/s * 5 million hours = 7.02e24 FLOP (This value is higher than the compute implied by parameters and tokens, and suggests utilization may have been lower for Scout than for Behemoth.)
Training
Training Code Accessibility
Llama 4 license (branding requirements, size cap 700M MAU)
https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E
no training code here
https://github.com/meta-llama/llama-models/tree/main/models/llama4Size Notes: "The overall data mixture for training consisted of more than 30 trillion tokens, which is more than double the Llama 3 pre-training mixture and includes diverse text, image, and video datasets."
Parameters
Parameters
109000000000Notes: "Our smaller model, Llama 4 Scout, is a general purpose model with 17 billion active parameters, 16 experts, and 109 billion total parameters that delivers state-of-the-art performance for its class."
