Model Details

Domain:

Task:

Model Access:

Open weights (restricted use)

AI Tools Usage

This model is commonly used behind the scenes in AI tools.

Introduction

Now we’re officially releasing Gemma 2 to researchers and developers globally. Available in both 9 billion (9B) and 27 billion (27B) parameter sizes, Gemma 2 is higher-performing and more efficient at inference than the first generation, with significant safety advancements built in. In fact, at 27B, it offers competitive alternatives to models more than twice its size, delivering the kind of performance that was only possible with proprietary models as recently as December. And that’s now achievable on a single NVIDIA H100 Tensor Core GPU or TPU host, significantly reducing deployment costs.

Benchmarking

FLOPs3.12e+22

Notes: "For the 2.6B model, we train on a 2x16x16 configuration of TPUv5e, totaling 512 chips" 6ND = 6 FLOP / token / parameter * 2600000000 parameters * 2000000000000 tokens = 3.12e+22 FLOP

Training

Training Code Accessibilityhttps://huggingface.co/google/gemma-2-2b Gemma 2 is available under our commercially-friendly Gemma license, giving developers and researchers the ability to share and commercialize their innovations.

HardwareGoogle TPU v5e

Hardware Quantity512

Size Notes: "the 2.6B on 2 trillion tokens"