>
G
Model Details
Domain:
Task:
Model Access:
Open weights (restricted use)
Introduction
Benchmarking
FLOPs4.51e+22
Notes: 6ND = 6*2506434560.00 parameters * 3*10^12 training tokens = 4.5115822e+22 (assuming 1 epoch)
Training
Training Code Accessibilityhttps://ai.google.dev/gemma/terms no illegal use or abuse
HardwareGoogle TPU v5e
Hardware Quantity512
Size Notes: "Gemma 2B and 7B are trained on 3T and 6T tokens respectively of primarily-English data from web documents, mathematics, and code." Not explicitly stated that this doesn't involve multiple epochs, but I expect it does not.
Parameters
Parameters2506434560
Notes: Table 2, sum of embedding and non-embedding parameters: 2B 524,550,144 + 1,981,884,416 = 2506434560
Authors
Gemma Team, Google DeepMind
Multimodal
Vision
Video
Speech