>
G
Model Details
Introduction
We introduce the Granite series of decoder-only foundation models for generative artificial intelligence (AI) tasks that are ready for enterprise use. We report on the architecture, capabilities, underlying data and data governance, training algorithms, compute infrastructure, energy and carbon footprint, testing and evaluation, socio-technical harms and mitigations, and usage policies.
Benchmarking
FLOPs3e+23
Notes: 6*2500000000000*20000000000=3e+23
Training
Training Code AccessibilityApache 2.0 https://huggingface.co/ibm-granite/granite-20b-code-base-8k no pretraining code here, inference and fine-tuning only https://github.com/ibm-granite/granite-code-models
Size Notes: For pre-training, we used 0.5 trillion English, 0.4 trillion multilingual (es, fr, de, pt), and 1.6 trillion code tokens.
Parameters
Parameters20000000000
Authors
IBM Research