Model Details

Domain:

Task:

Model Access:

Open weights (non-commercial)

AI Tools Usage

This model is commonly used behind the scenes in AI tools.

Introduction

Benchmarking

FLOPs2.11e+22

Notes: 6ND = 2.7e9 * 1.3e12 * 6 = 2,106E+22 "stable-code-3b is a 2.7B billion parameter decoder-only language model pre-trained on 1.3 trillion tokens of diverse textual and code datasets. "

Training

Training Code Accessibilitynon-commercial by default. looks like they charge for commercial licenses? https://stability.ai/news/introducing-stability-ai-membership

HardwareNVIDIA A100 SXM4 40 GB

Hardware Quantity256

Size Notes: 1.3T tokens "stable-code-3b is a 2.7B billion parameter decoder-only language model pre-trained on 1.3 trillion tokens of diverse textual and code datasets. "

Parameters

Parameters2796431360

Notes: 2796431360 from https://huggingface.co/stabilityai/stable-code-3b#model-architecture "stable-code-3b is a 2.7B billion parameter decoder-only language model pre-trained on 1.3 trillion tokens of diverse textual and code datasets. "