>
S
Model Details
Domain:
Model Access:
Open weights (non-commercial)
Introduction
Benchmarking
FLOPs
2.11e+22Notes: 6ND = 2.7e9 * 1.3e12 * 6 = 2,106E+22 "stable-code-3b is a 2.7B billion parameter decoder-only language model pre-trained on 1.3 trillion tokens of diverse textual and code datasets. "
Training
Training Code Accessibility
non-commercial by default. looks like they charge for commercial licenses?
https://stability.ai/news/introducing-stability-ai-membershipHardware
NVIDIA A100 SXM4 40 GBHardware Quantity
256Size Notes: 1.3T tokens "stable-code-3b is a 2.7B billion parameter decoder-only language model pre-trained on 1.3 trillion tokens of diverse textual and code datasets. "
Parameters
Parameters
2796431360Notes: 2796431360 from https://huggingface.co/stabilityai/stable-code-3b#model-architecture "stable-code-3b is a 2.7B billion parameter decoder-only language model pre-trained on 1.3 trillion tokens of diverse textual and code datasets. "
Authors
Pinnaparaju, Nikhil and Adithyan, Reshinth and Phung, Duy and Tow, Jonathan and Baicoianu, James and and Cooper, Nathan
Image generation