>
F
Model Details
Domain:
Model Access:
Open weights (unrestricted)
Citations:
0
Introduction
Benchmarking
FLOPs2.4e+23
Notes: C = 6ND = 6 * 40B * 1000B = 2.4e+23 FLOP (assuming one epoch) Table 1 from https://arxiv.org/pdf/2311.16867 Falcon paper 2,800 petaflop-days * 1e15 * 24 * 3600 = 2.4192e+23 FLOPs
Training
Training Code Accessibilityapache 2.0
HardwareNVIDIA A100
Hardware Quantity384
Size Notes: 1000B tokens ~= 750B words
Parameters
Parameters40000000000
Notes: Model comes in 7B and 40B variants.
