>
F
Model Details
Domain:
Model Access:
Open weights (restricted use)
Introduction
Falcon2-11B is an 11B parameters causal decoder-only model built by TII and trained on over 5,000B tokens of RefinedWeb enhanced with curated corpora. The model is made available under the TII Falcon License 2.0, the permissive Apache 2.0-based software license which includes an acceptable use policy that promotes the responsible use of AI.
Benchmarking
FLOPs
3.6e+23Notes: trained on 5.5T tokens 6 * 11B * 5.5T = 3.6e23
Training
Training Code Accessibility
Open but has an acceptable use policy: https://falconllm-staging.tii.ae/falcon-2-acceptable-use-policy.html
https://huggingface.co/tiiuae/falcon-11BHardware
NVIDIA A100 SXM4 40 GBSize Notes: 5.5T tokens: https://falconllm.tii.ae/falcon-2.html
Parameters
Parameters
11000000000Notes: 11B
