Model Details

Domain:

Task:

Model Access:

Open weights (restricted use)

AI Tools Usage

This model is commonly used behind the scenes in AI tools.

Introduction

Falcon2-11B is an 11B parameters causal decoder-only model built by TII and trained on over 5,000B tokens of RefinedWeb enhanced with curated corpora. The model is made available under the TII Falcon License 2.0, the permissive Apache 2.0-based software license which includes an acceptable use policy that promotes the responsible use of AI.

Benchmarking

FLOPs3.6e+23

Notes: trained on 5.5T tokens 6 * 11B * 5.5T = 3.6e23

Training

Training Code AccessibilityOpen but has an acceptable use policy: https://falconllm-staging.tii.ae/falcon-2-acceptable-use-policy.html https://huggingface.co/tiiuae/falcon-11B

HardwareNVIDIA A100 SXM4 40 GB

Size Notes: 5.5T tokens: https://falconllm.tii.ae/falcon-2.html