>
D
Model Details
Domain:
Model Access:
Open weights (restricted use)
Introduction
DCLM-Baseline-7B is a 7 billion parameter language model trained on the DCLM-Baseline dataset, which was curated as part of the DataComp for Language Models (DCLM) benchmark. This model is designed to showcase the effectiveness of systematic data curation techniques for improving language model performance.
Benchmarking
FLOPs
1.05e+23Notes: 6 FLOP / parameter / token * 7 * 10^9 parameters * 2.5 * 10^12 tokens = 1.05e+23 FLOP
Training
Training Code Accessibility
Apple Sample Code license (no patent rights, copyright-only)
https://huggingface.co/apple/DCLM-7BHardware
NVIDIA H100 SXM5 80GBSize Notes: Total Training Tokens: 2.5T
Parameters
Parameters
7000000000Notes: 7B
