>
B
Model Details
Domain:
Model Access:
Open weights (restricted use)
Citations:
894
Introduction
In this technical report, we present Baichuan 2, a series of large-scale multilingual language models containing 7 billion and 13 billion parameters, trained from scratch, on 2.6 trillion tokens.
Benchmarking
FLOPs
1.09e+23Notes: 7b * 2.6t * 6 = 1.092e23 Also mentions 1,024 NVIDIA A800 GPUs at 180 TFLOPS per GPU
Training
Training Code Accessibility
https://huggingface.co/baichuan-inc/Baichuan2-7B-Base
license here: https://github.com/baichuan-inc/Baichuan2?tab=readme-ov-file
Baichuan 2 模型社区许可协议 (Community License Agreement)
restrictions on commercial applications with many DAUs and particular types of businesses
Apache 2.0 for codeHardware
NVIDIA A800 PCIe 40 GBHardware Quantity
1024Parameters
Parameters
7000000000Authors
Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, Fei Deng, Feng Wang, Feng Liu, Guangwei Ai, Guosheng Dong, Haizhou Zhao, Hang Xu, Haoze Sun, Hongda Zhang, Hui Liu, Jiaming Ji, Jian Xie, Juntao Dai, Kun Fang, Lei Su, Liang Song, Lifeng Liu, Liyun Ru, Luyao Ma, Mang Wang, Mickel Liu, MingAn Lin, Nuolan Nie, Peidong Guo, Ruiyang Sun, Tao Zhang, Tianpeng Li, Tianyu Li, Wei Cheng, Weipeng Chen, Xiangrong Zeng, Xiaochuan Wang, Xiaoxi Chen, Xin Men, Xin Yu, Xuehai Pan, Yanjun Shen, Yiding Wang, Yiyu Li, Youxin Jiang, Yuchen Gao, Yupeng Zhang, Zenan Zhou, Zhiying Wu
Multimodal
Speech