>
Y
Model Details
Domain:
Model Access:
Open weights (restricted use)
Introduction
The Yi series models are large language models trained from scratch by developers at 01.AI.
Benchmarking
FLOPs
6.1e+23Notes: "The dataset we use contains Chinese & English only. We used approximately 3T tokens" sounds like this means it was trained on 3T tokens, not necessarily that the dataset contains 3T tokens? If so, 34b * 3T * 6 = 6.1e23
Training
Training Code Accessibility
apply for commercial license:
no training code
https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt
the model https://huggingface.co/01-ai/Yi-34B-Chat Apache 2.0
"If you create derivative works based on this model, please include the following attribution in your derivative works: ...."Hardware
NVIDIA A100Hardware Quantity
128Size Notes: "language models pretrained from scratch on 3.1T highly-engineered large amount of data, and finetuned on a small but meticulously polished alignment data."
Parameters
Parameters
34000000000Notes: 34b
Authors
Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shiming Yang, Tao Yu, Wen Xie, Wenhao Huang, Xiaohui Hu, Xiaoyi Ren, Xinyao Niu, Pengcheng Nie, Yuchi Xu, Yudong Liu, Yue Wang, Yuxuan Cai, Zhenyu Gu, Zhiyuan Liu, Zonghong Dai
