>
Q
Model Details
Domain:
Model Access:
Open weights (unrestricted)
Introduction
After months of efforts, we are pleased to announce the evolution from Qwen1.5 to Qwen2. This time, we bring to you: - Pretrained and instruction-tuned models of 5 sizes, including Qwen2-0.5B, Qwen2-1.5B, Qwen2-7B, Qwen2-57B-A14B, and Qwen2-72B; - Having been trained on data in 27 additional languages besides English and Chinese; - State-of-the-art performance in a large number of benchmark evaluations; - Significantly improved performance in coding and mathematics; - Extended context length support up to 128K tokens with Qwen2-7B-Instruct and Qwen2-72B-Instruct. (Technical report to follow)
Benchmarking
FLOPs
3.02e+24Notes: 72 billion params, 7 trillion tokens 6 * 72 billion * 7 trillion ~= 3.02e24
Training
Training Code Accessibility
Apache 2.0Size Notes: "All models were pre-trained on a high-quality, large-scale dataset comprising over 7 trillion tokens, covering a wide range of domains and languages."
Parameters
Parameters
72710000000Notes: 72.71B parameters in total, of which 70.21B are non-embedding parameters
Authors
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, Zhihao Fan
Multimodal
Vision
Video