>
B
Model Details
Domain:
Model Access:
Open weights (non-commercial)
Introduction
Baichuan-7B is an open-source large-scale pre-trained model developed by Baichuan Intelligent Technology. Based on the Transformer architecture, it is a model with 7 billion parameters trained on approximately 1.2 trillion tokens. It supports both Chinese and English, with a context window length of 4096. It achieves the best performance of its size on standard Chinese and English authoritative benchmarks (C-EVAL/MMLU).
Benchmarking
FLOPs5.04e+22
Notes: 7b parameters * 1.2t tokens * 6 FLOP / parameter / token = 5.04e22 FLOP
Training
Training Code Accessibilityhttps://huggingface.co/baichuan-inc/Baichuan-7B no clear license https://github.com/baichuan-inc/Baichuan-7B
HardwareNVIDIA A800 PCIe 40 GB
Hardware Quantity1024
Parameters
Parameters7000559616
Multimodal
Speech