>
We introduce CodeGeeX, a large-scale multilingual code generation model with 13 billion parameters, pre-trained on a large code corpus of more than 20 programming languages. As of June 22, 2022, CodeGeeX has been trained on more than 850 billion tokens on a cluster of 1,536 Ascend 910 AI Processors.
Notes: Assume 1 epoch on 850B tokens. C=6DN=6*850B*13B https://www.wolframalpha.com/input?i=6+*+13+billion+*+850+billion
Size Notes: As of June 22, 2022, CodeGeeX has been trained on more than 850 billion tokens
Notes: "We introduce CodeGeeX, a large-scale multilingual code generation model with 13 billion parameters"