>
S
Model Details
Domain:
Model Access:
Open weights (unrestricted)
Introduction
SEA-LION version 1 was released in December 2023 and trained from scratch with 1 trillion tokens, 13 percent of which are from SEA.
Benchmarking
FLOPs2.19e+22
Notes: 6 FLOP / parameter / token * 3B parameters * 980B tokens = 1.764 × 10^22 FLOP 312000000000000 FLOP / GPU / sec * 336 hours * 3600 sec / hour * 240 GPUs * 0.3 [assumed utilization] = 2.7172454e+22 FLOP sqrt(1.764 × 10^22 * 2.7172454e+22) = 2.1893426e+22 FLOP
Training
Training Code AccessibilityMIT license https://huggingface.co/aisingapore/SEA-LION-v1-3B https://github.com/aisingapore/sealion
HardwareNVIDIA A100 SXM4 40 GB
Hardware Quantity240
Size Notes: "SEA-LION was trained on 980B tokens"
Parameters
Parameters3000000000
Authors
Lam Wen Zhi Clarence, Leong Wei Qi, Li Yier, Liu Bing Jie Darius, Lovenia Holy, Montalan Jann Railey, Ng Boon Cheong Raymond, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Tat-Wee David, Rengarajan Hamsawardhini, Susanto Yosephine, Tai Ngee Chia, Tan Choon Meng, Teo Jin Howe, Teo Eng Sipp Leslie, Teo Wei Yi, Tjhi William, Yeo Yeow Tong, Yong Xianbin
Related ModelsView all models
