>
S
Model Details
Domain:
Model Access:
Open weights (unrestricted)
Introduction
SEA-LION version 1 was released in December 2023 and trained from scratch with 1 trillion tokens, 13 percent of which are from SEA.
Benchmarking
FLOPs
4.33e+22Notes: 6 FLOP / parameter / token * 7B parameters * 980B tokens = 4.116e+22 FLOP 312000000000000 FLOP / GPU / sec * 256 GPUs * 22 days * 24 hours / day * 3600 sec / hour * 0.3 [assumed utilization] = 4.5546209e+22 FLOP sqrt(4.116e+22*4.5546209e+22) = 4.3297598e+22 FLOP
Training
Training Code Accessibility
MIT license
https://huggingface.co/aisingapore/SEA-LION-v1-7B
https://github.com/aisingapore/sealionHardware
NVIDIA A100 SXM4 40 GBHardware Quantity
256Size Notes: "SEA-LION was trained on 980B tokens"
Parameters
Parameters
7000000000Authors
Lam Wen Zhi Clarence, Leong Wei Qi, Li Yier, Liu Bing Jie Darius, Lovenia Holy, Montalan Jann Railey, Ng Boon Cheong Raymond, Ngui Jian Gang, Nguyen Thanh Ngan, Ong Tat-Wee David, Rengarajan Hamsawardhini, Susanto Yosephine, Tai Ngee Chia, Tan Choon Meng, Teo Jin Howe, Teo Eng Sipp Leslie, Teo Wei Yi, Tjhi William, Yeo Yeow Tong, Yong Xianbin
