>
I
Introduction
Large language models (LLMs) based on the Transformer architecture have been shown to possess a range of advanced capabilities in language generation and understanding. These capabilities have paved the way for deployment of LLMs in products like OpenAI’s Chat-GPT and Google’s Bard. At Inflection AI, our mission is to create personal AIs for everyone, and in May 2023 we released Pi (pi.ai) – an LLM-based personal AI which is designed to be empathetic, useful, and safe. In this work we introduce the foundation model powering Pi, dubbed Inflection-1, and evaluate its performance characteristics across a variety of benchmarks.
Benchmarking
FLOPs
1e+24Notes: <= 2.5e24 They define two "compute classes", one for models with more compute than PaLM 540B, i.e. GPT-4 and PaLM 2, and one for models with as much compute or less, i.e. GPT-3.5, Chinchilla, LLaMA, and Inflection-1. PaLM 540B required 2.5e24 FLOP to train (confirmed by Google)
Training
Hardware
NVIDIA H100 SXM5 80GB