Training compute, to scale

The long descent

Every model below sits at the number of floating-point operations its training took, on a log scale: each step down the ruler is ten times more compute. It starts with a 1950 maze-solving machine at 4.0 × 10¹ operations and reaches today’s largest estimate, 10²⁷, about 25 orders of magnitude later.

Every figure is an estimate from Epoch AI, “Data on AI models” (notable AI models) (CC BY 4.0), with Epoch’s own confidence label. Figures for recent closed models are outside estimates, not numbers their makers published. Evolving Data as of 8 October 2026; checked 10 October 2026.

Remember the scale is logarithmic: the last few steps hold far more computation than everything above them.

1950 · Bell Laboratories
Theseus
4.0 × 10¹ FLOP · Maze solving
Epoch’s confidence: confident
1960 · Stanford University
ADALINE
6.6 × 10³ FLOP · Pattern recognition
Epoch’s confidence: confident
1957 · Cornell Aeronautical Laboratory
Perceptron Mark I
7.0 × 10⁵ FLOP · Binary classification
Epoch’s confidence: likely · Chapter 4: Neural Networks
1980 · NHK Broadcasting Science Research Laboratories
Neocognitron
2.7 × 10⁸ FLOP · Character recognition
Epoch’s confidence: confident · Chapter 5: Vision, Speech & Reinforcement Learning
1959 · IBM
Samuel Neural Checkers
4.3 × 10⁸ FLOP · Checkers
Epoch’s confidence: likely · Chapter 1: What Is Artificial Intelligence?
1987 · Princeton University
NetTalk
2.8 × 10¹⁰ FLOP · Speech synthesis (transcription)
Epoch’s confidence: confident
1998 · AT&T
LeNet-5
2.8 × 10¹² FLOP · Character recognition
Epoch’s confidence: confident
1992 · IBM
TD-Gammon
1.8 × 10¹³ FLOP · Backgammon
Epoch’s confidence: speculative
2013 · DeepMind
DQN
2.8 × 10¹⁵ FLOP · Atari games
Epoch’s confidence: confident · Chapter 5: Vision, Speech & Reinforcement Learning
2013 · Google
Word2Vec (large)
3.9 × 10¹⁶ FLOP · Semantic embedding
Epoch’s confidence: likely · Chapter 6: Language Before Transformers
2012 · University of Toronto
AlexNet
4.7 × 10¹⁷ FLOP · Image classification
Epoch’s confidence: confident · Chapter 5: Vision, Speech & Reinforcement Learning
2017 · Google Research, Google Brain
Transformer
7.4 × 10¹⁸ FLOP · Translation
Epoch’s confidence: confident · Chapter 7: Transformers
2014 · Google
Seq2Seq LSTM
5.6 × 10¹⁹ FLOP · Translation
Epoch’s confidence: confident · Chapter 6: Language Before Transformers
2018 · Google
BERT-Large
2.9 × 10²⁰ FLOP · Language understanding
Epoch’s confidence: confident · Chapter 7: Transformers
2017 · DeepMind
AlphaGo Zero
6.5 × 10²⁰ FLOP · Go
Epoch’s confidence: confident
2019 · OpenAI
GPT-2 (1.5B)
1.9 × 10²¹ FLOP · Language modelling
Epoch’s confidence: speculative · Chapter 8: The Rise of Large Language Models
2020 · OpenAI
GPT-3 175B
3.1 × 10²³ FLOP · Language modelling
Epoch’s confidence: confident · Chapter 8: The Rise of Large Language Models
2022 · DeepMind
Chinchilla
5.8 × 10²³ FLOP · Language modelling
Epoch’s confidence: confident · Chapter 9: How an LLM Is Actually Built
2022 · Google Research
PaLM (540B)
2.5 × 10²⁴ FLOP · Language modelling
Epoch’s confidence: confident · Chapter 9: How an LLM Is Actually Built
2024 · DeepSeek
DeepSeek-V3
3.3 × 10²⁴ FLOP · Language modelling
Epoch’s confidence: confident · Chapter 11: Inside Modern LLMs
2023 · OpenAI
GPT-4
2.1 × 10²⁵ FLOP · Language and vision
Epoch’s confidence: likely · Chapter 8: The Rise of Large Language Models
2024 · Meta AI
Llama 3.1 405B
3.8 × 10²⁵ FLOP · Language modelling
Epoch’s confidence: confident · Chapter 9: How an LLM Is Actually Built
2025 · xAI
Grok 3
3.5 × 10²⁶ FLOP · Language and vision
Epoch’s confidence: likely
2026 · OpenAI
GPT-6 Astra
10²⁷ FLOP · Language and vision
Epoch’s confidence: likely