The compute used to train AI has grown about 4.3× a year since 2010
Epoch AI tracks 1,102 notable AI models. For 546 of them it has an estimate of the training compute: the number of calculations used to train the model, counted in FLOP.
Before 2010, training compute grew about 1.5× a year, doubling roughly every 20 months.
Since 2010 it has grown about 4.3× a year, doubling roughly every six months. Each step up the scale is ten times more compute.
The largest training run on record is GPT-6 Astra (OpenAI, Sep 2026), at an estimated 9.4 × 10²⁶ FLOP.
Data: Epoch AI, CC BY 4.0.
Show the data as a table
| Model | Organization | Published | Training compute (FLOP) |
|---|---|---|---|
| GPT-6 Astra | OpenAI | Sep 3, 2026 | 9.4 × 10²⁶ |
| Grok 4 | xAI | Jul 9, 2025 | 5 × 10²⁶ |
| GPT-4.5 | OpenAI | Feb 27, 2025 | 3.8 × 10²⁶ |
| Grok 3 | xAI | Feb 17, 2025 | 3.5 × 10²⁶ |
| Gemini 1.0 Ultra | Google DeepMind | Dec 6, 2023 | 5 × 10²⁵ |
| GPT-4 (Mar 2023) | OpenAI | Mar 15, 2023 | 2.1 × 10²⁵ |
| Minerva (540B) | Jun 29, 2022 | 2.7 × 10²⁴ | |
| GPT-3.5 (davinci-002) | OpenAI | Mar 15, 2022 | 2.6 × 10²⁴ |
| FLAN 137B | Google Research | Sep 3, 2021 | 2 × 10²⁴ |
| Jurassic-1-Jumbo | AI21 Labs | Aug 11, 2021 | 3.7 × 10²³ |
| GPT-3 175B (davinci) | OpenAI | May 28, 2020 | 3.1 × 10²³ |
| Meena | Google Brain | Jan 28, 2020 | 1.1 × 10²³ |
| AlphaStar | DeepMind | Oct 30, 2019 | 1.1 × 10²³ |
| T5-11B | Oct 23, 2019 | 3.3 × 10²² | |
| Megatron-BERT | Nvidia | Sep 17, 2019 | 2.2 × 10²² |
| Megatron-LM (8.3B) | Nvidia | Sep 17, 2019 | 9.1 × 10²¹ |
| ResNeXt-101 32x48d | May 2, 2018 | 8.7 × 10²¹ | |
| GNMT | Sep 26, 2016 | 6.6 × 10²¹ | |
| AlphaGo Lee | DeepMind | Jan 27, 2016 | 1.9 × 10²¹ |
| AlphaGo Fan | DeepMind | Oct 1, 2015 | 3.8 × 10²⁰ |
| SNM-skip | Dec 3, 2014 | 3 × 10²⁰ | |
| Seq2Seq LSTM | Sep 10, 2014 | 5.6 × 10¹⁹ | |
| VGG16 | University of Oxford | Sep 4, 2014 | 1.2 × 10¹⁹ |
| VGG19 | University of Oxford | Sep 4, 2014 | 1.1 × 10¹⁹ |
| SPPNet | Microsoft, Xi’an Jiaotong University | Jun 18, 2014 | 3.4 × 10¹⁸ |
| DistBelief NNLM | Jan 16, 2013 | 2.6 × 10¹⁸ | |
| SB-LM | Jun 22, 2007 | 1.4 × 10¹⁸ | |
| Neural LM | University of Montreal / Université de Montréal | Nov 28, 2000 | 6.3 × 10¹⁵ |
| PoE MNIST | University College London (UCL) | Nov 28, 2000 | 5.2 × 10¹³ |
| LSTM | Technical University of Munich | Nov 15, 1997 | 3.2 × 10¹³ |
| Predictive Coding NN | Technical University of Munich | Dec 2, 1994 | 1.9 × 10¹³ |
| TD-Gammon | IBM | May 1, 1992 | 1.8 × 10¹³ |
| Zip CNN | AT&T, Bell Laboratories | Dec 1, 1989 | 1.5 × 10¹² |
| Handwritten digit recognition network | AT&T | Nov 27, 1989 | 1.8 × 10¹¹ |
| NetTalk (transcription) | Princeton University | Jun 6, 1987 | 2.8 × 10¹⁰ |
| Perceptron (1960) | Cornell Aeronautical Laboratory | Mar 30, 1960 | 7.2 × 10⁸ |
| Pandemonium (morse) | Massachusetts Institute of Technology (MIT) | Feb 1, 1959 | 6 × 10⁸ |
| Perceptron Mark I | Cornell Aeronautical Laboratory, Cornell University | Jan 1, 1957 | 6.9 × 10⁵ |
| Theseus | Bell Laboratories | Jul 2, 1950 | 4 × 10¹ |