How fast is AI training compute growing?
AI training compute has doubled about every six months since 2010, roughly four times more each year. Before deep learning took off, it doubled about every 21 months. Epoch AI puts the recent pace at 4 to 5 times a year, and the largest training runs in 2026 used close to a thousand septillion operations, about 45 times GPT-4.
Training compute is the total number of calculations a computer does to train an AI model, counted in floating-point operations (FLOP). As the chart by Our World in Data shows, the amount used by notable AI systems has climbed from about 40 FLOP for Claude Shannon’s maze-solving mouse Theseus in 1950 to tens of septillions for the large language models of 2023 [1][2]. Our World in Data notes that Google’s Gemini 1.0 Ultra, released 11 years after the AlexNet image recognition breakthrough of 2012, used 100 million times more training computation [1].
AI training compute growth by era
The growth rate changed sharply around 2010, when deep learning started to work at scale. Before then, growth roughly tracked cheaper and faster computer chips. Since then, developers have also spent far more money on bigger training runs, so compute has grown much faster than the hardware has improved.
| Period | Doubling time | Growth per year | Source |
|---|---|---|---|
| 1950 to 2010 | About 21 months | About 1.5 times | Our World in Data [1] |
| 2010 onwards | About 6 months | About 4 times | Our World in Data [1] |
| 2010 to May 2024, notable models | About 6 months | 4.1 times | Epoch AI [3] |
| Frontier models since 2018 | About 6 months | About 4 times | Epoch AI [3] |
The growth per year figures follow from the doubling times. Doubling every six months means two doublings a year, which is four times more. Epoch AI, which keeps the database the chart uses, recommends 4 to 5 times a year as the best summary of recent growth for notable and frontier models, and as the baseline for what to expect next [3]. Our World in Data’s longer article on scaling AI rounds the pre-2010 pace to roughly every two years [4].
Training compute of well-known AI systems
These estimates come from Epoch AI’s notable models database, the data behind the chart [2]. Most are estimated from published details about the hardware and training time rather than reported by the developer, and Epoch rates each one from confident to speculative.
| System | Year | Training compute (FLOP) | In words |
|---|---|---|---|
| Theseus | 1950 | 40 | Forty |
| Perceptron Mark I | 1957 | About 700,000 | Under a million |
| TD-Gammon | 1992 | 1.8 × 10¹³ | 18 trillion |
| AlexNet | 2012 | 4.7 × 10¹⁷ | 470 quadrillion |
| AlphaGo Zero | 2017 | 6.5 × 10²⁰ | 650 quintillion |
| GPT-3 | 2020 | 3.1 × 10²³ | 310 sextillion |
| GPT-4 | 2023 | 2.1 × 10²⁵ | 21 septillion |
| Gemini 1.0 Ultra | 2023 | 5 × 10²⁵ | 50 septillion |
| Llama 3.1 405B | 2024 | 3.8 × 10²⁵ | 38 septillion |
| GPT-4.5 | 2025 | 3.8 × 10²⁶ | 380 septillion |
| Grok 4 (speculative estimate) | 2025 | 5 × 10²⁶ | 500 septillion |
So from AlexNet in 2012 to Grok 4 in 2025, the compute used for a leading model rose about a billion times.
The largest AI training runs in 2026
The biggest model in Epoch AI’s database as of 13 November 2026 is OpenAI’s GPT-6 Astra, released in September 2026, at an estimated 9.4 × 10²⁶ FLOP, or 940 septillion [2]. Epoch bases that on a cluster of at least 100,000 Nvidia GB200 chips in Abilene, Texas, running for about 90 days, and gives a range of roughly 5 × 10²⁶ to 2 × 10²⁷ FLOP [2]. Next come Grok 4 (5 × 10²⁶), GPT-4.5 (3.8 × 10²⁶) and Grok 3 (3.5 × 10²⁶) [2].
Epoch counts 26 notable models trained with more than 10²⁵ FLOP, all released from 2023 onwards, including eight in 2026 so far [2]. That level matters because the EU AI Act presumes that a general-purpose AI model has high-impact capabilities, and so may carry systemic risk, when its training used more than 10²⁵ floating point operations [5]. These rankings change every few months as new models are released.
What a FLOP is and how big a septillion is
A FLOP is one floating-point operation, a single sum such as a multiplication of two decimal numbers. Training compute is the total count of them, which runs into numbers far beyond everyday use. The chart uses the short scale names used in the US and UK, where each step up is a thousand times bigger.
| Name | Power of ten | Written out |
|---|---|---|
| Trillion | 10¹² | 1,000,000,000,000 |
| Quadrillion | 10¹⁵ | 1 followed by 15 zeros |
| Quintillion | 10¹⁸ | 1 followed by 18 zeros |
| Sextillion | 10²¹ | 1 followed by 21 zeros |
| Septillion | 10²⁴ | 1,000,000,000,000,000,000,000,000 |
| Octillion | 10²⁷ | 1 followed by 27 zeros |
Our World in Data compares one septillion to the estimated number of stars in the universe [1]. Its dataset counts compute in petaFLOP, where one petaFLOP is 10¹⁵ operations, so GPT-4’s 2.1 × 10²⁵ FLOP is about 21 billion petaFLOP [2][6].
Why AI training compute keeps growing
Training compute grows because bigger training runs have kept producing more capable models, and companies have been willing to pay for them. Epoch AI estimates that about two thirds of the improvement in language model performance over the past decade came from larger scale [3].
- Hardware gets cheaper. Our World in Data notes that the computing speed of the GPUs used for training doubles roughly every 2.5 years per dollar spent [4]. That is much slower than the six-month doubling of compute, so most of the growth comes from buying and running more chips for longer.
- Spending has soared. Epoch estimates the compute for AlexNet cost about $16 in 2023 dollars, GPT-4 about $37 million and GPT-4.5 about $366 million [2].
- Models and datasets have grown alongside compute. Since 2010 training data has doubled about every nine to ten months and the number of model parameters about every year, according to Our World in Data [4].
Our World in Data also notes that companies now use more compute when a model answers a question, not only during training [4]. The training figures here do not include that.
Questions about AI training compute growth
How much is 1 septillion?
One septillion is 10²⁴, a 1 followed by 24 zeros, or a million million million million. Our World in Data compares it to the estimated number of stars in the universe [1]. GPT-4 used about 21 septillion FLOP to train [2].
What comes after septillion?
Octillion, which is 10²⁷ or a thousand septillion. The largest AI training run in Epoch AI’s database, GPT-6 Astra, is estimated at just under one octillion FLOP [2].
What does quintillion mean?
A quintillion is 10¹⁸, a 1 followed by 18 zeros, or a billion billion. AlexNet used about half a quintillion FLOP in 2012 [2].
How much compute was used to train GPT-4?
Epoch AI estimates 2.1 × 10²⁵ FLOP, on about 25,000 Nvidia A100 chips, at a compute cost of about $37 million in 2023 dollars [2]. OpenAI has not published the figure.
How much compute did Gemini Ultra use compared with AlexNet?
About 100 million times more. Gemini 1.0 Ultra is estimated at 5 × 10²⁵ FLOP against 4.7 × 10¹⁷ for AlexNet [1][2]. Our World in Data puts the compute for Gemini 1.0 at roughly what 50,000 high-end graphics cards would do working nonstop for a year [1].
Will AI training compute keep growing at this rate?
Epoch AI uses 4 to 5 times a year as its baseline expectation, before allowing for possible bottlenecks or speed-ups [3]. Each new run costs more than the last, so that pace depends on companies continuing to pay for it.
Sources
- Our World in Data, Charlie Giattino and Veronika Samborska, Since 2010, the training computation of notable AI systems has doubled every six months, 21 January 2025. The original page for the infographic.
- Epoch AI, Data on AI models, notable AI models dataset updated 9 November 2026.
- Epoch AI, Training compute of frontier AI models grows by 4-5x per year, 2024.
- Our World in Data, Veronika Samborska, Scaling up: how increasing inputs has made artificial intelligence more capable.
- EU Artificial Intelligence Act, Article 51, Classification of general-purpose AI models with systemic risk.
- Our World in Data, Computation used to train notable artificial intelligence systems.
Last reviewed 13 November 2026

