System Design: Back-of-the-envelope calculations

How long does it take to show a page of 30 image thumbnails? Jeff Dean asks this in his talk Software Engineering Advice from Building Large-Scale Distributed Systems¹1. Jeff Dean, Software Engineering Advice from Building Large-Scale Distributed Systems, Stanford CS295 talk.static.googleusercontent.com, and answers it in two lines.

• Design 1: read the images one after another, and make each thumbnail on the fly from a 256 KB image. That is 30 disk seeks at 10 ms each, plus 30 × 256 KB read at 30 MB/s: 30 × 10 ms + 30 × 256 KB / 30 MB/s = 560 ms.
• Design 2: issue the 30 reads in parallel. Now the page waits for only one seek and one read: 10 ms + 256 KB / 30 MB/s = 18 ms.

The second design is 30 times faster, and we know it before writing any code. That is a back-of-the-envelope calculation. In physics, dimensional analysis checks an equation before you trust it. In software, a rough estimate checks a design before you build it.

This post covers why to estimate, how to do it, which numbers to know, and six worked examples.

Why Estimate

An estimate has two uses.

Before you build, it chooses between designs. Dean’s two designs differ by a factor of 30. No amount of later tuning turns design 1 into design 2. An estimate also tells you when a design is impossible: if the disk cannot read fast enough, no code will fix it.

After you build, it finds the gap. Estimate what the system should do from the hardware’s limits. Then measure what it actually does. If the two are close, the system is near its limits. If the system is ten times slower than the estimate, either the estimate missed something or the system wastes work. Either way, you have found something worth knowing.

Simon Eskildsen, who wrote the napkin-math²2. Simon Eskildsen, Napkin Math, numbers measured March 2026.github.com project, makes this the point of his newsletter⁷7. Simon Eskildsen, Napkin Math newsletter.sirupsen.com: “I believe that first-principle thinking is required to break out of iterative improvement and make order of magnitude improvements.” Profiling tells you which part of the current system is slow. An estimate tells you how fast the system could be.

How to Estimate

Simon’s talk at SREcon19 is the best introduction to the skill:

His README lists four techniques. In my own words:

1. Break the question into quantities you can guess. This is Fermi decomposition. “What do our logs cost?” becomes: how many requests per second, how many bytes of log per request, and what a gigabyte of logs costs.
2. Use few assumptions. Simon’s limit is six. A calculation with more is usually harder than it needs to be, and each guess adds error.
3. Calculate with powers of ten. Write every number as c × 10^e. The goal is the right order of magnitude, which is e. The coefficient c matters much less.
4. Keep the units. If the units of the result are wrong, the calculation is wrong. Units are a free checksum.

A few conversions come up in almost every estimate:

Numbers to Know

An estimate is only as good as the numbers it starts from. There are three kinds: how long operations take, how much data moves per second, and what resources cost.

Latency: the 2012 table

This is the table that Dean and Peter Norvig made famous, collected by Jonas Bonér in Latency Numbers Every Programmer Should Know⁵5. Jonas Bonér, Latency Numbers Every Programmer Should Know.gist.github.com:

The exact values matter less than the ratios. A memory reference is 200 times an L1 hit. A round trip inside a data centre is 1,000 times a memory reference. A round trip across the Atlantic is 300 times that. Colin Scott’s interactive version⁶6. Colin Scott, Latency Numbers Every Programmer Should Know (interactive).colin-scott.github.io shows how each number has changed by year.

Throughput: the 2026 table

Hardware has moved since 2012, mostly in storage and networks. Simon re-measures his numbers on current cloud machines. The rows below come from his README, measured in March 2026. “1 GiB” is how long it takes to move a gigabyte at that rate.

Two changes stand out. A random SSD read is now 100 µs, and a disk seek of 10 ms is no longer on the critical path of most systems. And the network inside a zone is now about as fast as reading a local SSD sequentially, which is one reason storage moved to the network.

A database on one machine

How much can a single PostgreSQL server do? These are throughput numbers for a database with 100 million rows:

• Read-only: one query that reads one row from 100M rows.
• Read-write: one query, one insert and three updates.

Turn that into a daily number: the $300 machine does about 25,000 read-write transactions per second, or 2.5 × 10^4 × 10^5 = 2.5 × 10^9 per day. Few products have two billion writes a day.

Cost

Simon’s cost numbers, which he says are roughly the same across cloud providers:

Prices for the same resource still differ between vendors:

For more storage vendors, see S3-Compatible Cloud Storage Costs⁸8. transactional.blog, S3-Compatible Cloud Storage Costs, 2023.transactional.blog.

Machine learning

Machine learning has its own short list. Most estimates need only the numbers below.

One GPU. From Google’s How to Scale Your Model¹⁰10. Google DeepMind, How to Scale Your Model, 2025.jax-ml.github.io:

FP8 doubles the FLOPs. An H100 draws 700 W.

The last column is the most useful number in the table. It is FLOP/s divided by bandwidth: how many operations the GPU can do in the time it reads one byte from memory. An operation that does fewer operations per byte waits on memory. One that does more waits on compute. The scaling book puts it as a batch size: a BF16 matrix multiply on an H100 needs a batch of about 280 tokens before compute becomes the limit.

Compute. From EleutherAI’s Transformer Math 101¹¹11. EleutherAI, Transformer Math 101, 2023.blog.eleuther.ai and Kipply’s Transformer Inference Arithmetic¹²12. Kipply, Transformer Inference Arithmetic, 2022.kipp.ly, with P the number of parameters and D the number of training tokens:

For a mixture-of-experts model, P is the number of parameters active for each token, not the total.

Memory.

For Llama 3 70B (80 layers, 8 KV heads, head size 128), the KV cache is about 0.33 MB per token. For Llama 3 405B (126 layers), it is about 0.5 MB per token, so one 128K-token context needs about 68 GB.

Price and utilization. Renting an H100 cost about $3 per GPU-hour in October 2026. The Silicon Data index¹⁶16. Silicon Data, H100 Rental Price Index.silicondata.com read $2.81 on 15 October. Prices on hyperscalers are several times higher, and the market moves by 20% in a month. Large training runs use about 40% of the peak FLOPs. Llama 3 405B reached 38–43%.

Worked Examples

Sorting a gigabyte

Dean’s second example: how long does it take to quicksort 1 GB of 4-byte numbers on one CPU? He estimates two costs and adds them.

1. Branch mispredictions. 1 GB of 4-byte numbers is 2^28 numbers. Quicksort makes about log(2^28) = 28 passes, so about 2^33 comparisons. The comparisons are unpredictable, so about half mispredict: 2^32 × 5 ns = 21 s.
2. Memory bandwidth. Each pass streams the whole gigabyte: 2^30 bytes × 28 passes = 28 GB. At 4 GB/s, that is about 7 s.

Total: about 30 seconds. The estimate also says where the time goes. Mispredictions cost three times as much as memory traffic, so a sort that avoids branches should help more than one that saves memory. Dean’s next slide says what to do with the estimate: “Write microbenchmarks!”

The thumbnails again, in 2026

Dean’s thumbnail example assumed a disk seek of 10 ms. Redo design 1 with the 2026 table: a random SSD read takes 100 µs, and 256 KB at 8 GiB/s takes about 30 µs. Thirty serial reads now take 30 × 130 µs ≈ 4 ms.

The method gives the same answer as in 2012: parallel reads are still faster. But the difference now is 4 ms, which no user will notice. In 2012 it was more than half a second. The numbers age. Redo old estimates with current ones before you trust their conclusions.

Twitter on one machine

In Production Twitter on One Machine?⁴4. Tristan Hume, Production Twitter on One Machine? 100Gbps NICs and NVMe are fast, January 2023.thume.ca, Tristan Hume estimates whether one large server could serve Twitter’s core load. A few of his steps:

• Writes. 500 million tweets per day is 5 × 10^8 / 86,400 ≈ 5,800 tweets per second.
• Storage. An average tweet is 140 bytes of text and 36 bytes of metadata, 176 bytes in all. That is 88 GB per day, or 32 TB per year. Two days of tweets fit in RAM.
• Reads. 100 billion timeline impressions per day is about 1.2 million tweets delivered per second.

His conclusion is that the core timeline fits on one machine with room to spare. He benchmarked it, and wrote that “my estimate of the full average tweet delivery rate of Twitter is 35x less than what my 8 core laptop can fetch!” Images, video and search are what need more machines. He does not recommend building this way. The point is to see where the real load is.

His numbers also show a common mistake. The average is 5,800 tweets per second. He uses 9,100 per second for the sustained peak, and 150,000 per second for the maximum rate. Size a system for the peak, not the average. The peak can be 25 times the average.

What do our logs cost?

This is the question at the top of Simon’s README: what does logging cost for a service that handles 100,000 requests per second? Assume one log line of 1 KB per request.

• Volume. 10^5 requests/s × 10^3 bytes = 10^8 bytes/s, or 100 MB/s. Over a month: 10^8 × 2.5 × 10^6 s = 2.5 × 10^14 bytes, about 250 TB.
• In a hosted log service at $0.50 per GB: 2.5 × 10^5 GB × $0.50 ≈ $125,000 per month.
• In blob storage at $0.02 per GB: about $5,000 per month. Text logs compress about 4 times, which brings it to about $1,250.

The storage tier changes the bill by a factor of 100. But blob storage also charges per write. Write each log line as its own object, and the writes cost 10^5 × 2.5 × 10^6 = 2.5 × 10^11 writes × $5 per million, about $1.25 million per month. Batch the lines into 128 MB files, and the same data takes about 2 million writes: $10.

The whole calculation used three assumptions and five minutes. It shows that the expensive decision is not which log library to use, but where the logs go and how they are batched.

Training Llama 3 405B

Meta’s Llama 3 paper¹³13. Llama Team, Meta, The Llama 3 Herd of Models, 2024.arxiv.org gives enough numbers to check the formulas against each other.

• Compute. 405 billion parameters and 15.6 trillion tokens: 6 × 4 × 10^11 × 1.6 × 10^13 ≈ 3.8 × 10^25 FLOPs. The paper reports 3.8 × 10^25.
• Utilization. The model card¹⁴14. Meta, Llama 3.1 405B model card.huggingface.co gives 30.84 million H100-hours. At peak speed, that is 3.1 × 10^7 h × 3,600 s × 10^15 FLOP/s ≈ 1.1 × 10^26 FLOPs. The run used 3.8 × 10^25, about 35% of the peak. The paper reports 38–43% MFU. The two estimates agree.
• Cost. 31 million GPU-hours at $3 is about $90 million in rent.

Two more numbers follow from the same rules. Chinchilla’s rule gives 20 tokens per parameter. Llama 3 8B was trained on 15 trillion tokens, about 1,900 per parameter. Meta spent extra training compute once to get a smaller model that is cheaper to serve on every request. And DeepSeek-V3¹⁵15. DeepSeek-AI, DeepSeek-V3 Technical Report, 2024.arxiv.org has 671 billion parameters, but only 37 billion are active for each token. With 14.8 trillion tokens, 6 × 3.7 × 10^10 × 1.5 × 10^13 ≈ 3.3 × 10^24 FLOPs. A dense model of the same total size would need about 18 times more. Its paper reports 2.788 million H800-hours for the full training.

Generating tokens

How fast can one H100 generate text for one user? At batch size 1, every new token reads all the weights from memory once, and does only 2 FLOPs per parameter it reads. That is far below the ~295 FLOPs per byte the GPU needs to be compute-bound, so memory bandwidth sets the speed.

An 8B model in BF16 is 16 GB. Reading it takes 16 GB / 3.35 TB/s ≈ 5 ms. The limit is about 200 tokens per second for one user, however fast the GPU computes.

Batching changes the picture. Requests in the same batch share one read of the weights, so throughput grows with the batch until compute becomes the limit. For a 70B model at a large batch:

• Each token costs 2 × 7 × 10^10 = 1.4 × 10^11 FLOPs.
• An H100 at 40% utilization does about 4 × 10^14 FLOP/s, or about 2,800 tokens per second.
• That is about 10^7 tokens per hour. At $3 per hour, the compute costs about $0.30 per million tokens.

This is a lower bound for one GPU’s compute. It ignores memory for the KV cache, idle time and the prefill of long prompts. But it says where the cost of serving goes: into keeping the batch large.

Where Estimates Go Wrong

• Averages hide peaks. Hume’s maximum tweet rate is 25 times the average. Size for the peak, and know how big the peak is.
• Averages hide tails. Dean’s parallel design waits for the slowest of 30 reads, not the average one. His slide says the estimate “ignores variance, so really more like 30-60 ms, probably”, not 18 ms.
• Numbers age. The 2012 table says a disk seek costs 10 ms. Most systems no longer have one. Use current numbers.
• Hardware is not the only limit. A database’s write rate is often limited by fsync, not by the CPU: at 300 µs per fsync, one disk does about 3,000 synced writes per second, unless writes are grouped into one fsync.

When an estimate and a measurement disagree, do not average them. Find out which one is wrong. That is where the learning is.

Practice

The best way to learn is to estimate your own systems, then measure them. The second-best way is Simon’s newsletter⁷, which sends a problem every few weeks and the solution in the next issue.

Other lists of numbers:

• Latency Numbers Every Programmer Should Know⁵, Jonas Bonér
• Napkin Math², Simon Eskildsen
• Numbers every LLM Developer should know⁹9. Anyscale, Numbers every LLM Developer should know.github.com, Anyscale

References

1. Jeff Dean, Software Engineering Advice from Building Large-Scale Distributed Systems, Stanford CS295 talk.
2. Simon Eskildsen, Napkin Math, numbers measured March 2026.
3. Simon Eskildsen, Advanced Napkin Math: Estimating System Performance from First Principles, SREcon19 EMEA.
4. Tristan Hume, Production Twitter on One Machine? 100Gbps NICs and NVMe are fast, January 2023.
5. Jonas Bonér, Latency Numbers Every Programmer Should Know.
6. Colin Scott, Latency Numbers Every Programmer Should Know (interactive).
7. Simon Eskildsen, Napkin Math newsletter.
8. transactional.blog, S3-Compatible Cloud Storage Costs, 2023.
9. Anyscale, Numbers every LLM Developer should know.
10. Google DeepMind, How to Scale Your Model, 2025.
11. EleutherAI, Transformer Math 101, 2023.
12. Kipply, Transformer Inference Arithmetic, 2022.
13. Llama Team, Meta, The Llama 3 Herd of Models, 2024.
14. Meta, Llama 3.1 405B model card.
15. DeepSeek-AI, DeepSeek-V3 Technical Report, 2024.
16. Silicon Data, H100 Rental Price Index.