System Design: Back-of-the-envelope calculations

In physics, dimensional analysis is a go-to heuristic for checking equations. In computer science, back-of-the-envelope calculations serve a similar purpose for rough estimations. Jeff Dean emphasised the importance of mastering this skill for every programmer in his talk Software Engineering Advice from Building Large-Scale Distributed Systems. Why do we need back-of-the-envelope calculations? The calculation allow us to quickly assess the physical limitations of a running system. If there has a sigficant gap between our expectations(theoretical limitation) and the actual cost, there might be an opportunity to optimize our systems.

Table of Contents

Performance

Latency Comparison Numbers (~2012)

PostgreSQL Throughput

• RO: read only query(one row from 100M rows)
• RW: read write query(one query, one insert, three updates)

Cost Estimation

As an architecture, what we are concerned with should not only focus on performance but also consider cost. Below are the computing costs and storage costs for the cloud-hosted service.

Base Costs:

• CPU: $10/Core/month
• Memory: $2-3/GB/month
• SSD: $0.10/GB/month
• S3: $0.02/GB/month
• HDD: $0.01 - $0.05/GB/month
• Network (egress, between zones or regions): $0.05 - $0.10/GB/month

Please note: For hardware, Reserved pricing is typically about half the cost of On-Demand Pricing.

Compute

PostgreSQL Storage

S3 Storage

For more vendor pricing, please refer to S3-Compatible Cloud Storage Costs.

The Power of 2

Example

• https://thume.ca/2023/01/02/one-machine-twitter/, a excellent example of back-of-the-envelope calculations for if you want to build a Twitter-like service.

Other Lists

• Numbers every LLM Developer should know
• Numbers every programmer should know

Reference

• https://github.com/sirupsen/napkin-math
• https://www.youtube.com/watch?v=IxkSlnrRFqc
• https://deploy.equinix.com/product/servers/c3-small/