URL shortener
Design a service that turns long URLs into short links and redirects visitors. Assume 100M new links a month and 100 reads for every write.
Ask first
- Custom aliases? Do links expire?
- Do we need click analytics?
- Can links be edited or deleted?
- Is it a problem if codes are guessable?
Numbers
| writes | 100M a month, about 40 per second |
| reads | 100 x writes, about 4,000 per second, peaks maybe 10x |
| storage | 100M x 12 x 5 years x about 500 bytes, about 3 TB |
| code length | 62^7 is about 3.5 trillion, so 7 characters is plenty |
API
POST /urls { url, alias?, expiresAt? } 201 { code, shortUrl }
GET /{code} 302, Location: url
GET /urls/{code}/stats { clicks, ... }Data
urls(code PRIMARY KEY, long_url, owner_id, created_at, expires_at) clicks: events to a log (Kafka), counted in batches
Deep dive: How do you make the code?
A counter encoded in base62. The counter is a database sequence, or each server takes a block of a million ids at a time so servers never coordinate per request. No collisions, no retry loop. If codes must not be guessable, pass the counter through a reversible shuffle, or use random codes with a unique constraint and retry on the rare collision.
301 or 302?
301 is cached by browsers forever, so you stop seeing repeat clicks. Use 302 if analytics matter.
How do redirects stay fast at peak?
They are pure lookups of data that never changes. CDN or edge cache first, then Redis, then the database. Cache for a long time.
Counting clicks without a hot row?
Don't UPDATE a counter on every click. Write click events to a log and aggregate them in batches.
See the core idea run
Core logic (JavaScript, tested)
// A counter (DB sequence, or id blocks handed to each server) encoded in base62. No collisions, no retries.
const B62 = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";
function encode62(n) {
let s = "";
do {
s = B62[n % 62] + s;
n = Math.floor(n / 62);
} while (n > 0);
return s;
}
function decode62(s) {
let n = 0;
for (const c of s) n = n * 62 + B62.indexOf(c);
return n;
}Failures
| Redis is down | Reads fall through to database replicas. Size them for it, or shed load. |
| Sequence or id service is down | Servers already hold a block of ids, so writes continue for a while. |
| Abuse and phishing links | Scan URLs in the background, block lists, rate limit link creation. |
Trade-offs
- Counter codes are simple and collision free; random codes hide your volume.
- Strong consistency when creating a link, eventual consistency for stats.