Part 3 · System design · Design problems
Chat and real-time messaging
One to one and group chat, delivered to online users in real time, with history and sync for users who were offline.
Ask first
- Largest group size?
- Read receipts, typing indicators?
- Keep history forever?
- End to end encryption?
Numbers
| connections | say 1M online; one gateway holds about 50k WebSockets, so about 20 gateways |
| messages | 100M a day, about 1,200 per second, peaks 10x |
| storage | 100M x 200 bytes, about 20 GB a day |
API
WebSocket send { convId, clientMsgId, body } ack { seq }
WebSocket receive { convId, seq, from, body }
GET /conversations/{id}/messages?afterSeq=... (sync after reconnect)Data
messages(conv_id, seq, sender_id, body, created_at) PRIMARY KEY (conv_id, seq) presence: user to gateway server, in Redis with a heartbeat TTL
Deep dive: How do you keep order?
Per conversation, not globally. One sequence counter per conversation (Redis INCR, or a single owner per conversation). Clients sort by seq.
How do you avoid duplicates?
The client sends a clientMsgId and the server dedupes on it, so resending after a lost ack doesn't post twice.
How does a message reach the right server?
Look up the recipient's gateway in the presence map and forward via pub/sub. If they are offline, store it and send a push notification.
What happens on reconnect?
The client sends the last seq it saw and gets everything after it. A gap in seq numbers tells the client something is missing.
See the core idea run
Core logic (JavaScript, tested)
// The server stamps a per-conversation sequence number. Clients may get messages late, twice, or out of order
// (reconnects, retries), so they buffer and show them in seq order.
class InOrderInbox {
constructor() {
this.next = 1;
this.buf = new Map();
}
receive(msg) {
if (msg.seq < this.next || this.buf.has(msg.seq)) return []; // duplicate
this.buf.set(msg.seq, msg);
const ready = [];
while (this.buf.has(this.next)) {
ready.push(this.buf.get(this.next));
this.buf.delete(this.next++);
}
return ready; // show these now, in order
}
gap() {
// the range to fetch from the server: GET /messages?afterSeq=next-1
return this.buf.size ? [this.next, Math.min(...this.buf.keys()) - 1] : null;
}
}Failures
| A gateway dies | Clients reconnect to another one and sync from their last seq. |
| Pub/sub drops a message | The client sees a seq gap and fetches the missing range. |
| A huge group | Fan out through a queue, not inline; very large channels are pulled when opened. |
Trade-offs
- WebSockets for real time, long polling as a fallback.
- Order per conversation is cheap. Global order is expensive and nobody needs it.
Say out loud: WebSocket gateways with a presence map in Redis. Each conversation has its own sequence number, so clients order by seq, dedupe by client message id, and fill gaps after a reconnect by asking for everything after the last seq they saw.