Chat and real-time messaging0/59 done
Part 3 · System design · Design problems

Chat and real-time messaging

One to one and group chat, delivered to online users in real time, with history and sync for users who were offline.

Ask first

  • Largest group size?
  • Read receipts, typing indicators?
  • Keep history forever?
  • End to end encryption?
Numbers
connectionssay 1M online; one gateway holds about 50k WebSockets, so about 20 gateways
messages100M a day, about 1,200 per second, peaks 10x
storage100M x 200 bytes, about 20 GB a day
API
WebSocket send    { convId, clientMsgId, body }   ack { seq }
WebSocket receive { convId, seq, from, body }
GET /conversations/{id}/messages?afterSeq=...    (sync after reconnect)
Data
messages(conv_id, seq, sender_id, body, created_at)  PRIMARY KEY (conv_id, seq)
presence: user to gateway server, in Redis with a heartbeat TTL
Deep dive: How do you keep order?

Per conversation, not globally. One sequence counter per conversation (Redis INCR, or a single owner per conversation). Clients sort by seq.

How do you avoid duplicates?

The client sends a clientMsgId and the server dedupes on it, so resending after a lost ack doesn't post twice.

How does a message reach the right server?

Look up the recipient's gateway in the presence map and forward via pub/sub. If they are offline, store it and send a push notification.

What happens on reconnect?

The client sends the last seq it saw and gets everything after it. A gap in seq numbers tells the client something is missing.

See the core idea run

Core logic (JavaScript, tested)
// The server stamps a per-conversation sequence number. Clients may get messages late, twice, or out of order
// (reconnects, retries), so they buffer and show them in seq order.
class InOrderInbox {
  constructor() {
    this.next = 1;
    this.buf = new Map();
  }
  receive(msg) {
    if (msg.seq < this.next || this.buf.has(msg.seq)) return []; // duplicate
    this.buf.set(msg.seq, msg);
    const ready = [];
    while (this.buf.has(this.next)) {
      ready.push(this.buf.get(this.next));
      this.buf.delete(this.next++);
    }
    return ready; // show these now, in order
  }
  gap() {
    // the range to fetch from the server: GET /messages?afterSeq=next-1
    return this.buf.size ? [this.next, Math.min(...this.buf.keys()) - 1] : null;
  }
}
Failures
A gateway diesClients reconnect to another one and sync from their last seq.
Pub/sub drops a messageThe client sees a seq gap and fetches the missing range.
A huge groupFan out through a queue, not inline; very large channels are pulled when opened.
Trade-offs
  • WebSockets for real time, long polling as a fallback.
  • Order per conversation is cheap. Global order is expensive and nobody needs it.
Say out loud: WebSocket gateways with a presence map in Redis. Each conversation has its own sequence number, so clients order by seq, dedupe by client message id, and fill gaps after a reconnect by asking for everything after the last seq they saw.