Non-functional Requirements

• Minimizing Latency
  • Geographically distributed cache management systems and servers
  • CDNs (Content Delivery Networks)
• Consistency
  • Provide unique IDs to messages using Sequencer or other mechanisms
  • Use FIFO messaging queue with strict ordering
• Availability
  • Provide multiple WebSocket servers and managers to establish connections between users
  • Replication of messages and data associated with users and groups on different servers
  • Follow disaster recovery protocols
• Security
  • Via end-to-end encryption
• Scalability
  • Performance tuning of servers
  • Horizontal scalability of services

1. Why do we need a WebSocket Manager when WebSocket servers can talk to each other?

1. Trap: Interviewer checks if you understand scaling and mapping of billions of connections.
2. Answer: Without a centralized manager, servers don’t know which user is connected to which server. A WebSocket Manager (with Redis/Consistent Hashing) efficiently maintains user-to-server mapping, enabling quick message routing. Direct server-to-server queries would add latency and complexity at scale.

---

2. How do you ensure message ordering across servers?

1. Trap: Interviewer wants to see if you rely only on Kafka or think about sequencing.
2. Answer: Use a sequencer (centralized or distributed with logical clocks) to assign increasing message IDs. Kafka preserves ordering within a partition, so partition messages by (chatId or groupId). This ensures strict FIFO order for each conversation or group.

---

3. What happens if the WebSocket Manager itself fails?

1. Trap: Checks if you thought about single point of failure.
2. Answer: WebSocket Manager must be replicated with leader-election (e.g., Raft/ZooKeeper/Etcd). Clients cache recent mappings, so even if the manager is temporarily unavailable, servers can still route messages until recovery.

---

4. How do you scale group chats with millions of users (like WhatsApp Broadcast)?

1. Trap: They want to test group fan-out design.
2. Answer: Don’t push to every user directly from one server. Instead:
   • Use Kafka (or Pulsar) for fan-out via partitions.
   • Each user’s WebSocket server consumes messages relevant to its connected users.
   • For very large groups, use sharding + multicast trees to reduce fan-out cost.

---

5. Why store messages in MySQL and not just Kafka?

1. Trap: Checks if you confuse persistence with pub-sub.
2. Answer: Kafka is a log, not a long-term store. MySQL (or Cassandra) provides durable, queryable history for compliance, replay, and recovery. Kafka retains only for short TTL, while MySQL ensures permanent storage (e.g., 30 days or more).

---

6. How do you deliver messages to offline users?

1. Trap: Interviewer checks push notification handling.
2. Answer: Store undelivered messages in durable DB (MySQL/Cassandra). When the user reconnects, fetch from DB and deliver. If still offline, trigger push notification via APNS/FCM. Expire after policy (30 days).

---

7. How do you achieve low latency across continents?

1. Trap: Latency ≠ just WebSocket optimization.
2. Answer: Deploy geo-distributed WebSocket servers. Use GeoDNS / Anycast for nearest-region routing. Redis clusters and Kafka clusters replicated across regions (with conflict resolution) ensure messages stay close to users.

---

8. How do you ensure exactly-once delivery?

1. Trap: Messaging systems usually give at-least-once or at-most-once.
2. Answer: Use idempotent message IDs + deduplication at receiver side. Store delivery receipts. If a duplicate arrives, ignore based on messageId. This simulates exactly-once semantics on top of at-least-once infra.

---

9. Why use Redis for caching user-server mappings?

1. Trap: Checks if you know alternatives.
2. Answer: Redis provides in-memory, distributed, and fast lookups (sub-ms). Alternatives like Consistent Hashing or gossip protocols exist, but Redis supports TTLs, pub-sub, and replication, making it ideal.

---

10. How do you handle media messages differently from text?

1. Trap: They check if you know bandwidth/storage issues.
2. Answer: Media files are uploaded to Blob storage/CDN. Only references (fileId, hash) are shared via WebSocket/DB. This avoids duplicating heavy payloads in message queues or DB.

---

11. How do you prevent one slow consumer in a group from delaying others?

1. Trap: Classic backpressure problem.
2. Answer: Use Kafka consumer groups per user. Each user’s queue is independent. Slow consumer does not affect others because partition offset is tracked separately.

---

12. How do you ensure security (end-to-end encryption) with this architecture?

1. Trap: Tests if you confuse encryption in transit with E2E.
2. Answer: Messages are encrypted on sender’s device using recipient’s public key (Signal Protocol). WebSocket servers, Kafka, MySQL only see ciphertext. Only recipient’s device can decrypt, ensuring E2E security.

History

In January 2008, Jan Koum, who worked at Yahoo, tried to get a job at Facebook but didn’t make it. Instead of giving up, he kept going. The next year, he got an iPhone and saw how powerful the new App Store could be. This gave him an idea. Along with some old friends from Yahoo, he created WhatsApp, a simple app to send messages without the heavy costs of SMS. WhatsApp became super popular, with a million people signing up every single day!

---

Tech Stack:

• WhatsApp uses Erlang> for its core servers because it's built for high scalability with a tiny footprint.
• Erlang supports hot-loading, allowing code updates without restarting servers or redirecting traffic, ensuring nearly 100% uptime.
• Unlike Java or C++, Erlang’s threads are lightweight and managed by the language itself, making context switching much cheaper and efficient.
• Operating System: WhatsApp runs on FreeBSD servers, chosen for stability and performance.
• Message Protocol: Uses a customized, lightweight version of XMPP optimized for instant delivery and low latency.
• Databases: Distributed Erlang-based databases (like Mnesia) store messages reliably with replication and sharding for scalability.
• Infrastructure: Initially hosted on SoftLayer, now on Facebook’s data centers for better scalability.
• Security: Employs end-to-end encryption to keep user messages private.
• Cross cutting concerns : They focus on this aread more, Cross-cutting concerns are things that affect many parts of a product. And are hard to separate. e.g Security, Logging, Alerting, Performance, Exception_handling, Monitoring

---

Don’t reinvent the wheel everytime:

Ejabberd is an open-source real-time messaging server written in Erlang. And they built WhatsApp on top of ejabberd. Also they rewrote some of the ejabberd core components to meet their needs.
Besides WhatsApp leveraged third-party services such as Google Push to provide push notifications.

---

Scalability

• WhatsApp used diagonal scaling to keep the costs and operational complexity low.
• Diagonal scaling is a hybrid of horizontal and vertical scaling. The computing resources get added both vertically and horizontally.
• They ran WhatsApp servers on the FreeBSD operating system. Because they had previous experience with FreeBSD while working at Yahoo.
• Besides FreeBSD offered a reliable network stack.
• They fine-tuned FreeBSD to accommodate 2 million+ connections per server. And modified kernel parameters such as files and sockets.
• They overprovisioned servers to handle sudden traffic spikes and keep headroom for failures. For example, failures such as network partitions or hardware faults.

---

They kept the team size small - 32 engineers.

WhatsApp is one of the most successful instant messengers in the market. In 2014, Facebook acquired WhatsApp for a whopping 19 billion USD. According to Forbes, Jan Koum has a net worth of 14 billion USD in 2023.

This lean yet powerful technology stack lets WhatsApp handle billions of messages daily with great speed, reliability, and uptime.