Ecommerce Platform like Amazon or Flipkart

Functional Requirements

User-facing functionality:

Product search and browse.
Adding items to cart or wishlist.
Placing orders.
User profile and order tracking.
Notifications for updates (e.g., delivery status).

Backend functionality:

Inventory management.
Pricing and offers.
Order processing and serviceability checks.
Payment integration.
Shipping and logistics coordination.

Non-Functional Requirements:

High scalability (handle millions of users and requests).
Low latency for search, cart, and checkout flows.
High availability (no downtime).
Eventual consistency for inventory updates.
Secure payments and sensitive data handling.

In high-traffic systems, especially during events like major sales, balancing non-functional requirements such as availability, consistency, and low latency is challenging. Different system components can prioritize these requirements differently based on their purpose:

Payment and Inventory Systems: Emphasize high consistency to prevent errors, even at the cost of availability during peak loads.
Search System: Focus on high availability to ensure smooth user interactions, accepting slight inconsistencies as acceptable trade-offs.
User-Facing Components: Prioritize low latency for an optimal user experience, even if data consistency has minor delays.

By tailoring non-functional priorities to specific components, the system ensures efficient performance while handling high traffic effectively.

High level design:

Managing Suppliers with Inbound Services:
- Inbound Services interact with supplier systems to fetch inventory and item data.
- When a new supplier or item is added, this information flows through Kafka via events generated by Inbound Services.
- Consumers listening to Kafka process these events to update user-facing services like the homepage and search results.
Item Service:
- Consumes Kafka events to onboard new items.
- Exposes APIs for adding, updating, and fetching items.
- Uses MongoDB to store unstructured item data (e.g., shirts vs. TVs with different attributes).
Search Consumer:
- Processes newly onboarded items and stores them in ElasticSearch.
- ElasticSearch supports efficient text-based and fuzzy search for a seamless user experience.
Search Service:
- Interacts with ElasticSearch to expose APIs for filtering, sorting, and searching products.
- Communicates with Serviceability and TAT service to ensure search results only show deliverable items.
- Serviceability calculates routes, delivery eligibility, and estimated delivery dates.
Wishlist and Cart Services:
- Provide APIs to fetch, add, update, or delete items from wishlists or carts.
- Use separate MySQL databases for scalability, as wishlists and carts can grow large during sales.
Analytics and Recommendations:
- Kafka feeds events from searches, wishlists, and carts to Spark Streaming Consumers.
- Generates real-time insights like most popular products or categories.
- Stores data in Hadoop for running algorithms like ALS to create personalized recommendations.
- Recommendations are managed by a Recommendation Service and displayed based on user preferences or product categories.
User Service:
- Manages user data with APIs for fetching, updating, adding, or deleting user details.
- Uses Redis for caching and MySQL for persistent storage.
- For example, user pin codes are cached for faster access when calculating delivery routes.
Serviceability and TAT Service:
- Works with Logistic and Warehouse Services to compute delivery eligibility and ETA.
- Pre-computes and caches results to deliver instant responses to users.
Placing Orders:
- The flow starts with user searches and ends with verified deliverable items ready for order placement.
- All steps in the system ensure seamless product discovery, delivery eligibility, and user satisfaction.

Order Placement Flow

Order Placement:
- When a user wants to place an order, the request goes to the Order Taking Service (part of the Order Management System).
- The Order Management System uses a MySQL database to track various tables like customer, item, and order tables.
- ACID properties of MySQL are crucial to prevent issues like two users ordering the last item (e.g., last AirPods) simultaneously.
Order Record in Redis:
- As soon as the order is placed, an order record is created in Redis with an order ID, date, time, and expiry time (e.g., 10:01 to 10:05).
- The order status is initially marked as "created".
Inventory Update:
- The system calls the Inventory Service to update the stock. For example, if 5 Sony TVs were available, the count will be reduced to 4.
- Inventory is updated before proceeding to payment, even though the payment hasn't been completed yet, to prevent overselling.
- If stock is low, only valid buyers proceed to the payment stage.
Payment Flow:
- The Order Taking Service then calls the Payment Service, which processes the payment through the Payment Gateway.
- Three possible payment outcomes are handled:
  - Success: Update the order status to "placed" and trigger a Kafka event.
  - Failure: Update order status to "canceled", rollback inventory, and trigger a Kafka event.
  - No Response: Order expiry triggers the order status to "canceled" and inventory is restored. Expiry handled by Redis.
Race Condition Prevention:
- If payment success and order expiry occur around the same time, race conditions are avoided by deleting Redis records once payment is confirmed.
- If payment is successful but order expired (due to timing), options include either refunding the customer or completing the order and marking it as "placed".
Inventory Update for Search:
- Once an order is confirmed, the inventory count is updated, and items going out of stock are removed from search results to prevent further orders.
MySQL Database Scalability:
- MySQL becomes a bottleneck as order data grows over time.
- To optimize, delivered or canceled orders are moved from MySQL to Cassandra for better scalability.
- Order Processing Service and Historical Order Service manage the data migration process.
- Historical order data in Cassandra helps reduce load on MySQL while maintaining efficient queries.
Cassandra Use Case:
- Cassandra is used for large datasets with finite queries (e.g., fetching orders by ID, status, created date, etc.), making it suitable for orders that are completed or canceled.
Orders View:
- Users can view both ongoing and completed orders via the Orders View.
- An intermediate service communicates with the Order Processing Service and Historical Order Service to fetch order details.
Notifications:
- Every order update (e.g., order placed, in transit, or delivered) triggers a notification to either the seller or the customer.
- Notifications are handled by a dedicated Notification Service.
Analytics and Recommendations:
- Kafka collects events from the system, which are processed by Spark Streaming for real-time reports.
- Data is dumped into a Hadoop cluster for further analysis and running algorithms to generate personalized product recommendations.
- These recommendations are stored in the Recommendation Service and shown to users accordingly.

API Design

1. Inbound Services APIs

Fetch Supplier Inventory

GET /suppliers/{supplier_id}/inventory

Description: Fetch inventory data for a supplier.

      {
        "supplier_id": "123",
        "items": [
            { "item_id": "101", "stock": 20 },
            { "item_id": "102", "stock": 10 }
        ]
      }

Add New Item

POST /items

Description: Add a new item to the system.

      {
        "item_id": "103",
        "name": "Sony 65” TV",
        "category": "Electronics",
        "attributes": {
            "brand": "Sony",
            "resolution": "4K"
        }
      }

2. Item Service APIs

Get Item Details

GET /items/{item_id}

      {
        "item_id": "103",
        "name": "Sony 65” TV",
        "category": "Electronics",
        "stock": 5,
        "price": 999.99
      }

Update Item

PUT /items/{item_id}

      {
        "price": 899.99,
        "stock": 4
      }

3. Search Service APIs

Search Items

GET /search

Query Parameters: ?query=TV&sort=price&filter=category:Electronics

      [
        { "item_id": "101", "name": "Sony 65” TV", "price": 899.99 },
        { "item_id": "102", "name": "Samsung 55” TV", "price": 799.99 }
      ]

4. Wishlist and Cart APIs

Get Wishlist

GET /users/{user_id}/wishlist

      [
        { "item_id": "101", "name": "Sony 65” TV", "price": 899.99 }
      ]

Add to Wishlist

POST /users/{user_id}/wishlist

      { "item_id": "101" }

Get Cart

GET /users/{user_id}/cart

      [
        { "item_id": "101", "quantity": 1, "price": 899.99 }
      ]

Add to Cart

POST /users/{user_id}/cart

      { "item_id": "101", "quantity": 1 }

5. Order APIs

Place Order

POST /orders

      {
        "user_id": "u123",
        "items": [{ "item_id": "101", "quantity": 1 }]
      }

Get Order Details

GET /orders/{order_id}

      {
        "order_id": "o123",
        "user_id": "u123",
        "status": "placed",
        "items": [{ "item_id": "101", "quantity": 1 }],
        "total_price": 899.99
      }

6. Notification APIs

Send Notification

POST /notifications

      {
        "user_id": "u123",
        "type": "order_placed",
        "message": "Your order has been placed successfully!"
      }

7. Analytics and Recommendations APIs

Get Recommendations

GET /users/{user_id}/recommendations

      [
        { "item_id": "102", "name": "Samsung 55” TV", "price": 799.99 }
      ]

Database Schema

1. MySQL Database (Order Management)

Table: users

Column	Type	Description
user_id	VARCHAR(50)	Primary Key
name	VARCHAR(255)	User name
email	VARCHAR(255)	User email
password_hash	VARCHAR(255)	Encrypted password
phone	VARCHAR(15)	Contact number

Table: items

Column	Type	Description
item_id	VARCHAR(50)	Primary Key
name	VARCHAR(255)	Item name
category	VARCHAR(255)	Item category
price	DECIMAL(10,2)	Item price
stock	INT	Stock count
description	TEXT	Item description

Table: orders

Column	Type	Description
order_id	VARCHAR(50)	Primary Key
user_id	VARCHAR(50)	Foreign Key to users
status	ENUM('created', 'placed', 'canceled', 'delivered')	Order status
total_price	DECIMAL(10,2)	Total order price
created_at	TIMESTAMP	Order creation timestamp

Table: order_items

Column	Type	Description
order_item_id	VARCHAR(50)	Primary Key
order_id	VARCHAR(50)	Foreign Key to orders
item_id	VARCHAR(50)	Foreign Key to items
quantity	INT	Quantity of the item
price	DECIMAL(10,2)	Price of the item at the time of order

2. MongoDB (Item Details)

Collection: items

      {
        "_id": "item_id",
        "name": "Sony 65” TV",
        "category": "Electronics",
        "attributes": {
            "brand": "Sony",
            "resolution": "4K",
            "screen_size": "65 inches"
        },
        "stock": 10,
        "price": 999.99,
        "description": "A high-quality 4K TV with advanced features"
      }

3. ElasticSearch (Search Index)

Index: items

      {
        "item_id": "101",
        "name": "Sony 65” TV",
        "category": "Electronics",
        "price": 899.99,
        "stock": 5,
        "description": "A high-quality 4K TV with advanced features"
      }

4. Redis (Order Cache)

Key-Value Store:

      Key: order:o123
      Value: {
        "order_id": "o123",
        "user_id": "u123",
        "status": "created",
        "expiry_time": "2025-01-15T10:05:00Z"
      }

5. Hadoop (Analytics Data)

Schema Example:

      {
        "user_id": "u123",
        "search_query": "4K TV",
        "timestamp": "2025-01-15T10:01:00Z",
        "clicked_items": ["101", "102"]
      }