Designing a URL Shortener Service

---

A URL shortener service converts long URLs into shorter, unique aliases that redirect to the original URLs. This is particularly useful for sharing links on platforms with character limits or for creating cleaner, more manageable links.

---

Functional Requirements:

• Generate a unique short URL for a given long URL.
• Redirect users to the original URL when the short URL is accessed.
• Allow users to customize their short URLs (optional).
• Support link expiration where URLs are inaccessible after a certain period.
• Provide analytics on link usage (optional).

---

Non-Functional Requirements:

• High availability (99.9% uptime).
• Low latency (operations should complete in milliseconds).
• Scalability (handle millions of requests per day).
• Durability (shortened URLs should remain functional for years).
• Security to prevent malicious use, such as phishing.

---

---

3. High-Level Design

The system comprises several components:

• Load Balancer: Handles incoming requests for URL shortening and redirection.
• Application Servers: Process business logic for creating and retrieving URLs.
• Database: Stores mappings between short URLs and original URLs.
• Cache: Speeds up retrieval of frequently accessed URLs.
• Analytics Service: Collects data on link usage (if implemented).

---

4. Database Design

We Need to consider following points for database design:

• We need to store billions of records.
• Most database operations are simple key-value lookups.
• Read queries are much higher than write queries.
• We don’t need joins between tables.
• Database needs to be highly scalable and available.

A NoSQL database like DynamoDB or Cassandra is ideal because it efficiently handles billions of key-value lookups while offering high scalability and availability.

---

5. API Design

We will use REST API

1. Endpoint: POST /api/v1/shorten
This endpoint generates a shortened URL for a provided long URL.
• Sample Request:

  {
    "long_url": "https://example.com/very/long/url/that/needs/shortening",
    "custom_alias": "",
    "expiry_date": "2024-12-31T23:59:59Z", // optional
    "user_id": "user123"
  }            
          

• Sample Response:

  {
    "short_url": "http://short.url/abC123",
    "long_url": "https://example.com/very/long/url/that/needs/shortening",
    "expiry_date": "2024-12-31T23:59:59Z",
    "created_at": "2024-08-10T10:30:00Z"
  }            
          

2. Endpoint: GET /{short_url_key}
This endpoint redirects the user to the original long URL.jhmng

---

5. URL Generation Strategy

We'll consider the original long URL:
Original URL: https://www.example.com/some/very/long/url/that/needs/to/be/shortened

1. Generate a Hash for the Original URL

We'll use SHA-256 to hash the original URL, which produces a 64-character hexadecimal string.
original_url = "https://www.example.com/some/very/long/url/that/needs/to/be/shortened"
hash of URL = b7e23e62ec2ac4db238b3955f8bdb9e7dbe2cfd5c430f2c56f47020e54cd93b9

2. Truncate the Hash to a Shorter Length

Since the hashed value is long (64 characters), we truncate it to a shorter length (let's say 8 characters) to create a compact short URL.
shortened hash : b7e23e62

3. Convert the Truncated Hash to Base 10 (Decimal)

The truncated hash (b7e23e62) is in hexadecimal (Base 16), so we need to convert it into Base 10 (decimal).
base 10 decimal value = 30642050937830457017007473131008547234

4. Convert the Base 10 Number to Base 62

Now that we have the decimal (Base 10) number, we will convert it to Base 62.
Base 62 encoding uses 62 characters: A-Z, a-z, 0-9.
The encoding process involves repeatedly dividing the number by 62 and recording the remainders to build the Base 62 string.
Base62-encoded short value = BfJ3HrsV

5. Store the Mapping in a Key-Value Database

Now, we store the mapping of the generated short URL (BfJ3HrsV) to the original URL in a key-value database.
Original url : https://www.example.com/some/very/long/url/that/needs/to/be/shortened
Short url : http://short.ly/BfJ3HrsV

6. Redirect to the Original URL
• When a user accesses a short URL, this service is responsible for redirecting the user to the original URL.
• The Service Layer queries the database to retrieve the original URL associated with the short URL.
• Once the long URL is retrieved, the service issues an HTTP redirect response, sending the user to the original URL.
• To reduce database load and improve latency, frequently accessed short URLs can be cached in an in-memory store like Redis.
• The Redirection Service should first check the cache before querying the database.

Note: If a collision occurs, we can handle it by rehashing or appending random strings or timestamp during hasing, ensuring no two URLs map to the same short URL.

---

6. Handling Expiration and Deletion

For URLs with expiration:

• Store expiration dates in the database.
• Implement a background job to remove expired URLs periodically.
• Return appropriate responses for expired URLs.

---

7. Security Considerations

• Implement rate limiting to prevent abuse.
• Validate input URLs to prevent malicious content.
• Use HTTPS to secure data in transit.
• Monitor for phishing attempts and misuse.

---

8. Scalability and Fault Tolerance

• Employ load balancers to distribute traffic.
• Use distributed databases and caching systems.
• Implement replication and failover mechanisms.
• Design stateless application servers to allow horizontal scaling.

---

9. Conclusion

Designing a URL shortener involves careful consideration of scalability, performance, and security. By addressing these aspects, the service can efficiently handle high traffic volumes while providing a reliable user experience.