Challenges in Microservices

Q1. How Do Services Locate Each Other Inside a Network?

Each instance of a microservice exposes a remote API with its own host and port. However, these endpoint URLs can be dynamic and may change frequently. How do other microservices or clients discover and invoke these services without hardcoding the details?

Q2. How Do New Service Instances Enter the Network?

When a microservice instance fails, new instances are spun up to ensure constant availability. These new instances may have different IP addresses or ports. How can these new service instances seamlessly join the network and start serving requests without manual intervention?

Q3. How Is Load Balancing and Information Sharing Managed?

With multiple instances of a microservice running, how do we ensure proper load balancing between them? Additionally, how is information about specific service instances shared across the network to enable efficient communication and invocation between microservices?

Solutions:

The challenges of locating services, integrating new service instances, and ensuring load balancing in microservices can be addressed using the following concepts:

How service communication happens in Traditional apps ?

How Traditional Load Balancers Operate

Traditional load balancers distribute incoming client requests across a pool of backend servers to optimize resource utilization and enhance application performance. They operate on the basis of algorithms and rules to determine which server receives the next request.

Basic Architecture

Traditional load balancers use a primary-secondary setup, with the primary load balancer handling all incoming traffic. The secondary load balancer acts as a failover mechanism in case the primary fails. The load balancer sits between the client and the backend servers, directing requests based on its load balancing algorithms.

Load Balancing Algorithms

The traditional load balancers use different algorithms to distribute the incoming requests:

• Round Robin: Distributes requests equally among all available servers. Each server is selected in turn, ensuring an even distribution of load. This is the simplest method but may not always be the most effective.
• Least Connections: Assigns requests to the server with the fewest active connections. This ensures that no server is overwhelmed by too many connections, which can be particularly useful in environments where connections vary widely.
• IP Hash: Routes requests based on the client's IP address. By hashing the IP address, it ensures consistent routing for repeated requests from the same client. This can help improve cache effectiveness but may not always balance the load optimally.
• Weighted Round Robin: Assigns weights to servers based on their capacities, directing more traffic to servers that can handle it better. This algorithm is useful for balancing load according to server capacities.

Handling Failures and Redundancy

Traditional load balancers handle server failures through redundancy. If a server fails, the load balancer will simply bypass it and continue to distribute traffic to the remaining servers. The secondary load balancer will take over in the event of the primary failure, ensuring high availability of the application.

Failover Mechanism: The secondary load balancer checks the health of the primary periodically. If the primary fails, the secondary assumes control, allowing the application to remain available to users without interruption.

Limitations of Traditional Load Balancers

• Scalability Issues: Traditional load balancers are often bound by hardware limitations. Scaling them horizontally (adding more load balancers) to handle increased traffic can be complex and costly.
• Static Configuration: They require manual setup and do not support dynamic changes in the environment. For instance, if a new server is added, it needs to be manually configured into the load balancer, which can be prone to human error.
• Limited Visibility: Traditional load balancers do not provide insight into the health or status of individual servers. They do not monitor detailed metrics, such as CPU usage or memory load, which can be crucial for troubleshooting and optimization.
• Complex Configuration for Cloud Environments: Adapting traditional load balancers for cloud environments often requires significant modifications or additional services like Elastic Load Balancers (ELB) to bridge the gap between on-premises infrastructure and cloud resources.

Solutions for Cloud-Native Applications

Cloud-native applications utilize modern tools like Service Discovery and Load Balancing as part of their architecture. These tools offer dynamic, automated methods for managing services and load distribution.

Service Discovery

Service Discovery enables applications to discover and communicate with other services in a dynamic environment. It reduces the need for hard-coded IP addresses and simplifies the process of adding new services.

How it Works: Services register themselves with the service registry upon startup, and the registry keeps track of all available services. Other services can query the registry to find the available instances of a specific service.

public class EurekaServiceRegistry implements ServiceRegistry {
    private final EurekaClient eurekaClient;

    public EurekaServiceRegistry(EurekaClient eurekaClient) {
        this.eurekaClient = eurekaClient;
    }

    @Override
    public List getAllServices() {
        return eurekaClient.getInstancesById("my-service-id");
    }
}

Configuration: In Spring Cloud, enable service discovery with @EnableEurekaClient for Eureka, or @EnableDiscoveryClient for Consul.

Service Registry

Service Registry keeps a catalog of services available in the system. This ensures services can register and deregister automatically, making it easier to handle failures and scale services dynamically.

How it Works: Services register themselves with the service registry, and the registry provides a central view of all service instances. This setup allows services to be self-healing, where a failed instance can be automatically removed from the registry and replaced by a new instance.

public class EurekaServiceRegistry implements ServiceRegistry {
    private final EurekaClient eurekaClient;

    public EurekaServiceRegistry(EurekaClient eurekaClient) {
        this.eurekaClient = eurekaClient;
    }

    @Override
    public void register(ServiceInstance serviceInstance) {
        eurekaClient.registerInstance(serviceInstance);
    }

    @Override
    public void deregister(ServiceInstance serviceInstance) {
        eurekaClient.deregisterInstance(serviceInstance.getServiceId(), serviceInstance.getInstanceId());
    }
}

Load Balancing in Cloud-Native Applications

Load balancing in cloud-native applications can be done either at the client-side or server-side.

Client-Side Service Discovery

Client-side load balancing involves the client application making decisions about which service instance to call. The client consults the service registry to discover the available instances of a service, chooses one, and then sends requests directly to that service.

Key aspects of client side service discovery

1. Service Registration: Client applications register themselves with the service registry upon startup. They provide essential information about their location, such as IP address, port, and metadata, which helps identify and categorize the service.
2. Service Discovery: When a client application needs to communicate with a specific service, it queries the service registry for available instances of that service. The registry responds with the necessary information, such as IP addresses and connection details.
3. Load Balancing: Client-side service discovery often involves load balancing to distribute the workload across multiple service instances. The client application can implement a load- balancing strategy to select a specific instance based on factors like round-robin, weighted distribution, or latency.

Spring cloud support for client side discovery:

1. Eureka Server: Spring Cloud Netflix's Eureka service which will act as a service discovery agent
2. Spring Cloud Load Balancer library for client-side load balancing
3. Feign Client: Netflix Feign client to look up for a service b/w microservices

Steps to Build a Eureka Server Application using Spring Cloud Netflix's Eureka

1. Set up a New Spring Boot Project

Start by creating a new Spring Boot project using your preferred IDE or by using Spring Initializr (https://start.spring.io/). Include the spring-cloud-starter-netflix-eureka-server Maven dependency in your project.

2. Configure the Properties

    server:
      port: 8070
    eureka:
      instance:
        hostname: localhost
      client:
        fetchRegistry: false
        registerWithEureka: false
      serviceUrl:
        defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/
    

3. Add the Eureka Server Annotation

In the main class of your project, annotate it with @EnableEurekaServer. This annotation configures the application to act as a Eureka Server.

4. Build and Run the Eureka Server

Build your project and run it as a Spring Boot application. Open a web browser and navigate to http://localhost:8070. You should see the Eureka Server dashboard, which displays information about registered service instances.

eureka:
  instance:
    preferIpAddress: true
  client:
    registerWithEureka: true
    fetchRegistry: true
  serviceUrl:
    defaultZone: "http://localhost:8070/eureka/"

Server-Side Service Discovery

In server-side load balancing, a central component (like a load balancer) decides which service instance to forward a request to. The client application does not need to be aware of the specific instances or their details; it simply sends requests to a load balancer.

How it Works: The load balancer chooses an appropriate instance to forward the request based on factors like health, load, or latency.