Load Balancers are mandatory when you need to design a scalable backend system. In simple terms, Load Balancers are responsible to route user requests to a server that can service that request in the least amount of time and thus reducing latency.
Without Load Balancer, requests can choke a server and can result in cascading failures. Load Balancers achieve this using a series of algorithms that can be configured manually or automatically.
Algorithms such as:
Round-robin scheduling: This is the most used and simple algorithm. In this algorithm, each request is forwarded to a server in the pool in a repeating sequential manner.
Weighted round-robin: Extended from Round Robin, this algorithm is useful in cases when you have a pool of servers where some servers are better than others in terms of configurations like RAM, CPU or Storage. You can assign more weight to such servers which will result in more requests being handled by such servers.
Least connections: Sometimes request count is not the right metric to judge load on a particular server. There can be heavy requests like reports and analytics related APIs which can take more time than average requests. In such cases routing requests to servers with the least number of active connections will result in reduced load and latency.
Least response time: Considering server size and types of requests, response times of servers can be calculated by Load Balancer and then new requests are routed to a server with the least response time
IP hash: Depending on business use cases, some users or users of selected regions can get different levels of performance from an app. In such cases, hashed IP is used to route requests to the right set of servers.
Additional features can be enabled on a Load Balancer too which can help in offloading API Gateway or remove the need for it completely.
Features such as:
Health checking: Load Balancers use the heartbeat protocol to monitor the health and therefore, reliability of end-servers. If some unlucky server is dead, it is removed from the load balancer’s pool until it is revived again.
TLS termination: Load Balancers reduce the burden on servers by handling TLS termination i.e decrypt the encrypted request and forward it to servers.
Predictive analytics: Load Balancers can predict traffic patterns through analytics performed over traffic passing through them or using statistics of traffic obtained over time.
Reduced human intervention: Features like Health Checking and Predictive analytics can help in automating Load balancers to add/replace servers according to traffic patterns and server health.
Service discovery: Load Balancers can be configured to forward requests to appropriate hosting servers by fetching them from the service registry.
Security: Load Balancers can also help in basic security handling like denial-of-service (DoS) at different layers of the OSI model (layers 3, 4, and 7).