Load Balancing
Learn how load balancers distribute traffic, protect availability, perform health checks, and shape request routing.
What you will be able to do
A load balancer sits between clients and service instances. It spreads requests, removes unhealthy instances, and gives the system one stable entry point while the backend changes.
What Load Balancers Do
Load balancing is not only round-robin traffic distribution. Production load balancers route by protocol, path, host, health, region, weight, and sometimes user affinity.
| Responsibility | Why it matters |
|---|---|
| Traffic distribution | Prevents one healthy instance from taking all traffic. |
| Health checks | Removes broken instances from rotation. |
| TLS termination | Centralizes HTTPS handling where appropriate. |
| Routing | Sends /api, /static, or tenant traffic to correct targets. |
| Failover | Keeps service reachable when an instance or zone fails. |
| Connection handling | Protects services from connection spikes and slow clients. |
L4 vs L7 Load Balancing
Layer 4 load balancing works at the TCP/UDP connection level. Layer 7 load balancing understands HTTP details such as host, path, headers, and sometimes cookies.
| Type | Sees | Good for | Example |
|---|---|---|---|
| L4 | IP, port, TCP/UDP | Very fast network-level routing | Database proxy or TCP service. |
| L7 | HTTP path, host, headers | API/web routing and policies | Route /api to services and /assets to static backend. |
| Global load balancing | Region and edge health | Multi-region entry point | Route users to nearest healthy region. |
Scenario: Video Learning Platform
A learning platform serves course pages, API requests, image assets, video metadata, and assessment submissions. These flows should not all route the same way.
| Traffic | Routing choice | Reason |
|---|---|---|
| Course pages | Web frontend service/CDN | Read-heavy, cacheable content. |
| Assessment submit | API service with stricter timeouts | Write path and scoring workflow. |
| Images/static assets | CDN/object storage | Avoid hitting application servers. |
| Admin APIs | Separate route and auth policy | Higher privilege and lower traffic. |
| Health checks | Dedicated lightweight endpoint | Avoid expensive checks that overload service. |
Design principle
A load balancer can hide failed instances, but it cannot fix a bad health check, overloaded dependency, or stateful server that cannot receive any request.
Health Checks
Health checks decide whether an instance receives traffic. A poor health check can route users to broken servers or remove healthy servers during a temporary dependency issue.
| Check type | What it means | Risk |
|---|---|---|
| Liveness | Is the process running? | Too shallow for dependency failures. |
| Readiness | Can this instance receive traffic? | Should fail during startup or severe local issues. |
| Deep dependency check | Can key dependencies be reached? | May eject all instances if shared dependency is down. |
| Synthetic transaction | Can a real flow work? | Useful but expensive and should not run too frequently. |
Cloud Load Balancing
| Need | AWS | GCP | Azure |
|---|---|---|---|
| HTTP/L7 | Application Load Balancer/API Gateway | Cloud Load Balancing/API Gateway | Application Gateway/API Management |
| TCP/L4 | Network Load Balancer | Network Load Balancing | Azure Load Balancer |
| Global edge routing | CloudFront/Global Accelerator/Route 53 | Cloud Load Balancing/Cloud CDN/Cloud DNS | Azure Front Door/Traffic Manager |
| Kubernetes ingress | AWS Load Balancer Controller | GKE Ingress/Gateway | Application Gateway Ingress Controller |
| Service mesh routing | App Mesh/Istio on EKS | Anthos Service Mesh/Istio | Open Service Mesh/Istio on AKS |
Beginner Mistakes
- Forgetting health checks when adding multiple instances.
- Using a deep health check that removes every instance when a shared database has a temporary issue.
- Keeping user session only in instance memory and then load balancing across instances.
- Sending static assets through API servers instead of CDN/object storage.
- Assuming load balancing solves database or external provider bottlenecks.
Guided Practice
Practice task
Design routing for a news website with web pages, login API, image assets, and admin routes. Choose which traffic should go through app servers and which should use CDN or separate policies.
Sample Answer
| Traffic | Routing |
|---|---|
| Public article pages | CDN plus frontend/app origin depending freshness. |
| Images and static assets | CDN backed by object storage. |
| Login API | L7 route to auth/API service with rate limiting. |
| Admin routes | Separate host/path with stricter auth and allowlist if needed. |
| Health checks | Readiness endpoint that confirms app can accept traffic without expensive full dependency checks. |
Before You Continue
- You should know what load balancers do.
- You should understand L4, L7, and global routing at a high level.
- You should understand health check trade-offs.
- You are ready to learn why stateless services make load balancing and scaling safer.