Lesson 13Beginner1h 14mAssessment-backed

Vertical vs Horizontal Scaling

Learn when to make one machine bigger, when to add more machines, and what scaling changes about state, cost, reliability, and operations.

What you will be able to do

Explain vertical and horizontal scaling in practical terms.

Identify when scale pressure comes from CPU, memory, I/O, database, network, or dependencies.

Understand why horizontal scaling usually requires stateless services and load balancing.

Connect scaling choices to AWS, GCP, Azure, Kubernetes, and autoscaling.

Avoid scaling before measuring the bottleneck.

Scaling means increasing a system's ability to handle load. The professional question is not how do we scale everything, but what exactly is saturated and what is the simplest safe way to relieve it.

Two Ways to Scale

Vertical scaling makes one machine bigger. Horizontal scaling adds more machines or instances. Vertical scaling is often simpler; horizontal scaling usually gives better availability and growth headroom but requires better architecture discipline.

Vertical scaling compared with horizontal scaling — Vertical scaling increases capacity of one node. Horizontal scaling distributes traffic across many nodes.

Approach	How it works	Use when	Risk
Vertical scaling	More CPU, memory, disk, or network on one node	Early stage, simple bottleneck, database needs quick relief	Hard limit, bigger failure blast radius, cost jumps.
Horizontal scaling	Add more instances behind routing/load balancing	Stateless services, read-heavy traffic, high availability	Requires load balancing, deployment automation, shared state design.
Specialized scaling	Scale a bottleneck layer only	Database, cache, workers, CDN, queue, search layer is saturated	Wrong bottleneck diagnosis wastes money.

Scenario: Ticket Launch

A ticketing site launches concert tickets at 10 AM. Traffic jumps from 100 requests/sec to 20,000 requests/sec. Scaling the web server alone will not save the system if inventory locking, payment calls, or database writes are the actual bottleneck.

Pressure	Symptom	Scaling response
CPU on API servers	High CPU, low dependency latency	Add API instances or bigger instances.
Database writes	Slow commits and lock waits	Reduce write contention, queue non-critical work, partition carefully.
Hot product page reads	Same page read repeatedly	Cache page/data and use CDN.
Payment provider latency	Threads blocked waiting outside system	Timeouts, bulkheads, async confirmation where product allows.
Traffic spike fairness	Some clients overwhelm others	Rate limiting, waiting room, queueing, backpressure.

Autoscaling Is Not Magic

Autoscaling reacts to signals such as CPU, memory, request count, queue depth, or custom metrics. It still needs safe deployments, warm-up time, capacity limits, and bottleneck-aware metrics.

Signal	Good for	Caution
CPU	CPU-bound services	May miss database or network bottlenecks.
Request count	Web/API scaling	Not all requests cost the same.
Queue depth	Worker scaling	Needs max concurrency and DLQ monitoring.
Memory	Memory-heavy services	Memory leaks can cause runaway scaling.
Latency	User experience pressure	Scaling may not fix slow dependency.

Design principle

Scale the bottleneck, not the diagram. Measure saturation, identify the constrained resource, then choose the least complex scaling move that protects the user flow.

Cloud Scaling Mapping

Need	AWS	GCP	Azure
VM autoscaling	EC2 Auto Scaling	Managed Instance Groups	Virtual Machine Scale Sets
Container autoscaling	ECS Service Auto Scaling/EKS HPA	Cloud Run autoscaling/GKE HPA	Container Apps scaling/AKS HPA
Serverless scaling	Lambda concurrency	Cloud Functions/Cloud Run concurrency	Azure Functions scale controller
Database scaling	RDS/Aurora read replicas, DynamoDB capacity	Cloud SQL replicas, Spanner, Firestore	Azure SQL replicas, Cosmos DB RU/s
Queue-based worker scaling	SQS depth + ECS/Lambda	Pub/Sub/Cloud Tasks + Cloud Run	Service Bus + Functions/Container Apps

Beginner Mistakes

Scaling every service instead of finding the bottleneck.
Adding more API servers when the database is saturated.
Assuming autoscaling is instant.
Ignoring connection pools and dependency limits.
Scaling horizontally while keeping session state inside one server.

Guided Practice

Practice task

A course platform becomes slow during assessment submission. Name five possible bottlenecks and one metric that would confirm each.

Sample Answer

Possible bottleneck	Confirming metric
API CPU	High CPU and request queueing on API instances.
Database writes	High write latency, lock wait, or connection saturation.
Scoring worker	Growing queue depth and oldest message age.
External plagiarism provider	High dependency latency and timeout rate.
Cache/session store	High Redis latency or connection count.

Before You Continue

You should know vertical vs horizontal scaling.
You should identify bottlenecks before scaling.
You should understand autoscaling signals and limits.
You are ready to learn how load balancers distribute traffic across scaled services.

Transactions and Consistency Basics

Load Balancing